Adaptive Isolation Forest and Cascading XGBoost–DNN Ensemble for High-Confidence Network Intrusion Detection with Explainable AI Integration

Khaldoon H.  Alhussayni

Authors

Khaldoon H. Alhussayni Cybersecurity Department, Information Technology College, University of Babylon, Babil, Iraq

Keywords:

Internet of Things, Adaptive Isolation Forest, XGBoost, Deep Neural Network, Network Intrusion Detection, Explainable AI

Abstract

The growing use of Internet of Things infrastructure underscores the importance of advanced threat detection systems that can handle evolving attack vectors. This paper presents a confidence-based framework for cascading information that employs self-adjusting irregularity recognition and supervised learning to enhance the accuracy and efficacy of IoT security. The proposed method starts with an adaptive isolation forest with dynamic thresholding to reduce false positives. Then, it uses a cascading architecture that includes an XGBoost for quick learning and a deep neural network for deep feature extraction. To make the model easy to understand and to build trust and transparency among analysts, the framework is based on the Explainable Artificial Intelligence (XAI) approach, SHAP. Evaluation of the CICIoT2023 dataset demonstrates the potential of the framework for deployment in a real-world IoT environment and closes the gap in anomaly detection accuracy, computational efficiency, and interpretability, achieving significant performance improvements at fixed thresholds and across standard accuracy metrics, including Precision, Recall, and F1 score. 0.9948, 0.9947, 0.9948, and 0.9945, respectively, while holding the comparative prediction time.

References

F. T. Liu, K. M. Ting, and Z. H. Zhou, “Isolation-based anomaly detection,” ACM Trans. Knowl. Discov. Data, vol. 6, Mar. 2012, doi: 10.1145/2133360.2133363.

M. S. Lakshmi, G. Rajavikram, V. Dattatreya, B. Swarna Jyothi, S. Patil, and M. Bhavsingh, “Evaluating the Isolation Forest Method for Anomaly Detection in Software-Defined Networking Security,” 2023.

M. M. Breunig, H. P. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying Density-Based Local Outliers,” in SIGMOD 2000 - Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, Inc, 2000, pp. 93–104. doi: 10.1145/342009.335388.

W. Chua et al., “Web Traffic Anomaly Detection Using Isolation Forest,” Informatics, vol. 11, Dec. 2024, doi: 10.3390/informatics11040083.

Y. Chabchoub, M. U. Togbe, A. Boly, and R. Chiky, “An In-Depth Study and Improvement of Isolation Forest,” IEEE Access, vol. 10, pp. 10219–10237, 2022, doi: 10.1109/ACCESS.2022.3144425.

Z. Liang, Y. G. Wang, W. Lu, and X. Cao, “Boosting Semi-Supervised Learning with Dual-Threshold Screening and Similarity Learning,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 20, Sep. 2024, doi: 10.1145/3672563.

W. Liu, W. Zeng, K. He, Y. Jiang, and J. He, “An Adaptive Unsupervised Learning Approach for Credit Card Fraud Detection,” in 12th International Conference on Learning Representations, ICLR 2024, International Conference on Learning Representations, ICLR, 2024.

G. Wei, X. Li, L. Huang, J. Nie, and Z. Wei, “Unsupervised domain adaptation via reliable pseudolabeling based memory module and dynamic distance threshold learning,” Knowl. Based. Syst., vol. 275, Sep. 2023, doi: 10.1016/j.knosys.2023.110667.

C. Chen et al., “Deep Learning on Computational-Resource-Limited Platforms: A Survey,” 2020, Hindawi Limited. doi: 10.1155/2020/8454327.

M. M. H. Shuvo, S. K. Islam, J. Cheng, and B. I. Morshed, “Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review,” Jan. 2023, Institute of Electrical and Electronics Engineers Inc. doi: 10.1109/JPROC.2022.3226481.

M. Vishwakarma and N. Kesswani, “StaEn-IDS: An Explainable Stacking Ensemble Deep Neural Network-Based Intrusion Detection System for IoT,” IEEE Access, vol. 13, pp. 109713–109728, 2025, doi: 10.1109/ACCESS.2025.3582391.

Y. Laraig, Y. Ben Maissa, S. Roy, P.-M. Tardif, and B. El bhiri, “A Feature-Aware Adaptive Ensemble Framework for IoT Intrusion Detection Systems,” Institute of Electrical and Electronics Engineers (IEEE), Dec. 2025, pp. 1–6. doi: 10.1109/wimob66857.2025.11257442.

T. Hasan and S. Tasnim, “Multidimensional Feature Learning Enhancement in IoT Intrusion Detection: An Adaptive Cost-Sensitive Autoencoder and Weighted Ensemble Approach,” in 2024 IEEE 10th World Forum on Internet of Things, WF-IoT 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 536–541. doi: 10.1109/WF-IoT62078.2024.10811174.

Y. Wang et al., “FREEMATCH: SELF-ADAPTIVE THRESHOLDING FOR SEMI-SUPERVISED LEARNING,” in 11th International Conference on Learning Representations, ICLR 2023, International Conference on Learning Representations, ICLR, 2023.

J. A. Simioni, E. K. Viegas, A. O. Santin, and E. de Matos, “An Energy-Efficient Intrusion Detection Offloading Based on DNN for Edge Computing,” IEEE Internet Things J., vol. 12, pp. 20326–20342, 2025, doi: 10.1109/JIOT.2025.3544060.

A. M. Alashjaee and F. Alqahtani, “Enhanced intrusion detection system IoT network security model by feed forward neural network and machine learning,” Sci. Rep., vol. 15, Dec. 2025, doi: 10.1038/s41598-025-20047-0.

H. Sharma, P. Kumar, and K. Sharma, “Deep Learning based Ensemble Model for Intrusion Detection in IoT Network,” in 2025 International Conference on Innovations in Intelligent Systems: Advancements in Computing, Communication, and Cybersecurity (ISAC3), 2025, pp. 1–6. doi: 10.1109/ISAC364032.2025.11156772.

T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, Aug. 2016, pp. 785–794. doi: 10.1145/2939672.2939785.

L. Deng and D. Yu, “Deep learning: Methods and applications,” 2013, Now Publishers Inc. doi: 10.1561/2000000039.

E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, “CICIoT2023: A real-time dataset and benchmark for large-scale attacks in IoT environment,” Sensors, vol. 23, no. 13, p. 5941, 2023.

N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002, doi: 10.1613/jair.953.