The Effect of Hyperparameter Optimization on the Estimation of Performance Metrics in Network Traffic Prediction using the Gradient Boosting Machine Model
Received: 7 December 2022 | Revised: 10 February 2023 | Accepted: 18 February 2023 | Online: 2 June 2023
Corresponding author: Machoke Mwita
Abstract
Information and Communication Technology (ICT) has changed the way we communicate and access information, resulting in the high generation of heterogeneous data. The amount of network traffic generated constantly increases in velocity, veracity, and volume as we enter the era of big data. Network traffic classification and intrusion detection are very important for the early detection and identification of unnecessary network traffic. The Machine Learning (ML) approach has recently entered the center stage in network traffic accurate classification. However, in most cases, it does not apply model hyperparameter optimization. In this study, gradient boosting machine prediction was used with different hyperparameter optimization configurations, such as interaction depth, tree number, learning rate, and sampling. Data were collected through an experimental setup by using the Sophos firewall and Cisco router data loggers. Data analysis was conducted with R software version 4.2.0 with Rstudio Integrated Development Environment. The dataset was split into two partitions, where 70% was used for training the model and 30% for testing. At a learning rate of 0.1, interaction depth of 14, and tree number of 2500, the model estimated the highest performance metrics with an accuracy of 0.93 and R of 0.87 compared to 0.90 and 0.85 before model optimization. The same configuration attained the minimum classification error of 0.07 than 0.10 before model optimization. After model tweaking, a method was developed for achieving improved accuracy, R square, mean decrease in Gini coefficients for more than 8 features, lower classification error, root mean square error, logarithmic loss, and mean square error in the model.
Keywords:
network traffic, machine learning, big data, data loggers, feature selection, gradient boosting machine predictionDownloads
References
M. Soysal and E. G. Schmidt, "Machine learning algorithms for accurate flow-based network traffic classification: Evaluation and comparison," Performance Evaluation, vol. 67, no. 6, pp. 451–467, Jun. 2010. DOI: https://doi.org/10.1016/j.peva.2010.01.001
J. J. Estevez-Pereira, D. Fernandez, and F. J. Novoa, "Network Anomaly Detection Using Machine Learning Techniques," Proceedings, vol. 54, no. 1, 2020, Art. no. 8. DOI: https://doi.org/10.3390/proceedings2020054008
G. Ali, M. Ally Dida, and A. Elikana Sam, "Two-Factor Authentication Scheme for Mobile Money: A Review of Threat Models and Countermeasures," Future Internet, vol. 12, no. 10, Oct. 2020, Art. no. 160. DOI: https://doi.org/10.3390/fi12100160
M. Lotfollahi, M. Jafari Siavoshani, R. Shirali Hossein Zade, and M. Saberian, "Deep packet: a novel approach for encrypted traffic classification using deep learning," Soft Computing, vol. 24, no. 3, pp. 1999–2012, Feb. 2020. DOI: https://doi.org/10.1007/s00500-019-04030-2
K. Demertzis, K. Tsiknas, D. Takezis, C. Skianis, and L. Iliadis, "Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework," Electronics, vol. 10, no. 7, Jan. 2021, Art. no. 781. DOI: https://doi.org/10.3390/electronics10070781
G. S. Oreku, F. J. Mtenzi, and C. A. Shoniregun, "Traffic classification and packet detections to facilitate networks security," International Journal of Internet Technology and Secured Transactions, vol. 3, no. 3, pp. 240–252, Jan. 2011. DOI: https://doi.org/10.1504/IJITST.2011.041294
Q. Bi, K. E. Goodman, J. Kaminsky, and J. Lessler, "What is Machine Learning? A Primer for the Epidemiologist," American Journal of Epidemiology, vol. 188, no. 12, pp. 2222–2239, Dec. 2019. DOI: https://doi.org/10.1093/aje/kwz189
I. L. Cherif and A. Kortebi, "On using eXtreme Gradient Boosting (XGBoost) Machine Learning algorithm for Home Network Traffic Classification," in Wireless Days, Manchester, UK, Apr. 2019, pp. 1–6. DOI: https://doi.org/10.1109/WD.2019.8734193
S. Ageev, V. Karetnikov, E. Ol’khovik, and A. Privalov, "Adaptive method of detecting traffic anomalies in high-speed multi-service communication networks," E3S Web of Conferences, vol. 157, 2020, Art. no. 04027. DOI: https://doi.org/10.1051/e3sconf/202015704027
J. K. Mazima, A. Johnson, E. Manasseh, and S. Kaijage, "Stochastic Modeling Technology for Grain Crops Storage Application : Review," International Journal of Artificial Intelligence & Applications, vol. 7, no. 6, pp. 27–42, Nov. 2016. DOI: https://doi.org/10.5121/ijaia.2016.7603
M. Singh, G. Srivastava, and P. Kumar, "Internet Traffic Classification Using Machine Learning," International Journal of Database Theory and Application, vol. 9, pp. 45–54, Dec. 2016. DOI: https://doi.org/10.14257/ijdta.2016.9.12.05
R. Samrin and D. Vasumathi, "Hybrid Weighted K-Means Clustering and Artificial Neural Network for an Anomaly-Based Network Intrusion Detection System," Journal of Intelligent Systems, vol. 27, no. 2, pp. 135–147, Apr. 2018. DOI: https://doi.org/10.1515/jisys-2016-0105
F. Dehghani, N. Movahhedinia, M. R. Khayyambashi, and S. Kianian, "Real-Time Traffic Classification Based on Statistical and Payload Content Features," in 2nd International Workshop on Intelligent Systems and Applications, Wuhan, China, Dec. 2010, pp. 1–4. DOI: https://doi.org/10.1109/IWISA.2010.5473467
J. Yang, Y.-X. Wang, Y.-Y. Qiao, X.-X. Zhao, F. Liu, and G. Cheng, "On Evaluating Multi-class Network Traffic Classifiers Based on AUC," Wireless Personal Communications, vol. 83, no. 3, pp. 1731–1750, Aug. 2015. DOI: https://doi.org/10.1007/s11277-015-2473-4
A. M. Sadeghzadeh, S. Shiravi, and R. Jalili, "Adversarial Network Traffic: Towards Evaluating the Robustness of Deep-Learning-Based Network Traffic Classification," IEEE Transactions on Network and Service Management, vol. 18, no. 2, pp. 1962–1976, Jun. 2021. DOI: https://doi.org/10.1109/TNSM.2021.3052888
A. Gouveia and M. Correia, "Deep Learning for Network Intrusion Detection: An Empirical Assessment," in Recent Advances in Security, Privacy, and Trust for Internet of Things (IoT) and Cyber-Physical Systems (CPS), 1st Edition., Boca Raton, FL, USA: Chapman and Hall, 2020, pp. 191–206. DOI: https://doi.org/10.1201/9780429270567-8
"R Interface for the ‘H2O’ Scalable Machine Learning Platform," H2O. https://docs.h2o.ai/h2o/latest-stable/h2o-r/docs/index.html.
"R: The R Project for Statistical Computing." https://www.r-project.org/.
I. Satti, A. Elkarim, J. Agbinya, A. Hussein, and I. Satti, "Parallel SVM Based Classification Technique on big data: HPC center in Sudan," Australian Journal of Basic and Applied Sciences, vol. 14, pp. 1–14, Apr. 2020.
A. Malik et al., "Deep learning versus gradient boosting machine for pan evaporation prediction," Engineering Applications of Computational Fluid Mechanics, vol. 16, no. 1, pp. 570–587, Dec. 2022. DOI: https://doi.org/10.1080/19942060.2022.2027273
D. Cook, Practical Machine Learning with H2O: Powerful, Scalable Techniques for Deep Learning and AI, 1st ed. O’Reilly Media, 2016.
J. H. Friedman, "Stochastic gradient boosting," Computational Statistics & Data Analysis, vol. 38, no. 4, pp. 367–378, Feb. 2002. DOI: https://doi.org/10.1016/S0167-9473(01)00065-2
A. Natekin and A. Knoll, "Gradient Boosting Machines, A Tutorial," Frontiers in Neurorobotics, vol. 7, 2013, Art. no. 21. DOI: https://doi.org/10.3389/fnbot.2013.00021
E. A. Freeman, G. G. Moisen, J. W. Coulston, and B. T. Wilson, "Random forests and stochastic gradient boosting for predicting tree canopy cover: comparing tuning processes and model performance," Canadian Journal of Forest Research, vol. 46, no. 3, pp. 323–339, Mar. 2016. DOI: https://doi.org/10.1139/cjfr-2014-0562
M. Machoke, J. Mbelwa, J. Agbinya, and A. E. Sam, "Performance Comparison of Ensemble Learning and Supervised Algorithms in Classifying Multi-label Network Traffic Flow," Engineering, Technology & Applied Science Research, vol. 12, no. 3, pp. 8667–8674, Jun. 2022. DOI: https://doi.org/10.48084/etasr.4852
M. Alqahtani, A. Gumaei, H. Mathkour, and M. Maher Ben Ismail, "A Genetic-Based Extreme Gradient Boosting Model for Detecting Intrusions in Wireless Sensor Networks," Sensors, vol. 19, no. 20, Jan. 2019, Art. no. 4383. DOI: https://doi.org/10.3390/s19204383
J. J. Allaire, "RStudio: Integrated Development Environment for R," presented at the The R User Conference 2011, Coventry, UK, 2011.
C.-W. Wu, H.-L. Shen, C.-J. Lu, S.-H. Chen, and H.-Y. Chen, "Comparison of Different Machine Learning Classifiers for Glaucoma Diagnosis Based on Spectralis OCT," Diagnostics, vol. 11, no. 9, Sep. 2021, Art. no. 1718. DOI: https://doi.org/10.3390/diagnostics11091718
J. H. Friedman, "Greedy Function Approximation: A Gradient Boosting Machine," The Annals of Statistics, vol. 29, no. 5, pp. 1189–1232, 2001. DOI: https://doi.org/10.1214/aos/1013203451
R. Andersson, Classification of Video Traffic : An Evaluation of Video Traffic Classification using Random Forests and Gradient Boosted Trees. Karlstad, Sweden: Karlstad University, 2017.
H. Wan, "Gradient Descent Boosting: Convergence and Algorithm," 2017, [Online]. Available: https://courses.engr.illinois.edu/ece543/sp2017/projects/Haohua%20Wan.pdf.
D. Preethi and N. Khare, "Sparse auto encoder driven support vector regression based deep learning model for predicting network intrusions," Peer-to-Peer Networking and Applications, vol. 14, no. 4, pp. 2419–2429, Jul. 2021. DOI: https://doi.org/10.1007/s12083-020-00986-3
C. Pan, Y. Wang, H. Shi, J. Shi, and R. Cai, "Network Traffic Prediction Incorporating Prior Knowledge for an Intelligent Network," Sensors, vol. 22, no. 7, Jan. 2022, Art. no. 2674. DOI: https://doi.org/10.3390/s22072674
L.-H. Chang, Tsung-Han Lee, Hung-Chi Chu, and Cheng-Wei Su, "Application-Based Online Traffic Classification with Deep Learning Models on SDN Networks," Advances in Technology Innovation, vol. 5, no. 4, pp. 216–229, Jul. 2020. DOI: https://doi.org/10.46604/aiti.2020.4286
R. Dangi, A. Jadhav, G. Choudhary, N. Dragoni, M. K. Mishra, and P. Lalwani, "ML-Based 5G Network Slicing Security: A Comprehensive Survey," Future Internet, vol. 14, no. 4, Apr. 2022, Art. no. 116. DOI: https://doi.org/10.3390/fi14040116
Z. A. Qazi, J. Lee, T. Jin, G. Bellala, M. Arndt, and G. Noubir, "Application-awareness in SDN," in ACM SIGCOMM 2013 conference on SIGCOMM, New York, NY, USA, Aug. 2013, pp. 487–488. DOI: https://doi.org/10.1145/2486001.2491700
S. S. Alzahrani, "Data Mining Regarding Cyberbullying in the Arabic Language on Instagram Using KNIME and Orange Tools," Engineering, Technology & Applied Science Research, vol. 12, no. 5, pp. 9364–9371, Oct. 2022. DOI: https://doi.org/10.48084/etasr.5184
Q. H. Do, T. T. H. Doan, T. V. A. Nguyen, N. T. Duong, and V. V. Linh, "Prediction of Data Traffic in Telecom Networks based on Deep Neural Networks," Journal of Computer Science, vol. 16, no. 9, pp. 1268–1277, Sep. 2020. DOI: https://doi.org/10.3844/jcssp.2020.1268.1277
S. Mahajan, R. Harikrishnan, and K. Kotecha, "Prediction of Network Traffic in Wireless Mesh Networks Using Hybrid Deep Learning Model," IEEE Access, vol. 10, pp. 7003–7015, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3140646
K. P. Rusna and V. G. Kalpana, "Using Artificial Neural Networks for the Prediction of the Compressive Strength of Geopolymer Fly Ash," Engineering, Technology & Applied Science Research, vol. 12, no. 5, pp. 9120–9125, Oct. 2022. DOI: https://doi.org/10.48084/etasr.5185
Downloads
How to Cite
License
Copyright (c) 2023 Machoke Mwita, Jimmy Mbelwa, Johnson Agbinya, Anael Elikana Sam
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
