A Cluster-based Undersampling Technique for Multiclass Skewed Datasets
Received: 12 March 2023 | Revised: 29 March 2023 and 4 April 2023 | Accepted: 8 April 2023 | Online: 2 June 2023
Corresponding author: Rose Mary Mathew
Abstract
Imbalanced data classification is a demanding issue in data mining and machine learning. Models that learn with imbalanced input generate feeble performance in the minority class. Resampling methods can handle this issue and balance the skewed dataset. Cluster-based Undersampling (CUS) and Near-Miss (NM) techniques are widely used in imbalanced learning. However, these methods suffer from some serious flaws. CUS averts the impact of the distance factor on instances over the majority class. Near-miss method discards the inter-class data within the majority of class elements. To overcome these flaws, this study has come up with an undersampling technique called Adaptive K-means Clustering Undersampling (AKCUS). The proposed technique blends the distance factor and clustering over the majority class. The performance of the proposed method was analyzed with the aid of an experimental study. Three multiminority datasets with different imbalance ratios were selected and the models were created using K-Nearest Neighbor (kNN), Decision Tree (DT), and Random Forest (RF) classifiers. The experimental results show that AKCUS can attain better efficacy than the benchmark methods over multiminority datasets with high imbalance ratios.
Keywords:
K-means clustering, multiclass, resampling, skewed, undersamplingDownloads
References
G. Haixiang, L. Yijing, J. Shang, G. Mingyun, H. Yuanyue, and G. Bing, "Learning from class-imbalanced data: Review of methods and applications," Expert Systems with Applications, vol. 73, pp. 220–239, May 2017. DOI: https://doi.org/10.1016/j.eswa.2016.12.035
S. Tahzeeb and S. Hasan, "A Neural Network-Based Multi-Label Classifier for Protein Function Prediction," Engineering, Technology & Applied Science Research, vol. 12, no. 1, pp. 7974–7981, Feb. 2022. DOI: https://doi.org/10.48084/etasr.4597
W. M. S. Yafooz, E. A. Hizam, and W. A. Alromema, "Arabic Sentiment Analysis on Chewing Khat Leaves using Machine Learning and Ensemble Methods," Engineering, Technology & Applied Science Research, vol. 11, no. 2, pp. 6845–6848, Apr. 2021. DOI: https://doi.org/10.48084/etasr.4026
F. Belloum, L. Houichi, and M. Kherouf, "The Performance of Spectral Clustering Algorithms on Water Distribution Networks: Further Evidence," Engineering, Technology & Applied Science Research, vol. 12, no. 4, pp. 9056–9062, Aug. 2022. DOI: https://doi.org/10.48084/etasr.5116
S. Bhatia, "Adaptive K-Means Clustering," in Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference, Miami Beach, FL, USA, Jan. 2004.
I. Tomek, "Two Modifications of CNN," IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-6, no. 11, pp. 769–772, Aug. 1976. DOI: https://doi.org/10.1109/TSMC.1976.4309452
D. L. Wilson, "Asymptotic Properties of Nearest Neighbor Rules Using Edited Data," IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-2, no. 3, pp. 408–421, Jul. 1972. DOI: https://doi.org/10.1109/TSMC.1972.4309137
P. Hart, "The condensed nearest neighbor rule (Corresp.)," IEEE Transactions on Information Theory, vol. 14, no. 3, pp. 515–516, Feb. 1968. DOI: https://doi.org/10.1109/TIT.1968.1054155
J. Zhang and I. Mani, "kNN approach to unbalanced data distributions: a case study involving information extraction," presented at the ICML, Washington DC, USA, 2003.
R. Barandela, R. M. Valdovinos, and J. S. Sánchez, "New Applications of Ensembles of Classifiers," Pattern Analysis & Applications, vol. 6, no. 3, pp. 245–256, Dec. 2003. DOI: https://doi.org/10.1007/s10044-003-0192-z
A. Anand, G. Pugalenthi, G. B. Fogel, and P. N. Suganthan, "An approach for classification of highly imbalanced data using weighting and undersampling," Amino Acids, vol. 39, no. 5, pp. 1385–1391, Nov. 2010. DOI: https://doi.org/10.1007/s00726-010-0595-2
M. R. Smith, T. Martinez, and C. Giraud-Carrier, "An instance level analysis of data complexity," Machine Learning, vol. 95, no. 2, pp. 225–256, May 2014. DOI: https://doi.org/10.1007/s10994-013-5422-z
W. Liu, H. Zhang, Z. Ding, Q. Liu, and C. Zhu, "A comprehensive active learning method for multiclass imbalanced data streams with concept drift," Knowledge-Based Systems, vol. 215, Mar. 2021, Art. no. 106778. DOI: https://doi.org/10.1016/j.knosys.2021.106778
W. Lu, Z. Li, and J. Chu, "Adaptive Ensemble Undersampling-Boost: A novel learning framework for imbalanced data," Journal of Systems and Software, vol. 132, pp. 272–282, Oct. 2017. DOI: https://doi.org/10.1016/j.jss.2017.07.006
M. Galar, A. Fernández, E. Barrenechea, and F. Herrera, "EUSBoost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling," Pattern Recognition, vol. 46, no. 12, pp. 3460–3471, Dec. 2013. DOI: https://doi.org/10.1016/j.patcog.2013.05.006
X.-Y. Liu, J. Wu, and Z.-H. Zhou, "Exploratory Undersampling for Class-Imbalance Learning," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39, no. 2, pp. 539–550, Apr. 2009. DOI: https://doi.org/10.1109/TSMCB.2008.2007853
S.-J. Yen and Y.-S. Lee, "Cluster-based under-sampling approaches for imbalanced data distributions," Expert Systems with Applications, vol. 36, no. 3, Part 1, pp. 5718–5727, Apr. 2009. DOI: https://doi.org/10.1016/j.eswa.2008.06.108
J. Alcala-Fdez et al., "KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework," Journal of Multiple-Valued Logic and Soft Computing, vol. 17, no. 2, pp. 255–287, Jan. 2010.
J. Alcalá-Fdez et al., "KEEL: a software tool to assess evolutionary algorithms for data mining problems," Soft Computing, vol. 13, no. 3, pp. 307–318, Feb. 2009. DOI: https://doi.org/10.1007/s00500-008-0323-y
M. O. Ojo and A. Zahid, "Improving Deep Learning Classifiers Performance via Preprocessing and Class Imbalance Approaches in a Plant Disease Detection Pipeline," Agronomy, vol. 13, no. 3, Mar. 2023, Art. no. 887. DOI: https://doi.org/10.3390/agronomy13030887
Downloads
How to Cite
License
Copyright (c) 2023 Rose Mary Mathew, Ranganathan Gunasundari
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
