A Constrained Multi-Objective Learning Algorithm for Feed-Forward Neural Network Classifiers
This paper proposes a new approach to address the optimal design of a Feed-forward Neural Network (FNN) based classifier. The originality of the proposed methodology, called CMOA, lie in the use of a new constraint handling technique based on a self-adaptive penalty procedure in order to direct the entire search effort towards finding only Pareto optimal solutions that are acceptable. Neurons and connections of the FNN Classifier are dynamically built during the learning process. The approach includes differential evolution to create new individuals and then keeps only the non-dominated ones as the basis for the next generation. The designed FNN Classifier is applied to six binary classification benchmark problems, obtained from the UCI repository, and results indicated the advantages of the proposed approach over other existing multi-objective evolutionary neural networks classifiers reported recently in the literature.
Keywords:FNN Classifier, Constrained Multi-Objective Optimization, Pareto Dominance Criterion, Differential Evolution
C. Aggarwal, Data Classification: Algorithms and Applications, 1st Edition. Chapman and Hall/CRC, 2014
R. Ramli, H. Arof, F. Ibrahim, M. Y. I. Idris, A. S. M. Khairuddin, “Classification of Eyelid Position and Eyeball Movement Using EEG Signals”, Malaysian Journal of Computer Science. Vol. 28, No. 1, pp. pp. 28-45, 2015
S. M. A. Kalaiarasi, S. Gopala, C. Ali, T. Jason, “Artificial Neural Network Tree Approach In Data Mining”, Malaysian Journal of Computer Science, Vol. 20, No. 1, pp. 51-62, 2007 DOI: https://doi.org/10.22452/mjcs.vol20no1.5
V. Chaitali, B. Nikita, M. Darshana, “A survey on various classification techniques for clinical decision support system” International Journal of Computer Applications, Vol. 116, No. 23, pp. 14–17, 2015 DOI: https://doi.org/10.5120/20498-2369
A. Azzini, A. G. B. Tettamanzi, “Evolutionary ANNS: a state of the art survey”, Intelligenza Artificiale,Vol. 5, pp. 19–35, 2011 DOI: https://doi.org/10.3233/IA-2011-0002
Y. Jin, B. Sendhoff, “Pareto-based multi-objective machine learning: An overview and case studies”, IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews, Vol. 38, No. 3, pp. 397–415, 2008 DOI: https://doi.org/10.1109/TSMCC.2008.919172
T. Fawcett, “An introduction to roc analysis”, Pattern Recognition Letters, Vol. 27, No. 3, pp. 861–874, 2006 DOI: https://doi.org/10.1016/j.patrec.2005.10.010
J. Fieldsend, S. Singh, “Pareto evolutionary neural networks”, IEEE Transactions on Neural Networks, Vol. 16, No. 2, pp. 338–354, 2005 DOI: https://doi.org/10.1109/TNN.2004.841794
S. K. Sharma, P. Chandra, “Constructive neural networks: A review”, International Journal of Engineering Science and Technology, Vol. 2, No. 12, pp. 7847–7855, 2010
A. Engelbrecht, “A new pruning heuristic based on variance analysis of sensitivity information”, IEEE Transactions on Neural Networks, Vol. 11, No. 6, pp. 1386–1399, 2011 DOI: https://doi.org/10.1109/72.963775
H. Chen, Diversity and Regularization in Neural Network Ensembles, Ph.D. Thesis, School of Computer Science University of Birmingham, 2008
H. A. Abbass, R. Sarker, C. Newton, “PDE: a pareto-frontier differential evolution approach for multi-objective optimization problems”, 2001 Congress on Evolutionary Computation, Seoul, South Korea, pp. 971–978, 2001
H. A. Abbass, “A memetic Pareto evolutionary approach to artificial neural networks”, Australian Joint Conference on Artificial Intelligence, pp. 1–12, 2001 DOI: https://doi.org/10.1007/3-540-45656-2_1
H. A. Abbass, “An evolutionary artificial neural network approach for breast cancer diagnosis”, Artificial Intelligence in Medicine, Vol. 25, pp. 265–281, 2002 DOI: https://doi.org/10.1016/S0933-3657(02)00028-3
J. Teo, “Evolutionary Multi-Objective Optimization For Automatic Synthesis Of Artificial Neural Network Robot Controllers”, Malaysian Journal of Computer Science, Vol. 18, No. 2, pp. 54-62, 2005.
K. Krawiec, W. Jaskowski, M. Szubert, “Multi-objective convolutional learning for face labeling”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3451-3459, 2015
S. Wiegand, C. Igel, “Evolutionary multi-objective optimization of neural networks for face detection”, International Journal of Computational Intelligence and Applications, Vol. 4, No. 3, pp. 237–253), 2004 DOI: https://doi.org/10.1142/S1469026804001288
K. Deb, A. Pratab, S. Agarwal, T. Meyarivan, “A fast and elitist multi-objective genetic algorithm: NSGA2”, IEEE Transactions on Evolutionary Computation, Vol. 6, No. 2, pp. 182–197, 2002 DOI: https://doi.org/10.1109/4235.996017
N. Q. Sultan, M. S. Siti, M. Z. Azlan, “Multi-objective hybrid evolutionary algorithms for radial basis function neural network design”, Knowledge-Based Systems. Vol. 27, pp. 475–497, 2012 DOI: https://doi.org/10.1016/j.knosys.2011.10.001
N. Q. Sultan, M. S. Siti, Z. M. H. Siti, “Memetic multiobjective particle swarm optimization based radial basis function network for classification problems”, Information Sciences, Vol. 239, pp. 165–190, 2013 DOI: https://doi.org/10.1016/j.ins.2013.03.021
I. Kokshenev, A. Padua Braga, “An efficient multi-objective learning algorithm for RBF neural network”, Neurocomputing, Vol. 73, pp. 2799–2808, 2010 DOI: https://doi.org/10.1016/j.neucom.2010.06.022
M. Cruz-Ramirez, J. C. Fernandez, J. Sanchez-Monedero, C. Hervas-Martinez, “Memetic Pareto differential evolutionary artificial neural networks to determine growth multi-classes in predictive microbiology”, Evolutionary Intelligence, Vol. 3, pp. 187–199, 2010 DOI: https://doi.org/10.1007/s12065-010-0045-9
C. Igel, M. Husken, “Improving the Rprop learning algorithm”, Second International ICSC Symposium on Neural Computation (NC 2000, pp. 115–121, 2000
P. Ducange, B. Lazzerini, F. Marcelloni, “Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets”, Soft Computing, Vol. 14, pp. 713–728, 2010 DOI: https://doi.org/10.1007/s00500-009-0460-y
C. Ferri, J. Hernndez-Orallo, R. Modroiu, “An experimental comparison of performance measures for classification”, Pattern Recognition Letters, Vol. 30, pp. 27–38, 2009 DOI: https://doi.org/10.1016/j.patrec.2008.08.010
P. L. Bartlett, “The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network”, IEEE Transactions on Information Theory, Vol. 44, No. 2, pp. 525–536, 1998 DOI: https://doi.org/10.1109/18.661502
G. Dreyfus, J. Martinez, M. Samuelides, M. B. Cordon, F. Badran, S. Thiria, L. Herault, Réseaux de neurones: méthodologie et application, Eyrolles, 2004
R. Storn, “Differential evolution research-Trends and open questions”, Studies in Computational Intelligence, Vol. 143, pp. 1-31, 2008 DOI: https://doi.org/10.1007/978-3-540-68830-3_1
J. C. Fernandez Caballero, F. C. H. José Martinez, P. A. Gutiérrez, “Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks”, IEEE Transactions on Neural Networks, Vol. 21, No. 5, pp. 750–770, 2010 DOI: https://doi.org/10.1109/TNN.2010.2041468
How to Cite
MetricsAbstract Views: 602
PDF Downloads: 214
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.