A Constrained Multi-Objective Learning Algorithm for Feed-Forward Neural Network Classifiers


  • M. Njah Control and Energy Management laboratory (CEM Lab), Digital Research Center of Sfax, Tunisia
  • R. El Hamdi Control and Energy Management laboratory (CEM Lab), Digital Research Center of Sfax, Tunisia
Volume: 7 | Issue: 3 | Pages: 1685-1693 | June 2017 | https://doi.org/10.48084/etasr.968


This paper proposes a new approach to address the optimal design of a Feed-forward Neural Network (FNN) based classifier. The originality of the proposed methodology, called CMOA, lie in the use of a new constraint handling technique based on a self-adaptive penalty procedure in order to direct the entire search effort towards finding only Pareto optimal solutions that are acceptable. Neurons and connections of the FNN Classifier are dynamically built during the learning process. The approach includes differential evolution to create new individuals and then keeps only the non-dominated ones as the basis for the next generation. The designed FNN Classifier is applied to six binary classification benchmark problems, obtained from the UCI repository, and results indicated the advantages of the proposed approach over other existing multi-objective evolutionary neural networks classifiers reported recently in the literature.


FNN Classifier, Constrained Multi-Objective Optimization, Pareto Dominance Criterion, Differential Evolution


Download data is not yet available.


C. Aggarwal, Data Classification: Algorithms and Applications, 1st Edition. Chapman and Hall/CRC, 2014

R. Ramli, H. Arof, F. Ibrahim, M. Y. I. Idris, A. S. M. Khairuddin, “Classification of Eyelid Position and Eyeball Movement Using EEG Signals”, Malaysian Journal of Computer Science. Vol. 28, No. 1, pp. pp. 28-45, 2015

S. M. A. Kalaiarasi, S. Gopala, C. Ali, T. Jason, “Artificial Neural Network Tree Approach In Data Mining”, Malaysian Journal of Computer Science, Vol. 20, No. 1, pp. 51-62, 2007 DOI: https://doi.org/10.22452/mjcs.vol20no1.5

V. Chaitali, B. Nikita, M. Darshana, “A survey on various classification techniques for clinical decision support system” International Journal of Computer Applications, Vol. 116, No. 23, pp. 14–17, 2015 DOI: https://doi.org/10.5120/20498-2369

A. Azzini, A. G. B. Tettamanzi, “Evolutionary ANNS: a state of the art survey”, Intelligenza Artificiale,Vol. 5, pp. 19–35, 2011 DOI: https://doi.org/10.3233/IA-2011-0002

Y. Jin, B. Sendhoff, “Pareto-based multi-objective machine learning: An overview and case studies”, IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews, Vol. 38, No. 3, pp. 397–415, 2008 DOI: https://doi.org/10.1109/TSMCC.2008.919172

T. Fawcett, “An introduction to roc analysis”, Pattern Recognition Letters, Vol. 27, No. 3, pp. 861–874, 2006 DOI: https://doi.org/10.1016/j.patrec.2005.10.010

J. Fieldsend, S. Singh, “Pareto evolutionary neural networks”, IEEE Transactions on Neural Networks, Vol. 16, No. 2, pp. 338–354, 2005 DOI: https://doi.org/10.1109/TNN.2004.841794

S. K. Sharma, P. Chandra, “Constructive neural networks: A review”, International Journal of Engineering Science and Technology, Vol. 2, No. 12, pp. 7847–7855, 2010

A. Engelbrecht, “A new pruning heuristic based on variance analysis of sensitivity information”, IEEE Transactions on Neural Networks, Vol. 11, No. 6, pp. 1386–1399, 2011 DOI: https://doi.org/10.1109/72.963775

H. Chen, Diversity and Regularization in Neural Network Ensembles, Ph.D. Thesis, School of Computer Science University of Birmingham, 2008

H. A. Abbass, R. Sarker, C. Newton, “PDE: a pareto-frontier differential evolution approach for multi-objective optimization problems”, 2001 Congress on Evolutionary Computation, Seoul, South Korea, pp. 971–978, 2001

H. A. Abbass, “A memetic Pareto evolutionary approach to artificial neural networks”, Australian Joint Conference on Artificial Intelligence, pp. 1–12, 2001 DOI: https://doi.org/10.1007/3-540-45656-2_1

H. A. Abbass, “An evolutionary artificial neural network approach for breast cancer diagnosis”, Artificial Intelligence in Medicine, Vol. 25, pp. 265–281, 2002 DOI: https://doi.org/10.1016/S0933-3657(02)00028-3

J. Teo, “Evolutionary Multi-Objective Optimization For Automatic Synthesis Of Artificial Neural Network Robot Controllers”, Malaysian Journal of Computer Science, Vol. 18, No. 2, pp. 54-62, 2005.

K. Krawiec, W. Jaskowski, M. Szubert, “Multi-objective convolutional learning for face labeling”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3451-3459, 2015

S. Wiegand, C. Igel, “Evolutionary multi-objective optimization of neural networks for face detection”, International Journal of Computational Intelligence and Applications, Vol. 4, No. 3, pp. 237–253), 2004 DOI: https://doi.org/10.1142/S1469026804001288

K. Deb, A. Pratab, S. Agarwal, T. Meyarivan, “A fast and elitist multi-objective genetic algorithm: NSGA2”, IEEE Transactions on Evolutionary Computation, Vol. 6, No. 2, pp. 182–197, 2002 DOI: https://doi.org/10.1109/4235.996017

N. Q. Sultan, M. S. Siti, M. Z. Azlan, “Multi-objective hybrid evolutionary algorithms for radial basis function neural network design”, Knowledge-Based Systems. Vol. 27, pp. 475–497, 2012 DOI: https://doi.org/10.1016/j.knosys.2011.10.001

N. Q. Sultan, M. S. Siti, Z. M. H. Siti, “Memetic multiobjective particle swarm optimization based radial basis function network for classification problems”, Information Sciences, Vol. 239, pp. 165–190, 2013 DOI: https://doi.org/10.1016/j.ins.2013.03.021

I. Kokshenev, A. Padua Braga, “An efficient multi-objective learning algorithm for RBF neural network”, Neurocomputing, Vol. 73, pp. 2799–2808, 2010 DOI: https://doi.org/10.1016/j.neucom.2010.06.022

M. Cruz-Ramirez, J. C. Fernandez, J. Sanchez-Monedero, C. Hervas-Martinez, “Memetic Pareto differential evolutionary artificial neural networks to determine growth multi-classes in predictive microbiology”, Evolutionary Intelligence, Vol. 3, pp. 187–199, 2010 DOI: https://doi.org/10.1007/s12065-010-0045-9

C. Igel, M. Husken, “Improving the Rprop learning algorithm”, Second International ICSC Symposium on Neural Computation (NC 2000, pp. 115–121, 2000

P. Ducange, B. Lazzerini, F. Marcelloni, “Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets”, Soft Computing, Vol. 14, pp. 713–728, 2010 DOI: https://doi.org/10.1007/s00500-009-0460-y

C. Ferri, J. Hernndez-Orallo, R. Modroiu, “An experimental comparison of performance measures for classification”, Pattern Recognition Letters, Vol. 30, pp. 27–38, 2009 DOI: https://doi.org/10.1016/j.patrec.2008.08.010

P. L. Bartlett, “The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network”, IEEE Transactions on Information Theory, Vol. 44, No. 2, pp. 525–536, 1998 DOI: https://doi.org/10.1109/18.661502

G. Dreyfus, J. Martinez, M. Samuelides, M. B. Cordon, F. Badran, S. Thiria, L. Herault, Réseaux de neurones: méthodologie et application, Eyrolles, 2004

R. Storn, “Differential evolution research-Trends and open questions”, Studies in Computational Intelligence, Vol. 143, pp. 1-31, 2008 DOI: https://doi.org/10.1007/978-3-540-68830-3_1

J. C. Fernandez Caballero, F. C. H. José Martinez, P. A. Gutiérrez, “Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks”, IEEE Transactions on Neural Networks, Vol. 21, No. 5, pp. 750–770, 2010 DOI: https://doi.org/10.1109/TNN.2010.2041468


How to Cite

M. Njah and R. El Hamdi, “A Constrained Multi-Objective Learning Algorithm for Feed-Forward Neural Network Classifiers”, Eng. Technol. Appl. Sci. Res., vol. 7, no. 3, pp. 1685–1693, Jun. 2017.


Abstract Views: 602
PDF Downloads: 214

Metrics Information
Bookmark and Share