A Pruning Algorithm Based on Relevancy Index of Hidden Neurons Outputs
Abstract
Choosing the training algorithm and determining the architecture of artificial neural networks are very important issues with large application. There are no general methods which permit the estimation of the adequate neural networks size. In order to achieve this goal, a pruning algorithm based on the relevancy index of hidden neurons outputs is developed in this paper. The relevancy index depends on the output amplitude of each hidden neuron and estimates his contribution on the learning process. This method is validated with an academic example and it is tested on a wind turbine modeling problem. Compared with two modified versions of Optimal Brain Surgeon (OBS) algorithm, the developed approach gives interesting results.
Keywords:
pruning algorithm, OBS approach, relevancy index, hidden neuronsDownloads
References
P. Mehra, B. W. Wah, Artificial Neural Networks: Concepts and Theory, IEEE Comput. Society Press, 1992
J. M. Zurada Introduction to Artificial Neural Systems, St Paul, MN: West, 1992
V. E. Neagoe, C.T. Tudoran, “A neural machine vision model for road detection in autonomous navigation”, U.P.B. Sci. Bull., Series C, Vol. 73, No. 2, pp. 167-178, 2011
E. Şuşnea, “Using artificial neural networks in e-learning systems”, U.P.B. Sci. Bull., Series C, Vol. 72, No. 4, pp. 91-100, 2010
A. Mechernene, M. Zerikat, S. Chekroun, “Indirect field oriented adaptive control of induction motor based on neuro-fuzzy controller”, J. Electrical Systems, Vol. 7, No. 3, pp. 308-319, 2011
D. Liu, T. S. Chang, Y. Zhang, “A constructive algorithm for feedforward neural networks with incremental training”, IEEE Transactions on Circuits and Systems. Fundamental Theory and Applications, Vol. 49, No. 12, pp. 1876-1879, 2002 DOI: https://doi.org/10.1109/TCSI.2002.805733
R. Reed, “Pruning algorithms-A survey”, IEEE Trans. Neural Net., Vol. 4, No. 5, pp. 740-747, 1993 DOI: https://doi.org/10.1109/72.248452
H. Honggui, Q. Junfei, “A novel pruning algorithm for self-organizing neural network”, International Joint Conference on Neural Networks, Atlanta, Georgia, USA, pp. 22-27, 2009 DOI: https://doi.org/10.1109/IJCNN.2009.5178581
D. Juan, E. M. Joo, “A fast pruning algorithm for an efficient adaptive fuzzy neural network”, 8th IEEE International Conference on Control and Automation Xiamen, China, pp. 1030- 1035, 2010
Z. Zhang, J. Qiao, “A node pruning algorithm for feedforward neural network based on neural complexity”, International Conference on Intelligent Control and Information Processing, Dalian, China, pp. 406- 410, 2010 DOI: https://doi.org/10.1109/ICICIP.2010.5564272
Y. Le Cun, L. S. Denker, S. A. Solla, “Optimal brain damage” in Advances in Neural Information Processing systems, D.S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, pp. 598-605, 1990
B. Hassibi, D. Stork, G. Wolff, “Optimal brain surgeon and general network pruning”, IEEE Int. Conf. Neural Networks, Vol. 1, pp. 293–299, 1993
A. Stahlberger, M. Riedmiller, “Fast network pruning and feature extraction using the unit-OBS algorithm”, Advances in Neural Information Processing Systems, Denver, Vol. 9, pp. 2-5, 1996
J. -F. Qiao, Y. Zhang, H. -G. Han, “Fast unit pruning algorithm for feedforward neural network design”, Applied Mathematics and Computation, Vol. 205, pp. 622–627, 2008 DOI: https://doi.org/10.1016/j.amc.2008.05.049
B. Hassibi, D. G. Stork, “Second-order derivatives for network pruning: Optimal brain surgeon”, Advances in Neural Information Processing Systems, Vol. 5, pp. 164-171, 1993
E. D. Karnin, “A simple procedure for pruning backpropagation trained neural networks”, IEEE Trans. Neural Networks, Vol. 1, pp. 239-242, 1990 DOI: https://doi.org/10.1109/72.80236
N. Ciprian, M. Florin, “Operational parameters evaluation for optimal wind energy systems development”, U.P.B. Sci. Bull., Series C, Vol. 74, pp. 223-230, 2012
A. Pintea. D. Popescu, “A comparative study of digital IMC and RST regulators applied on a wind turbine”, U.P.B. Sci. Bull., Series C, Vol. 74, No. 4, pp. 27-38, 2012
P. Christou, “Advanced materials for turbine blade manufacture”, Reinforced Plastics, Vol. 51, No. 4, pp. 22-24, 2007 DOI: https://doi.org/10.1016/S0034-3617(07)70148-0
L. Fingersh, M. Hand, A. Laxson, “Wind turbine design cost and scaling model”, National Renewable Energy Laboratory, Technical Report NREL/TP-500-40566, 2006 DOI: https://doi.org/10.2172/897434
Y. D. Song, B. Dhinakaran, X. Y. Bao, “Variable speed control of wind turbines using nonlinear and adaptive algorithms”, Wind Engineering and Industrial Aerodynamics, Vol. 85, pp. 293-308, 2000 DOI: https://doi.org/10.1016/S0167-6105(99)00131-2
K. Reif, F. Sonnemann, R. Unbehauen, “Nonlinear state observation using H∞-filtering filtering riccati design”, IEEE Transactions On Automatic Control, Vol. 44, No. 1, pp. 203-208, 1999 DOI: https://doi.org/10.1109/9.739140
A. Khamlichi, B. Ayyat, M. Bezzazi, L. El Bakkali, V. C. Vivas, C. L. F. Castano, “Modelling and control of flexible wind turbines without wind speed measurements”, Australian Journal of Basic and Applied Sciences, Vol. 3, No. 4, pp. 3246-3258, 2009
S. Abid, M. Chtourou, M. Djemel, “Incremental and Stable Training Algorithm for Wind Turbine Neural Modeling”, Engineering Review (ER), Vol. 33, No. 3, pp. 165-172, 2013
Downloads
How to Cite
License
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
