A Pruning Algorithm Based on Relevancy Index of Hidden Neurons Outputs

S. Abid, M. Chtourou, M. Djemel

Abstract


Choosing the training algorithm and determining the architecture of artificial neural networks are very important issues with large application. There are no general methods which permit the estimation of the adequate neural networks size. In order to achieve this goal, a pruning algorithm based on the relevancy index of hidden neurons outputs is developed in this paper. The relevancy index depends on the output amplitude of each hidden neuron and estimates his contribution on the learning process. This method is validated with an academic example and it is tested on a wind turbine modeling problem. Compared with two modified versions of Optimal Brain Surgeon (OBS) algorithm, the developed approach gives interesting results.


Keywords


pruning algorithm; OBS approach; relevancy index; hidden neurons

Full Text:

PDF

References


P. Mehra, B. W. Wah, Artificial Neural Networks: Concepts and Theory, IEEE Comput. Society Press, 1992

J. M. Zurada Introduction to Artificial Neural Systems, St Paul, MN: West, 1992

V. E. Neagoe, C.T. Tudoran, “A neural machine vision model for road detection in autonomous navigation”, U.P.B. Sci. Bull., Series C, Vol. 73, No. 2, pp. 167-178, 2011

E. Şuşnea, “Using artificial neural networks in e-learning systems”, U.P.B. Sci. Bull., Series C, Vol. 72, No. 4, pp. 91-100, 2010

A. Mechernene, M. Zerikat, S. Chekroun, “Indirect field oriented adaptive control of induction motor based on neuro-fuzzy controller”, J. Electrical Systems, Vol. 7, No. 3, pp. 308-319, 2011

D. Liu, T. S. Chang, Y. Zhang, “A constructive algorithm for feedforward neural networks with incremental training”, IEEE Transactions on Circuits and Systems. Fundamental Theory and Applications, Vol. 49, No. 12, pp. 1876-1879, 2002

R. Reed, “Pruning algorithms-A survey”, IEEE Trans. Neural Net., Vol. 4, No. 5, pp. 740-747, 1993

H. Honggui, Q. Junfei, “A novel pruning algorithm for self-organizing neural network”, International Joint Conference on Neural Networks, Atlanta, Georgia, USA, pp. 22-27, 2009

D. Juan, E. M. Joo, “A fast pruning algorithm for an efficient adaptive fuzzy neural network”, 8th IEEE International Conference on Control and Automation Xiamen, China, pp. 1030- 1035, 2010

Z. Zhang, J. Qiao, “A node pruning algorithm for feedforward neural network based on neural complexity”, International Conference on Intelligent Control and Information Processing, Dalian, China, pp. 406- 410, 2010

Y. Le Cun, L. S. Denker, S. A. Solla, “Optimal brain damage” in Advances in Neural Information Processing systems, D.S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, pp. 598-605, 1990

B. Hassibi, D. Stork, G. Wolff, “Optimal brain surgeon and general network pruning”, IEEE Int. Conf. Neural Networks, Vol. 1, pp. 293–299, 1993

A. Stahlberger, M. Riedmiller, “Fast network pruning and feature extraction using the unit-OBS algorithm”, Advances in Neural Information Processing Systems, Denver, Vol. 9, pp. 2-5, 1996

J. -F. Qiao, Y. Zhang, H. -G. Han, “Fast unit pruning algorithm for feedforward neural network design”, Applied Mathematics and Computation, Vol. 205, pp. 622–627, 2008

B. Hassibi, D. G. Stork, “Second-order derivatives for network pruning: Optimal brain surgeon”, Advances in Neural Information Processing Systems, Vol. 5, pp. 164-171, 1993

E. D. Karnin, “A simple procedure for pruning backpropagation trained neural networks”, IEEE Trans. Neural Networks, Vol. 1, pp. 239-242, 1990

N. Ciprian, M. Florin, “Operational parameters evaluation for optimal wind energy systems development”, U.P.B. Sci. Bull., Series C, Vol. 74, pp. 223-230, 2012

A. Pintea. D. Popescu, “A comparative study of digital IMC and RST regulators applied on a wind turbine”, U.P.B. Sci. Bull., Series C, Vol. 74, No. 4, pp. 27-38, 2012

P. Christou, “Advanced materials for turbine blade manufacture”, Reinforced Plastics, Vol. 51, No. 4, pp. 22-24, 2007

L. Fingersh, M. Hand, A. Laxson, “Wind turbine design cost and scaling model”, National Renewable Energy Laboratory, Technical Report NREL/TP-500-40566, 2006

Y. D. Song, B. Dhinakaran, X. Y. Bao, “Variable speed control of wind turbines using nonlinear and adaptive algorithms”, Wind Engineering and Industrial Aerodynamics, Vol. 85, pp. 293-308, 2000

K. Reif, F. Sonnemann, R. Unbehauen, “Nonlinear state observation using H∞-filtering filtering riccati design”, IEEE Transactions On Automatic Control, Vol. 44, No. 1, pp. 203-208, 1999

A. Khamlichi, B. Ayyat, M. Bezzazi, L. El Bakkali, V. C. Vivas, C. L. F. Castano, “Modelling and control of flexible wind turbines without wind speed measurements”, Australian Journal of Basic and Applied Sciences, Vol. 3, No. 4, pp. 3246-3258, 2009

S. Abid, M. Chtourou, M. Djemel, “Incremental and Stable Training Algorithm for Wind Turbine Neural Modeling”, Engineering Review (ER), Vol. 33, No. 3, pp. 165-172, 2013




eISSN: 1792-8036     pISSN: 2241-4487