A Pruning Algorithm Based on Relevancy Index of Hidden Neurons Outputs

  • S. Abid Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, Tunisia
  • M. Chtourou Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, Tunisia
  • M. Djemel Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, Tunisia
Keywords: pruning algorithm, OBS approach, relevancy index, hidden neurons

Abstract

Choosing the training algorithm and determining the architecture of artificial neural networks are very important issues with large application. There are no general methods which permit the estimation of the adequate neural networks size. In order to achieve this goal, a pruning algorithm based on the relevancy index of hidden neurons outputs is developed in this paper. The relevancy index depends on the output amplitude of each hidden neuron and estimates his contribution on the learning process. This method is validated with an academic example and it is tested on a wind turbine modeling problem. Compared with two modified versions of Optimal Brain Surgeon (OBS) algorithm, the developed approach gives interesting results.

Author Biographies

S. Abid, Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, Tunisia
Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, B.P. 1173, 3038 Sfax, Tunisia
M. Chtourou, Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, Tunisia
Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, B.P. 1173, 3038 Sfax, Tunisia
M. Djemel, Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, Tunisia
Control & Energy Management Lab (CEM LAB), National School of Engineering of Sfax, University of Sfax, B.P. 1173, 3038 Sfax, Tunisia

Downloads

Download data is not yet available.

References

P. Mehra, B. W. Wah, Artificial Neural Networks: Concepts and Theory, IEEE Comput. Society Press, 1992

J. M. Zurada Introduction to Artificial Neural Systems, St Paul, MN: West, 1992

V. E. Neagoe, C.T. Tudoran, “A neural machine vision model for road detection in autonomous navigation”, U.P.B. Sci. Bull., Series C, Vol. 73, No. 2, pp. 167-178, 2011

E. Şuşnea, “Using artificial neural networks in e-learning systems”, U.P.B. Sci. Bull., Series C, Vol. 72, No. 4, pp. 91-100, 2010

A. Mechernene, M. Zerikat, S. Chekroun, “Indirect field oriented adaptive control of induction motor based on neuro-fuzzy controller”, J. Electrical Systems, Vol. 7, No. 3, pp. 308-319, 2011

D. Liu, T. S. Chang, Y. Zhang, “A constructive algorithm for feedforward neural networks with incremental training”, IEEE Transactions on Circuits and Systems. Fundamental Theory and Applications, Vol. 49, No. 12, pp. 1876-1879, 2002

R. Reed, “Pruning algorithms-A survey”, IEEE Trans. Neural Net., Vol. 4, No. 5, pp. 740-747, 1993

H. Honggui, Q. Junfei, “A novel pruning algorithm for self-organizing neural network”, International Joint Conference on Neural Networks, Atlanta, Georgia, USA, pp. 22-27, 2009

D. Juan, E. M. Joo, “A fast pruning algorithm for an efficient adaptive fuzzy neural network”, 8th IEEE International Conference on Control and Automation Xiamen, China, pp. 1030- 1035, 2010

Z. Zhang, J. Qiao, “A node pruning algorithm for feedforward neural network based on neural complexity”, International Conference on Intelligent Control and Information Processing, Dalian, China, pp. 406- 410, 2010

Y. Le Cun, L. S. Denker, S. A. Solla, “Optimal brain damage” in Advances in Neural Information Processing systems, D.S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, pp. 598-605, 1990

B. Hassibi, D. Stork, G. Wolff, “Optimal brain surgeon and general network pruning”, IEEE Int. Conf. Neural Networks, Vol. 1, pp. 293–299, 1993

A. Stahlberger, M. Riedmiller, “Fast network pruning and feature extraction using the unit-OBS algorithm”, Advances in Neural Information Processing Systems, Denver, Vol. 9, pp. 2-5, 1996

J. -F. Qiao, Y. Zhang, H. -G. Han, “Fast unit pruning algorithm for feedforward neural network design”, Applied Mathematics and Computation, Vol. 205, pp. 622–627, 2008

B. Hassibi, D. G. Stork, “Second-order derivatives for network pruning: Optimal brain surgeon”, Advances in Neural Information Processing Systems, Vol. 5, pp. 164-171, 1993

E. D. Karnin, “A simple procedure for pruning backpropagation trained neural networks”, IEEE Trans. Neural Networks, Vol. 1, pp. 239-242, 1990

N. Ciprian, M. Florin, “Operational parameters evaluation for optimal wind energy systems development”, U.P.B. Sci. Bull., Series C, Vol. 74, pp. 223-230, 2012

A. Pintea. D. Popescu, “A comparative study of digital IMC and RST regulators applied on a wind turbine”, U.P.B. Sci. Bull., Series C, Vol. 74, No. 4, pp. 27-38, 2012

P. Christou, “Advanced materials for turbine blade manufacture”, Reinforced Plastics, Vol. 51, No. 4, pp. 22-24, 2007

L. Fingersh, M. Hand, A. Laxson, “Wind turbine design cost and scaling model”, National Renewable Energy Laboratory, Technical Report NREL/TP-500-40566, 2006

Y. D. Song, B. Dhinakaran, X. Y. Bao, “Variable speed control of wind turbines using nonlinear and adaptive algorithms”, Wind Engineering and Industrial Aerodynamics, Vol. 85, pp. 293-308, 2000

K. Reif, F. Sonnemann, R. Unbehauen, “Nonlinear state observation using H∞-filtering filtering riccati design”, IEEE Transactions On Automatic Control, Vol. 44, No. 1, pp. 203-208, 1999

A. Khamlichi, B. Ayyat, M. Bezzazi, L. El Bakkali, V. C. Vivas, C. L. F. Castano, “Modelling and control of flexible wind turbines without wind speed measurements”, Australian Journal of Basic and Applied Sciences, Vol. 3, No. 4, pp. 3246-3258, 2009

S. Abid, M. Chtourou, M. Djemel, “Incremental and Stable Training Algorithm for Wind Turbine Neural Modeling”, Engineering Review (ER), Vol. 33, No. 3, pp. 165-172, 2013

How to Cite
[1]
S. Abid, M. Chtourou, and M. Djemel, “A Pruning Algorithm Based on Relevancy Index of Hidden Neurons Outputs”, Eng. Technol. Appl. Sci. Res., vol. 6, no. 4, pp. 1067-1074, May 2016.

Metrics

Abstract 348
PDF Downloads 35
Bookmark and Share