Systems Modeling Using Deep Elman Neural Network

L. Belhaj Salah, F. Fourati


In this paper, the modeling of complex systems using deep Elman neural network architecture is improved. The emphasis is to retrieve better deep Elman structure that emulates perfectly such dynamic systems. To achieve this goal, sigmoid activation functions in the hidden and output layers nodes are chosen and data files on considered systems for modeling and validation steps are given. Simulation results prove the ability and the efficiency of a deep Elman neural network with two hidden layers in this task.


Elman neural network; recurrent neural network; deep learning; complex systems; modeling

Full Text:



J. Schmidhuber, “Deep learning in neural networks: An overview”, Neural Networks, Vol. 61, pp. 85-117, 2015

N. Majumder, S. Poria, A. Gelbukh, E. Cambria, “Deep learning-based document modeling for personality detection from text”, IEEE Intelligent Systems, Vol. 32, pp. 74-79, 2017

Z. Jiang, L. Li, D. Huang, L. Lin, “Training word embeddings for deep learning in biomedical text mining tasks”, IEEE International Conference on Bioinformatics and Biomedicine, Washington, DC, USA, November 9-12, 2015

L. Deng, G. Hinton, B. Kingsbury, “New types of deep neual network learning for speech recognition and related applications: an overview”, IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, May 26-31, 2013

D. Chen, B. K. W. Mak, “Multitask learning of deep neural networks for low-resource speech recognition”, IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 23, No. 7, pp. 1172-1183, 2015

S. M. S. Islam, S. Rahman, M. M. Rahman, E. K. Dey, M. Shoyaib, “Application of deep learning to computer vision: a comprehensive study”, 5th International Conference on Informatics, Electronics and Vision, Dhaka, Bangladesh, May 13-14, 2016

N. Kruger, P. Janssen, S. Kalkan, M. Lappe, A. Leonardis, J. Piater, A. J. Rodriguez-Sanchez, L. Wiskott, “Deep hierarchies in the primate visual cortex: what can we learn for computer vision”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, No. 8, pp. 1847-1871, 2013

M. Chengcai, G. Xiaodong, W. Yuanyuan, “Fault diagnosis of power electronic system based on fault gradation and neural network group”, Neurocomputing, Vol. 72, pp. 2909-2914, 2009

S. Hochreiter, J. Schmidhuber, “Long short-term memory”, Neural Computing, Vol. 91, pp. 735-780, 1997

Z. C. Lipton, J. Berkowitz, C. Elkan, “A critical review of recurrent neural networks for sequence learning”, available at:, 2015

G. E. Hinton, R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks”, Science, Vol. 313, pp. 504-507, 2006

X. Chen, X. Lin, “Big data deep learning: challenges and perspectives”, IEEE Access, Vol. 2, pp. 514-525, 2014

J. Gunther, P. M. Pilarski, G. Helfrich, H. Shen, K. Diepold, “Intelligent laser welding through representation, prediction, and control learning: An architecture with deep neural networks and reinforcement learning”, Mechatronics, Vol. 34, pp. 1-11, 2016

B. Chandra, R. K. Sharma, “Deep learning with adaptive learning rate using Laplacian score”, Expert Systems with Applications, Vol. 63, pp. 1-7, 2016

A. Ali, F. Yangyu, “Unsupervised feature learning and automatic modulation classification using deep learning model”, Physical Communication, Vol. 25, No. 1, pp. 75-84, 2017

H. M. Fayek, M. Lech, L. Cacedon, “Evaluating deep learning architectures for Speech Emotion Recognition”, Neural Networks, Vol. 92, pp. 60-68, 2017

S. Achanta, S. V. Gangashetty, “Deep Elman Recurrent Neural Networks for Statistical Parametric Speech Synthesis”, Speech Communication”, Vol. 93, pp. 31-42, 2017

H. Zen, H. Sak, “Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis”, in: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4470-4474, IEEE, 2015

Y. Fan, Y. Qian, F. L. Xie, F. K. Soong, “TTS synthesis with bidirectional LSTM based recurrent neural networks”, Interspeech 2014, Singapore, September 14-18, 2014

Z. Wu, S. King, “Investigating gated recurrent networks for speech synthesis”, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, Shanghai, China, March 20-25, 2016

A. Rahman, V. Srikumar, A. D. Smith, “Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networks”, Applied Energy, Vol. 212, pp. 372-385, 2018

X. Qian, L. Han, Y. Wang, M. Ding, “Deep learning assisted robust visual tracking with adaptive particle filtering”, Signal Processing: Image Communication, Vol. 60, pp. 183-192, 2018

K. P. Ferentinos, “Deep learning models for plant disease detection and diagnosis”, Computers and Electronics in Agriculture, Vol. 145, pp. 311-318, 2018

Q. Zhang, L. T. Yang, Z. Chen, P. Li, “A survey on deep learning for big data”, Information Fusion, Vol. 42, pp. 146-157, 2018

R. Hao, C. Yi, J. Qu, Y. Xin, T. Qiu, “A novel adaptive fault detection methodology for complex system using deep belief networks and multiple models: A case study on cryogenic propellant loading system”, Neurocomputing, Vol. 275, pp. 2111-2125, 2018

F. Fourati, M. Chtourou, “A greenhouse control with feed-forward and recurrent neural networks”, Simulation Modelling Practice and Theory, Vol. 15, No. 8, pp. 1016-1028, 2007

J. L. Elman, “Finding structure in time”, Cognitive Science, Vol. 14, No. 2, pp. 179-211, 1990

D. T. Pham, X. Liu, “Dynamic system modeling using partially recurrent neural networks”, Journal of Systems Engineereing”, Vol. 2, No. 2, pp. 90-97, 1992

K. Kamijo, T. Tanigawa, “Stock price pattern recognition-a recurrent neural network approach”, International Joint Conference on Neural Networks, San Diego, CA, USA, June 17-21, 1990

D. T. Pham, X. Liu, “Training of Elman networks and dynamic system modelling”, International Journal of Systems Science Vol. 27, No. 2, pp. 221-226, 1996

A. Yan, W. Wang, C. Zhang, H. Zhao, “A fault prediction method that uses improved case-based reasoning to continuously predict the status of a shaft furnace”, Information Sciences, Vol. 259, pp. 269-281, 2014

F. Baghernezhad, K. Khorasani, “Computationally intelligent strategies for robust fault detection, isolation, and identification of mobile robots”, Neurocomputing, Vol. 171, pp. 335-346, 2016

H. B. Huang, X. R. Huang, R. X. Li, T. C. Lim, W. P. Ding, “Sound quality prediction of vehicle interior noise using deep belief networks”, Applied Acoustics,Vol. 113, pp. 149-161, 2016

D. Psaltis, A. Sideris, A. A. Yamamura, “A multilayer neural network Controller”, IEEE International Conference on Neural Networks, San Diego, California, June 21-24, 1987

M. Souissi, Modelisation et Commande du Climat d’une Serre Agricole, PhD Thesis, University of Tunis, Tunis, 2002 (in French)

eISSN: 1792-8036     pISSN: 2241-4487