Systems Modeling Using Deep Elman Neural Network
Abstract
In this paper, the modeling of complex systems using deep Elman neural network architecture is improved. The emphasis is to retrieve better deep Elman structure that emulates perfectly such dynamic systems. To achieve this goal, sigmoid activation functions in the hidden and output layers nodes are chosen and data files on considered systems for modeling and validation steps are given. Simulation results prove the ability and the efficiency of a deep Elman neural network with two hidden layers in this task.
Keywords:
Elman neural network, recurrent neural network, deep learning, complex systems, modelingDownloads
References
J. Schmidhuber, “Deep learning in neural networks: An overview”, Neural Networks, Vol. 61, pp. 85-117, 2015 DOI: https://doi.org/10.1016/j.neunet.2014.09.003
N. Majumder, S. Poria, A. Gelbukh, E. Cambria, “Deep learning-based document modeling for personality detection from text”, IEEE Intelligent Systems, Vol. 32, pp. 74-79, 2017 DOI: https://doi.org/10.1109/MIS.2017.23
Z. Jiang, L. Li, D. Huang, L. Lin, “Training word embeddings for deep learning in biomedical text mining tasks”, IEEE International Conference on Bioinformatics and Biomedicine, Washington, DC, USA, November 9-12, 2015 DOI: https://doi.org/10.1109/BIBM.2015.7359756
L. Deng, G. Hinton, B. Kingsbury, “New types of deep neual network learning for speech recognition and related applications: an overview”, IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, May 26-31, 2013 DOI: https://doi.org/10.1109/ICASSP.2013.6639344
D. Chen, B. K. W. Mak, “Multitask learning of deep neural networks for low-resource speech recognition”, IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 23, No. 7, pp. 1172-1183, 2015 DOI: https://doi.org/10.1109/TASLP.2015.2422573
S. M. S. Islam, S. Rahman, M. M. Rahman, E. K. Dey, M. Shoyaib, “Application of deep learning to computer vision: a comprehensive study”, 5th International Conference on Informatics, Electronics and Vision, Dhaka, Bangladesh, May 13-14, 2016 DOI: https://doi.org/10.1109/ICIEV.2016.7760071
N. Kruger, P. Janssen, S. Kalkan, M. Lappe, A. Leonardis, J. Piater, A. J. Rodriguez-Sanchez, L. Wiskott, “Deep hierarchies in the primate visual cortex: what can we learn for computer vision”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, No. 8, pp. 1847-1871, 2013 DOI: https://doi.org/10.1109/TPAMI.2012.272
M. Chengcai, G. Xiaodong, W. Yuanyuan, “Fault diagnosis of power electronic system based on fault gradation and neural network group”, Neurocomputing, Vol. 72, pp. 2909-2914, 2009 DOI: https://doi.org/10.1016/j.neucom.2008.06.033
S. Hochreiter, J. Schmidhuber, “Long short-term memory”, Neural Computing, Vol. 91, pp. 735-780, 1997
Z. C. Lipton, J. Berkowitz, C. Elkan, “A critical review of recurrent neural networks for sequence learning”, available at: https://arxiv.org/abs/1506.00019, 2015
G. E. Hinton, R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks”, Science, Vol. 313, pp. 504-507, 2006 DOI: https://doi.org/10.1126/science.1127647
X. Chen, X. Lin, “Big data deep learning: challenges and perspectives”, IEEE Access, Vol. 2, pp. 514-525, 2014 DOI: https://doi.org/10.1109/ACCESS.2014.2325029
J. Gunther, P. M. Pilarski, G. Helfrich, H. Shen, K. Diepold, “Intelligent laser welding through representation, prediction, and control learning: An architecture with deep neural networks and reinforcement learning”, Mechatronics, Vol. 34, pp. 1-11, 2016 DOI: https://doi.org/10.1016/j.mechatronics.2015.09.004
B. Chandra, R. K. Sharma, “Deep learning with adaptive learning rate using Laplacian score”, Expert Systems with Applications, Vol. 63, pp. 1-7, 2016 DOI: https://doi.org/10.1016/j.eswa.2016.05.022
A. Ali, F. Yangyu, “Unsupervised feature learning and automatic modulation classification using deep learning model”, Physical Communication, Vol. 25, No. 1, pp. 75-84, 2017 DOI: https://doi.org/10.1016/j.phycom.2017.09.004
H. M. Fayek, M. Lech, L. Cacedon, “Evaluating deep learning architectures for Speech Emotion Recognition”, Neural Networks, Vol. 92, pp. 60-68, 2017 DOI: https://doi.org/10.1016/j.neunet.2017.02.013
S. Achanta, S. V. Gangashetty, “Deep Elman Recurrent Neural Networks for Statistical Parametric Speech Synthesis”, Speech Communication”, Vol. 93, pp. 31-42, 2017 DOI: https://doi.org/10.1016/j.specom.2017.08.003
H. Zen, H. Sak, “Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis”, in: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4470-4474, IEEE, 2015 DOI: https://doi.org/10.1109/ICASSP.2015.7178816
Y. Fan, Y. Qian, F. L. Xie, F. K. Soong, “TTS synthesis with bidirectional LSTM based recurrent neural networks”, Interspeech 2014, Singapore, September 14-18, 2014
Z. Wu, S. King, “Investigating gated recurrent networks for speech synthesis”, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, Shanghai, China, March 20-25, 2016 DOI: https://doi.org/10.1109/ICASSP.2016.7472657
A. Rahman, V. Srikumar, A. D. Smith, “Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networks”, Applied Energy, Vol. 212, pp. 372-385, 2018 DOI: https://doi.org/10.1016/j.apenergy.2017.12.051
X. Qian, L. Han, Y. Wang, M. Ding, “Deep learning assisted robust visual tracking with adaptive particle filtering”, Signal Processing: Image Communication, Vol. 60, pp. 183-192, 2018 DOI: https://doi.org/10.1016/j.image.2017.09.001
K. P. Ferentinos, “Deep learning models for plant disease detection and diagnosis”, Computers and Electronics in Agriculture, Vol. 145, pp. 311-318, 2018 DOI: https://doi.org/10.1016/j.compag.2018.01.009
Q. Zhang, L. T. Yang, Z. Chen, P. Li, “A survey on deep learning for big data”, Information Fusion, Vol. 42, pp. 146-157, 2018 DOI: https://doi.org/10.1016/j.inffus.2017.10.006
R. Hao, C. Yi, J. Qu, Y. Xin, T. Qiu, “A novel adaptive fault detection methodology for complex system using deep belief networks and multiple models: A case study on cryogenic propellant loading system”, Neurocomputing, Vol. 275, pp. 2111-2125, 2018 DOI: https://doi.org/10.1016/j.neucom.2017.10.063
F. Fourati, M. Chtourou, “A greenhouse control with feed-forward and recurrent neural networks”, Simulation Modelling Practice and Theory, Vol. 15, No. 8, pp. 1016-1028, 2007 DOI: https://doi.org/10.1016/j.simpat.2007.06.001
J. L. Elman, “Finding structure in time”, Cognitive Science, Vol. 14, No. 2, pp. 179-211, 1990 DOI: https://doi.org/10.1207/s15516709cog1402_1
D. T. Pham, X. Liu, “Dynamic system modeling using partially recurrent neural networks”, Journal of Systems Engineereing”, Vol. 2, No. 2, pp. 90-97, 1992
K. Kamijo, T. Tanigawa, “Stock price pattern recognition-a recurrent neural network approach”, International Joint Conference on Neural Networks, San Diego, CA, USA, June 17-21, 1990 DOI: https://doi.org/10.1109/IJCNN.1990.137572
D. T. Pham, X. Liu, “Training of Elman networks and dynamic system modelling”, International Journal of Systems Science Vol. 27, No. 2, pp. 221-226, 1996 DOI: https://doi.org/10.1080/00207729608929207
A. Yan, W. Wang, C. Zhang, H. Zhao, “A fault prediction method that uses improved case-based reasoning to continuously predict the status of a shaft furnace”, Information Sciences, Vol. 259, pp. 269-281, 2014 DOI: https://doi.org/10.1016/j.ins.2013.04.025
F. Baghernezhad, K. Khorasani, “Computationally intelligent strategies for robust fault detection, isolation, and identification of mobile robots”, Neurocomputing, Vol. 171, pp. 335-346, 2016 DOI: https://doi.org/10.1016/j.neucom.2015.06.050
H. B. Huang, X. R. Huang, R. X. Li, T. C. Lim, W. P. Ding, “Sound quality prediction of vehicle interior noise using deep belief networks”, Applied Acoustics,Vol. 113, pp. 149-161, 2016 DOI: https://doi.org/10.1016/j.apacoust.2016.06.021
D. Psaltis, A. Sideris, A. A. Yamamura, “A multilayer neural network Controller”, IEEE International Conference on Neural Networks, San Diego, California, June 21-24, 1987
M. Souissi, Modelisation et Commande du Climat d’une Serre Agricole, PhD Thesis, University of Tunis, Tunis, 2002 (in French)
Downloads
How to Cite
License
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
