A Low-cost Artificial Neural Network Model for Raspberry Pi

  • S. N. Truong Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology and Education, Vietnam
Keywords: artificial neural network, deep learning, speech recognition, image recognition, ternary neural networks

Abstract

In this paper, a ternary neural network with complementary binary arrays is proposed for representing the signed synaptic weights. The proposed ternary neural network is deployed on a low-cost Raspberry Pi board embedded system for the application of speech and image recognition. In conventional neural networks, the signed synaptic weights of –1, 0, and 1 are represented by 8-bit integers. To reduce the amount of required memory for signed synaptic weights, the signed values were represented by a complementary binary array. For the binary inputs, the multiplication of two binary numbers is replaced by the bit-wise AND operation to speed up the performance of the neural network. Regarding image recognition, the MINST dataset was used for training and testing of the proposed neural network. The recognition rate was as high as 94%. The proposed ternary neural network was applied to real-time object recognition. The recognition rate for recognizing 10 simple objects captured from the camera was 89%. The proposed ternary neural network with the complementary binary array for representing the signed synaptic weights can reduce the required memory for storing the model’s parameters and internal parameters by 75%. The proposed ternary neural network is 4.2, 2.7, and 2.4 times faster than the conventional ternary neural network for MNIST image recognition, speech commands recognition, and real-time object recognition respectively.

Downloads

Download data is not yet available.

References

A. Krizhevsky, I. Sutskever, G. E. Hinton, “Imagenet classification with deep convolutional neural networks”, Advances in Neural Information Processing Systems, Lake Tahoe, USA, December 3-8, 2012

K. He, X. Zhang, S. Ren, J. Sun, “Deep residual learning for image recognition”, in: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, IEEE, 2016

A. Graves, N. Jaitly, “Towards end-to-end speech recognition with recurrent neural networks”, International Conference on Machine Learning, Beijing, China, June 21-26, 2014

P. B. Patil, “Multilayered network for LPC based speech recognition”, IEEE Transactions on Consumer Electronic, Vol. 44, No. 2, pp. 435-438, 1998

B. M .Zahran, “Using neural networks to predict the hardness of aluminum alloys”, Engineering, Technology & Applied Science Research, Vol. 5, No. 1, pp. 757-759, 2015

G. S. Fesghandis, A. Pooya, M. Kazemi, Z. N. Azimi, “Comparison of multilayer perceptron and radial basis function neural networks in predicting the success of new product development”, Engineering, Technology & Applied Science Research, Vol. 7, No. 1, pp. 1425-1428, 2015

H. Jang, A. Park, K. Jung, “Neural network implementation using CUDA and Open MP”, in: Proceedings - Digital Image Computing: Techniques and Applications, pp. 155-161, IEEE, 2008

Y. Wang, J. Lin, Z. Wang, “An energy-efficient architecture for binary weights convolution neural networks”, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 26, No. 2, pp. 280-293, 2017

T. Simons, D. J. Lee, “A review of binarized neural networks”, Electronics, Vol. 8, No. 6, pp. 1-25, 2019

M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio, “BinaryNet: Training deep neural networks with weights and activations constrained to +1 or −1”, available at: https://arxiv.org/abs/1602.02830, 2016

C. Baldassi, A. Braunstein, N. Brunel, R. Zecchina, “Efficient supervised learning in networks with binary synapses”, Proceedings of the National Academy of Science of the USA, Vol. 104, No. 26, pp. 11079-11084, 2007

K. Hwang, W. Sung, “Fixed-point feedforward deep neural network design using weights +1, 0, and −1”, 2014 IEEE Workshop on Signal Processing Systems, Belfast, UK, October 20–22, 2014

H. Yonekawa, S. Sato, H. Nakahara, “A ternary weight binary input convolutional neural network: Realization on the embedded processor”, IEEE 48th International Symposium on Multiple-Valued Logic, Linz, Austria, May 16-18, 2018

S. Yin, P. Ouyang, J. Yang, T. Lu, X. Li, L. Liu, S. Wei, “An energy-efficient reconfigurable processor for binary-and ternary-weight neural networks with flexible data bit width”, IEEE Journal of Solid-State Circuits, Vol. 54, No. 4, pp. 1120-1136, 2018

L. Deng, P. Jiao, J. Pei, Z. Wu, G. Li “GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework”, Neural Networks, Vol. 100, pp. 49-58, 2018

L. F. Abbott, W. G. Regehr, “Synaptic computation”, Nature, Vol. 431, pp. 796-803, 2004

R. S. Zucker, W. G. Regehr, “Short-term synaptic plasticity”, Annual Review of Physiology, Vol. 64, pp. 355–405, 2002

R. Lamprecht, J. LeDoux, “Structural plasticity and memory”, Nature Reviews, Neuroscience, Vol. 5, No. 1, pp. 45-54, 2004

T. Mitchell, Machine learning, McGraw-Hill, 1997

L. Deng, “The MNIST database of handwritten digit images for machine learning research [Best of the Web]”, IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 141-142, 2012

P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition”, available at: https://arxiv.org/abs/1804.03209, 2018

Metrics

Abstract Views: 129
PDF Downloads: 66

Metrics Information
Bookmark and Share