A Low-cost Artificial Neural Network Model for Raspberry Pi

  • S. N. Truong Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology and Education, Vietnam

Abstract

In this paper, a ternary neural network with complementary binary arrays is proposed for representing the signed synaptic weights. The proposed ternary neural network is deployed on a low-cost Raspberry Pi board embedded system for the application of speech and image recognition. In conventional neural networks, the signed synaptic weights of –1, 0, and 1 are represented by 8-bit integers. To reduce the amount of required memory for signed synaptic weights, the signed values were represented by a complementary binary array. For the binary inputs, the multiplication of two binary numbers is replaced by the bit-wise AND operation to speed up the performance of the neural network. Regarding image recognition, the MINST dataset was used for training and testing of the proposed neural network. The recognition rate was as high as 94%. The proposed ternary neural network was applied to real-time object recognition. The recognition rate for recognizing 10 simple objects captured from the camera was 89%. The proposed ternary neural network with the complementary binary array for representing the signed synaptic weights can reduce the required memory for storing the model’s parameters and internal parameters by 75%. The proposed ternary neural network is 4.2, 2.7, and 2.4 times faster than the conventional ternary neural network for MNIST image recognition, speech commands recognition, and real-time object recognition respectively.

Keywords: artificial neural network, deep learning, speech recognition, image recognition, ternary neural networks

Downloads

Download data is not yet available.

References

A. Krizhevsky, I. Sutskever, G. E. Hinton, “Imagenet classification with deep convolutional neural networks”, Advances in Neural Information Processing Systems, Lake Tahoe, USA, December 3-8, 2012

K. He, X. Zhang, S. Ren, J. Sun, “Deep residual learning for image recognition”, in: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, IEEE, 2016 DOI: https://doi.org/10.1109/CVPR.2016.90

A. Graves, N. Jaitly, “Towards end-to-end speech recognition with recurrent neural networks”, International Conference on Machine Learning, Beijing, China, June 21-26, 2014

P. B. Patil, “Multilayered network for LPC based speech recognition”, IEEE Transactions on Consumer Electronic, Vol. 44, No. 2, pp. 435-438, 1998 DOI: https://doi.org/10.1109/30.681960

B. M .Zahran, “Using neural networks to predict the hardness of aluminum alloys”, Engineering, Technology & Applied Science Research, Vol. 5, No. 1, pp. 757-759, 2015 DOI: https://doi.org/10.48084/etasr.529

G. S. Fesghandis, A. Pooya, M. Kazemi, Z. N. Azimi, “Comparison of multilayer perceptron and radial basis function neural networks in predicting the success of new product development”, Engineering, Technology & Applied Science Research, Vol. 7, No. 1, pp. 1425-1428, 2015 DOI: https://doi.org/10.48084/etasr.936

H. Jang, A. Park, K. Jung, “Neural network implementation using CUDA and Open MP”, in: Proceedings - Digital Image Computing: Techniques and Applications, pp. 155-161, IEEE, 2008 DOI: https://doi.org/10.1109/DICTA.2008.82

Y. Wang, J. Lin, Z. Wang, “An energy-efficient architecture for binary weights convolution neural networks”, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 26, No. 2, pp. 280-293, 2017 DOI: https://doi.org/10.1109/TVLSI.2017.2767624

T. Simons, D. J. Lee, “A review of binarized neural networks”, Electronics, Vol. 8, No. 6, pp. 1-25, 2019 DOI: https://doi.org/10.3390/electronics8060661

M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio, “BinaryNet: Training deep neural networks with weights and activations constrained to +1 or −1”, available at: https://arxiv.org/abs/1602.02830, 2016

C. Baldassi, A. Braunstein, N. Brunel, R. Zecchina, “Efficient supervised learning in networks with binary synapses”, Proceedings of the National Academy of Science of the USA, Vol. 104, No. 26, pp. 11079-11084, 2007 DOI: https://doi.org/10.1073/pnas.0700324104

K. Hwang, W. Sung, “Fixed-point feedforward deep neural network design using weights +1, 0, and −1”, 2014 IEEE Workshop on Signal Processing Systems, Belfast, UK, October 20–22, 2014 DOI: https://doi.org/10.1109/SiPS.2014.6986082

H. Yonekawa, S. Sato, H. Nakahara, “A ternary weight binary input convolutional neural network: Realization on the embedded processor”, IEEE 48th International Symposium on Multiple-Valued Logic, Linz, Austria, May 16-18, 2018 DOI: https://doi.org/10.1109/ISMVL.2018.00038

S. Yin, P. Ouyang, J. Yang, T. Lu, X. Li, L. Liu, S. Wei, “An energy-efficient reconfigurable processor for binary-and ternary-weight neural networks with flexible data bit width”, IEEE Journal of Solid-State Circuits, Vol. 54, No. 4, pp. 1120-1136, 2018 DOI: https://doi.org/10.1109/JSSC.2018.2881913

L. Deng, P. Jiao, J. Pei, Z. Wu, G. Li “GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework”, Neural Networks, Vol. 100, pp. 49-58, 2018 DOI: https://doi.org/10.1016/j.neunet.2018.01.010

L. F. Abbott, W. G. Regehr, “Synaptic computation”, Nature, Vol. 431, pp. 796-803, 2004 DOI: https://doi.org/10.1038/nature03010

R. S. Zucker, W. G. Regehr, “Short-term synaptic plasticity”, Annual Review of Physiology, Vol. 64, pp. 355–405, 2002 DOI: https://doi.org/10.1146/annurev.physiol.64.092501.114547

R. Lamprecht, J. LeDoux, “Structural plasticity and memory”, Nature Reviews, Neuroscience, Vol. 5, No. 1, pp. 45-54, 2004 DOI: https://doi.org/10.1038/nrn1301

T. Mitchell, Machine learning, McGraw-Hill, 1997

L. Deng, “The MNIST database of handwritten digit images for machine learning research [Best of the Web]”, IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 141-142, 2012 DOI: https://doi.org/10.1109/MSP.2012.2211477

P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition”, available at: https://arxiv.org/abs/1804.03209, 2018

Metrics

Abstract Views: 444
PDF Downloads: 210

Metrics Information
Bookmark and Share