Multi-national and Multi-language License Plate Detection using Convolutional Neural Networks
Abstract
Many real-life machine and computer vision applications are focusing on object detection and recognition. In recent years, deep learning-based approaches gained increasing interest due to their high accuracy levels. License Plate (LP) detection and classification have been studied extensively over the last decades. However, more accurate and language-independent approaches are still required. This paper presents a new approach to detect LPs and recognize their country, language, and layout. Furthermore, a new LP dataset for both multi-national and multi-language detection, with either one-line or two-line layouts is presented. The YOLOv2 detector with ResNet feature extraction core was utilized for LP detection, and a new low complexity convolutional neural network architecture was proposed to classify LPs. Results show that the proposed approach achieves an average detection precision of 99.57%, whereas the country, language, and layout classification accuracy is 99.33%.
Keywords:
license plate detection, license plate classification, LPD, Yolo detector, convolutional neural network, deep learningDownloads
References
Z.-Q. Zhao, P. Zheng, S.-T. Xu, and X. Wu, "Object Detection With Deep Learning: A Review," IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 11, pp. 3212-3232, Nov. 2019. DOI: https://doi.org/10.1109/TNNLS.2018.2876865
P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part-Based Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sep. 2010. DOI: https://doi.org/10.1109/TPAMI.2009.167
R. Lienhart and J. Maydt, "An extended set of Haar-like features for rapid object detection," presented at International Conference on Image Processing, Rochester, NY, USA, Sep. 22-25, 2002.
D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004. DOI: https://doi.org/10.1023/B:VISI.0000029664.99615.94
N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Jun. 2005, vol. 1, pp. 886-893.
P. Matlani and M. Shrivastava, "An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid Feature Selection Techniques," Engineering, Technology & Applied Science Research, vol. 9, no. 2, pp. 3939-3944, Apr. 2019. DOI: https://doi.org/10.48084/etasr.2571
C. Cortes and V. Vapnik, "Support Vector Networks", Machine Learning, Vol. 20, No. 3, pp. 273-297, Sep. 1995. DOI: https://doi.org/10.1007/BF00994018
Y. Freund and R. E. Schapire, "A desicion-theoretic generalization of on-line learning and an application to boosting," in European Conference on Computational Learning Theory, Mar. 1995, pp. 23-37. DOI: https://doi.org/10.1007/3-540-59119-2_166
P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part-Based Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sep. 2010. DOI: https://doi.org/10.1109/TPAMI.2009.167
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," in Advances in Neural Information Processing Systems 25, Lake Tahoe, NV, USA, Dec. 2012, pp. 1097-1105.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, Jun. 2014, pp. 580-587. DOI: https://doi.org/10.1109/CVPR.2014.81
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015. DOI: https://doi.org/10.1038/nature14539
K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," presented at the International Conference on Learning Representations, May 2015, arXiv: abs/1409.1556.
C. Szegedy et al., "Going Deeper With Convolutions," presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, Jun. 7-12, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594
K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778. DOI: https://doi.org/10.1109/CVPR.2016.90
D. Virmani, P. Girdhar, P. Jain, and P. Bamdev, "FDREnet: Face Detection and Recognition Pipeline," Engineering, Technology & Applied Science Research, vol. 9, no. 2, pp. 3933-3938, Apr. 2019. DOI: https://doi.org/10.48084/etasr.2492
L. Liu et al., "Deep Learning for Generic Object Detection: A Survey," International Journal of Computer Vision, vol. 128, no. 2, pp. 261-318, Feb. 2020. DOI: https://doi.org/10.1007/s11263-019-01247-4
R. Girshick, "Fast R-CNN," in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440-1448. DOI: https://doi.org/10.1109/ICCV.2015.169
S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, Jun. 2017. DOI: https://doi.org/10.1109/TPAMI.2016.2577031
K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN," in 2017 IEEE International Conference on Computer Vision (ICCV), Oct. 2017, pp. 2980-2988. DOI: https://doi.org/10.1109/ICCV.2017.322
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 779-788. DOI: https://doi.org/10.1109/CVPR.2016.91
W. Liu et al., "SSD: Single Shot MultiBox Detector," in European Conference on Computer Vision - ECCV 2016, pp. 21-37. DOI: https://doi.org/10.1007/978-3-319-46448-0_2
J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp. 7263-7271. DOI: https://doi.org/10.1109/CVPR.2017.690
J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 6517-6525. DOI: https://doi.org/10.1109/CVPR.2017.690
S. Du, M. Ibrahim, M. Shehata, and W. Badawy, "Automatic License Plate Recognition (ALPR): A State-of-the-Art Review," IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 2, pp. 311-325, Feb. 2013. DOI: https://doi.org/10.1109/TCSVT.2012.2203741
J. Han, J. Yao, J. Zhao, J. Tu, and Y. Liu, "Multi-Oriented and Scale-Invariant License Plate Detection Based on Convolutional Neural Networks," Sensors, vol. 19, no. 5, p. 1175, Jan. 2019. DOI: https://doi.org/10.3390/s19051175
L. Xie, T. Ahmad, L. Jin, Y. Liu, and S. Zhang, "A New CNN-Based Method for Multi-Directional Car License Plate Detection," IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 2, pp. 507-517, Feb. 2018. DOI: https://doi.org/10.1109/TITS.2017.2784093
S. G. Kim, H. G. Jeon, and H. I. Koo, "Deep-learning-based license plate detection method using vehicle region extraction," Electronics Letters, vol. 53, no. 15, pp. 1034-1036, Jun. 2017. DOI: https://doi.org/10.1049/el.2017.1373
W. Min, X. Li, Q. Wang, Q. Zeng, and Y. Liao, "New approach to vehicle license plate location based on new model YOLO-L and plate pre-identification," IET Image Processing, vol. 13, no. 7, pp. 1041-1049, Mar. 2019. DOI: https://doi.org/10.1049/iet-ipr.2018.6449
R. Laroca et al., "A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector," in 2018 International Joint Conference on Neural Networks (IJCNN), Jul. 2018, pp. 1-10. DOI: https://doi.org/10.1109/IJCNN.2018.8489629
Z. Xu et al., "Towards End-to-End License Plate Detection and Recognition: A Large Dataset and Baseline," in European Conference on Computer Vision - ECCV 2018, 2018, pp. 261-277. DOI: https://doi.org/10.1007/978-3-030-01261-8_16
M. R. Asif, Q. Chun, S. Hussain, M. S. Fareed, and S. Khan, "Multinational vehicle license plate detection in complex backgrounds," Journal of Visual Communication and Image Representation, vol. 46, pp. 176-186, Jul. 2017. DOI: https://doi.org/10.1016/j.jvcir.2017.03.020
N. Dorbe, A. Jaundalders, R. Kadikis, and K. Nesenbergs, "FCN and LSTM Based Computer Vision System for Recognition of Vehicle Type, License Plate Number, and Registration Country," Automatic Control and Computer Sciences, vol. 52, no. 2, pp. 146-154, Mar. 2018. DOI: https://doi.org/10.3103/S0146411618020104
C. Henry, S. Y. Ahn, and S.-W. Lee, "Multinational License Plate Recognition Using Generalized Character Sequence Detection," IEEE Access, vol. 8, pp. 35185-35199, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2974973
H. Li, P. Wang, and C. Shen, "Toward End-to-End Car License Plate Detection and Recognition With Deep Neural Networks," IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 3, pp. 1126-1136, Mar. 2019. DOI: https://doi.org/10.1109/TITS.2018.2847291
J. Yépez, R. D. Castro-Zunti, and S. B. Ko, "Deep learning-based embedded license plate localisation system," IET Intelligent Transport Systems, vol. 13, no. 10, pp. 1569-1578, Jul. 2019. DOI: https://doi.org/10.1049/iet-its.2019.0082
M. R. Asif, C. Qi, T. Wang, M. S. Fareed, and S. A. Raza, "License plate detection for multi-national vehicles: An illumination invariant approach in multi-lane environment," Computers & Electrical Engineering, vol. 78, pp. 132-147, Sep. 2019. DOI: https://doi.org/10.1016/j.compeleceng.2019.07.012
Z. Selmi, M. B. Halima, U. Pal, and M. A. Alimi, "DELP-DAR system for license plate detection and recognition," Pattern Recognition Letters, vol. 129, pp. 213-223, Jan. 2020. DOI: https://doi.org/10.1016/j.patrec.2019.11.007
S. Park, H. Yoon, and S. Park, "Multi-Style License Plate Recognition System using K-Nearest Neighbors," KSII Transactions on Internet and Information Systems, vol. 13, no. 5, pp. 2509-2528, May 2019. DOI: https://doi.org/10.3837/tiis.2019.05.015
Caltech Computational Vision: Archive, California Institute of Technology, 1999. [Online]. Available: http://www.vision.caltech.edu/html-files/archive.html.
K. Kraupner, "Using Multilayered Perceptron for Recognition of Alphanumeric Characters on License Plates", Ph.D. dissertation, University of Zagreb, Croatia, 2003.
L. Dlagnekov, "Video-Based Car Surveillance: License Plate, Make, and Model Recognition", M. S. thesis, University of California, San Diego, 2005.
O. Martinsky, "Algorithmic and Mathematical Principles of Automatic Number Plate Recognition Systems", B.Sc. thesis, Brno University of Technology, Croatia, 2007.
Medialab LPR Database, Multimedia Technology Laboratory, National Technical University of Athens, Greece. [Online]. Available: http://www.medialab.ntua.gr/research/LPRdatabase.html.
J. Spanhel, J. Sochor, R. Juranek, A. Herout, L. Marsík, and P. Zemcik, "Holistic recognition of low quality license plates by CNN using track annotated data," in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Aug. 2017, pp. 1-6. DOI: https://doi.org/10.1109/AVSS.2017.8078501
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
C. M. Bishop, Pattern Recognition and Machine Learning, 1st ed., New York, NY, USA: Springer, 2006.
S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," in Proceedings of the 32nd International Conference on Machine Learning, Jun. 2015, pp. 448-456.
Downloads
How to Cite
License
Copyright (c) 2020 Authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
