Dynamic Adaptation in Deep Learning for Enhanced Hand Gesture Recognition
Received: 28 April 2024 | Revised: 26 May 2024 | Accepted: 27 May 2024 | Online: 2 August 2024
Corresponding author: Abdirahman Osman Hashi
Abstract
The field of Human-Computer Interaction (HCI) is progressing quickly with the incorporation of gesture recognition, which requires advanced systems capable of comprehending intricate human movements. This study introduces a new Dynamic Adaptation Convolutional Neural Network (DACNN) that can adjust to different human hand shapes, orientations, and sizes. This allows for more accurate identification of hand gestures over a wide range of variations. The proposed model includes a thorough process of collecting and preparing data from the Sign Language MNIST dataset. This is followed by a strong data augmentation procedure that provides a wide variety of realistic variations. The architecture utilizes sophisticated convolutional layers to leverage the capabilities of deep learning to extract and synthesize essential gesture features. A rigorous training procedure, supplemented with a ReduceLROnPlateau callback, was used to assure the model's generalization and efficiency. The experimental findings provide remarkable results, showing a substantial accuracy of 99% in categorizing a wide range of hand movements. This study makes a significant contribution to the field of hand gesture recognition by introducing morphological operations, thus enriching input data quality and expanding the model's applicability in diverse HCI environments.
Keywords:
hand gesture gecognition, human-computer interaction, deep learning, neural network architecture, morphological data processing, adaptive learning systems, real-time gesture analysisDownloads
References
R. Rastgoo, K. Kiani, S. Escalera, and M. Sabokrou, "Multi-modal zero-shot dynamic hand gesture recognition," Expert Systems with Applications, vol. 247, Aug. 2024, Art. no. 123349.
L. I. Barona López, F. M. Ferri, J. Zea, Á. L. Valdivieso Caraguay, and M. E. Benalcázar, "CNN-LSTM and post-processing for EMG-based hand gesture recognition," Intelligent Systems with Applications, vol. 22, Jun. 2024, Art. no. 200352.
H. Liang, L. Fei, S. Zhao, J. Wen, S. Teng, and Y. Xu, "Mask-guided multiscale feature aggregation network for hand gesture recognition," Pattern Recognition, vol. 145, Jan. 2024, Art. no. 109901.
Z. Wang et al., "Optimization of inter-subject sEMG-based hand gesture recognition tasks using unsupervised domain adaptation techniques," Biomedical Signal Processing and Control, vol. 92, Jun. 2024, Art. no. 106086.
A. Bahuguna, G. Bhaumik, and M. C. Govil, "Local Extrema Min-Max Pattern: A novel descriptor for extracting compact and discrete features for hand gesture recognition," Biomedical Signal Processing and Control, vol. 93, Jul. 2024, Art. no. 106203.
P. Balaji and M. Ranjan Prusty, "Multimodal fusion hierarchical self-attention network for dynamic hand gesture recognition," Journal of Visual Communication and Image Representation, vol. 98, Feb. 2024, Art. no. 104019.
Z. Zhang, Q. Shen, and Y. Wang, "Electromyographic hand gesture recognition using convolutional neural network with multi-attention," Biomedical Signal Processing and Control, vol. 91, May 2024, Art. no. 105935.
T. R. Gadekallu et al., "Hand gesture recognition based on a Harris Hawks optimized Convolution Neural Network," Computers and Electrical Engineering, vol. 100, May 2022, Art. no. 107836.
T. Triwiyanto, I. P. A. Pawana, and W. Caesarendra, "Deep learning approach to improve the recognition of hand gesture with multi force variation using electromyography signal from amputees," Medical Engineering & Physics, vol. 125, Mar. 2024, Art. no. 104131.
T. M. Tai, Y. J. Jhang, Z. W. Liao, K. C. Teng, and W. J. Hwang, "Sensor-Based Continuous Hand Gesture Recognition by Long Short-Term Memory," IEEE Sensors Letters, vol. 2, no. 3, pp. 1–4, Sep. 2018.
D. Sarma and M. K. Bhuyan, "Methods, Databases and Recent Advancement of Vision-Based Hand Gesture Recognition for HCI Systems: A Review," SN Computer Science, vol. 2, no. 6, Aug. 2021, Art. no. 436.
T. L. Dang, S. D. Tran, T. H. Nguyen, S. Kim, and N. Monet, "An improved hand gesture recognition system using keypoints and hand bounding boxes," Array, vol. 16, Dec. 2022, Art. no. 100251.
Q. Gao, Y. Chen, Z. Ju, and Y. Liang, "Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human–Robot Interaction," IEEE Sensors Journal, vol. 22, no. 18, pp. 17421–17430, Sep. 2022.
A. Mujahid et al., "Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model," Applied Sciences, vol. 11, no. 9, Jan. 2021, Art. no. 4164.
H. G. Doan and N. T. Nguyen, "Fusion Machine Learning Strategies for Multi-modal Sensor-based Hand Gesture Recognition," Engineering, Technology & Applied Science Research, vol. 12, no. 3, pp. 8628–8633, Jun. 2022.
S. Wang et al., "Hand gesture recognition framework using a lie group based spatio-temporal recurrent network with multiple hand-worn motion sensors," Information Sciences, vol. 606, pp. 722–741, Aug. 2022.
R. M. Badiger, R. Yakkundimath, G. Konnurmath, and P. M. Dhulavvagol, "Deep Learning Approaches for Age-based Gesture Classification in South Indian Sign Language," Engineering, Technology & Applied Science Research, vol. 14, no. 2, pp. 13255–13260, Apr. 2024.
C. Xia, A. Saito, and Y. Sugiura, "Using the virtual data-driven measurement to support the prototyping of hand gesture recognition interface with distance sensor," Sensors and Actuators A: Physical, vol. 338, May 2022, Art. no. 113463.
W. Zhou and K. Chen, "A lightweight hand gesture recognition in complex backgrounds," Displays, vol. 74, Sep. 2022, Art. no. 102226.
D. Chopra, N. Joshi, and I. Mathur, "A Review on Machine Translation in Indian Languages," Engineering, Technology & Applied Science Research, vol. 8, no. 5, pp. 3475–3478, Oct. 2018.
S. R. Bose and V. S. Kumar, "In-situ recognition of hand gesture via Enhanced Xception based single-stage deep convolutional neural network," Expert Systems with Applications, vol. 193, May 2022, Art. no. 116427.
Z. Wang, G. Li, and L. Yang, "Dynamic Hand Gesture Recognition Based on Micro-Doppler Radar Signatures Using Hidden Gauss–Markov Models," IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 2, pp. 291–295, Oct. 2021.
M. Al-Hammadi et al., "Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation," IEEE Access, vol. 8, pp. 192527–192542, 2020.
"Sign Language MNIST." [Online]. Available: https://www.kaggle.com/datasets/datamunge/sign-language-mnist.
L. K. S. Tolentino, R. O. S. Juan, A. C. Thio-ac, M. A. B. Pamahoy, J. R. R. Forteza, and X. J. O. Garcia, "Static Sign Language Recognition Using Deep Learning," International Journal of Machine Learning and Computing, vol. 9, no. 6, pp. 821–827, Dec. 2019.
S. Chavan, X. Yu, and J. Saniie, "Convolutional Neural Network Hand Gesture Recognition for American Sign Language," in 2021 IEEE International Conference on Electro Information Technology (EIT), Mt. Pleasant, MI, USA, May 2021, pp. 188–192.
S. Joudaki and A. Rehman, "Dynamic hand gesture recognition of sign language using geometric features learning," International Journal of Computational Vision and Robotics, vol. 12, no. 1, pp. 1–16, Jan. 2022.
Downloads
How to Cite
License
Copyright (c) 2024 Abdirahman Osman Hashi, Siti Zaiton Mohd Hashim, Azurah Bte Asamah
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
