A Two-Stream Convolutional Attention Network for Hand Gesture Recognition and Classification
Received: 17 September 2025 | Revised: 6 November 2025 | Accepted: 15 November 2025 | Online: 9 February 2026
Corresponding author: Aekkarat Suksukont
Abstract
This study presents a two-stream convolutional attention network for hand gesture recognition and classification, with dual branches that effectively extract a range of spatial features. To further improve its capabilities, Convolutional Neural Network (CNN), Residual Blocks (RB), Attention Mechanism (AM), and Convolutional Block Attention Module (CBAM) are integrated, enabling the network to focus on specific regions of the input while reducing background noise. This structure allows the model to capture variations in hand gestures more effectively. Experiments utilized the Hand-Sign-Images Dataset (HSID) and the Hand Gesture Dataset (HGD), each offering a diverse array of hand gesture patterns. The results indicate that the model achieved high training accuracies of 100% and 98.68%, demonstrating its effectiveness in recognizing complex hand gestures. The proposed approach not only enhances recognition and classification accuracy but also offers strong adaptability, making it suitable for applications such as robot control and assistive communication systems.
Keywords:
convolutional neural network, attention mechanism, convolutional block attention module, hand gesture recognition, hand gesture classificationDownloads
References
S. Yadav and S. Jain, "Gesture Recognition System for Human-Computer Interaction using Computer Vision," in 2024 11th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Mar. 2024, pp. 1–4. DOI: https://doi.org/10.1109/ICRITO61523.2024.10522212
A. S. Editya, N. Kurniati, M. M. Alamin, A. Lisdiyanto, and A. L. Pramana, "Realtime of Hand Gesture Recognition for Telerobotics Controller Based Leap Motion Using Random Forest," in 2023 International Conference on Advanced Mechatronics, Intelligent Manufacture and Industrial Automation (ICAMIMIA), Aug. 2023, pp. 764–768. DOI: https://doi.org/10.1109/ICAMIMIA60881.2023.10427773
J. Li, C. Li, J. Han, Y. Shi, G. Bian, and S. Zhou, "Robust Hand Gesture Recognition Using HOG-9ULBP Features and SVM Model," Electronics, vol. 11, no. 7, Jan. 2022, Art. no. 988. DOI: https://doi.org/10.3390/electronics11070988
M. A. A. Razak, F. Y. A. Rahman, R. Mohamad, S. Shahbuddin, Y. W. M. Yusof, and S. I. Suliman, "Hand Gesture Recognition based on Convolution Neural Network (CNN) and Support Vector Machine (SVM)," in 2023 IEEE 14th Control and System Graduate Research Colloquium (ICSGRC), Dec. 2023, pp. 123–126. DOI: https://doi.org/10.1109/ICSGRC57744.2023.10215427
H. G. Doan and N. T. Nguyen, "Fusion Machine Learning Strategies for Multi-modal Sensor-based Hand Gesture Recognition," Engineering, Technology & Applied Science Research, vol. 12, no. 3, pp. 8628–8633, June 2022. DOI: https://doi.org/10.48084/etasr.4913
A. O. Hashi, S. Z. M. Hashim, and A. B. Asamah, "Dynamic Adaptation in Deep Learning for Enhanced Hand Gesture Recognition," Engineering, Technology & Applied Science Research, vol. 14, no. 4, pp. 15836–15841, Aug. 2024. DOI: https://doi.org/10.48084/etasr.7670
F. S. Khan, M. N. H. Mohd, D. M. Soomro, S. Bagchi, and M. D. Khan, "3D Hand Gestures Segmentation and Optimized Classification Using Deep Learning," IEEE Access, vol. 9, pp. 131614–131624, 2021. DOI: https://doi.org/10.1109/ACCESS.2021.3114871
A. Fatayer, W. Gao, and Y. Fu, "sEMG-Based Gesture Recognition Using Deep Learning From Noisy Labels," IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 9, pp. 4462–4473, Sept. 2022. DOI: https://doi.org/10.1109/JBHI.2022.3179630
T. D. Qi, F. L. Cibrian, M. Raswan, T. Kay, H. M. Camarillo-Abad, and Y. Wen, "Toward Intuitive 3D Interactions in Virtual Reality: A Deep Learning- Based Dual-Hand Gesture Recognition Approach," IEEE Access, vol. 12, pp. 67438–67452, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3400295
L. Qu, H. Wu, T. Yang, L. Zhang, and Y. Sun, "Dynamic Hand Gesture Classification Based on Multichannel Radar Using Multistream Fusion 1-D Convolutional Neural Network," IEEE Sensors Journal, vol. 22, no. 24, pp. 24083–24093, Sept. 2022. DOI: https://doi.org/10.1109/JSEN.2022.3216604
A. Osman Hashi, S. Zaiton Mohd Hashim, and A. Bte Asamah, "A Systematic Review of Hand Gesture Recognition: An Update From 2018 to 2024," IEEE Access, vol. 12, pp. 143599–143626, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3421992
S. Jiang, P. Kang, X. Song, B. P. L. Lo, and P. B. Shull, "Emerging Wearable Interfaces and Algorithms for Hand Gesture Recognition: A Survey," IEEE Reviews in Biomedical Engineering, vol. 15, pp. 85–102, 2022. DOI: https://doi.org/10.1109/RBME.2021.3078190
V. Madaan, N. Sharma, D. Gupta, and S. Aluvala, "Hand Gesture Detection Using VGG-16," in 2024 International Conference on Information Science and Communications Technologies (ICISCT), Aug. 2024, pp. 491–495. DOI: https://doi.org/10.1109/ICISCT64202.2024.10956830
Y. Gu et al., "WiGRUNT: WiFi-Enabled Gesture Recognition Using Dual-Attention Network," IEEE Transactions on Human-Machine Systems, vol. 52, no. 4, pp. 736–746, Dec. 2022. DOI: https://doi.org/10.1109/THMS.2022.3163189
D. R. T. Hax, P. Penava, S. Krodel, L. Razova, and R. Buettner, "A Novel Hybrid Deep Learning Architecture for Dynamic Hand Gesture Recognition," IEEE Access, vol. 12, pp. 28761–28774, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3365274
M. Garg, D. Ghosh, and P. M. Pradhan, "Multiscaled Multi-Head Attention-Based Video Transformer Network for Hand Gesture Recognition," IEEE Signal Processing Letters, vol. 30, pp. 80–84, 2023. DOI: https://doi.org/10.1109/LSP.2023.3241857
M. A. A. Mosleh and A. H. Gumaei, "An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models," IEEE Access, vol. 12, pp. 191030–191045, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3512455
E. Yenisari and S. Yavuz, "Deep Learning-Based Sign Language Recognition Using Efficient Multi-Feature Attention Mechanism," IEEE Access, vol. 13, pp. 126684–126699, 2025. DOI: https://doi.org/10.1109/ACCESS.2025.3586096
J.-W. Choi, C.-W. Park, and J.-H. Kim, "FMCW Radar-Based Real-Time Hand Gesture Recognition System Capable of Out-of-Distribution Detection," IEEE Access, vol. 10, pp. 87425–87434, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3200757
M. Jaiswal, V. Sharma, A. Sharma, S. Saini, and R. Tomar, "Quantized CNN-based efficient hardware architecture for real-time hand gesture recognition," Microelectronics Journal, vol. 151, Sept. 2024, Art. no. 106345. DOI: https://doi.org/10.1016/j.mejo.2024.106345
Q. Shangguan et al., "A Lightweight CNN Approach for Hand Gesture Recognition via GAF Encoding of A-Mode Ultrasound Signals," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 33, pp. 3734–3743, 2025. DOI: https://doi.org/10.1109/TNSRE.2025.3608180
X. Dai, Z. Zhang, G. Liu, and J. Cai, "Application of CNN-LSTM feature fusion architecture based on self-made data glove in gesture recognition," Biomedical Signal Processing and Control, vol. 112, Feb. 2026, Art. no. 108573. DOI: https://doi.org/10.1016/j.bspc.2025.108573
M. Zakariah and A. Alnuaim, "Recognizing human activities with the use of Convolutional Block Attention Module," Egyptian Informatics Journal, vol. 27, Sept. 2024, Art. no. 100536. DOI: https://doi.org/10.1016/j.eij.2024.100536
B. Niu, J. Li, and Y. Wang, "A Simplified Convolutional Block Attention Module for Robust Hand Gesture Recognition with High Density Surface Electromyography," in 2025 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), July 2025, pp. 1–7. DOI: https://doi.org/10.1109/AIM64088.2025.11175815
J. P. Sahoo, S. P. Sahoo, S. Ari, and S. K. Patra, "Hand Gesture Recognition Using Densely Connected Deep Residual Network and Channel Attention Module for Mobile Robot Control," IEEE Transactions on Instrumentation and Measurement, vol. 72, 2023, Art. no. 5008011. DOI: https://doi.org/10.1109/TIM.2023.3246488
Y.-Y. Yang, H.-H. Yang, and J.-K. Yang, "YOLO-LRHG: Long Range Hand Gesture detection using YOLO with attention mechanism," in 2024 IEEE 7th International Conference on Electronic Information and Communication Technology (ICEICT), July 2024, pp. 985–990. DOI: https://doi.org/10.1109/ICEICT61637.2024.10671140
A. Singh. "Hand-Sign-Images." Kaggle. https://www.kaggle.com/datasets/ash2703/handsignimages.
A. Khan. "Sign Language Gesture Images Dataset." Kaggle. https://www.kaggle.com/datasets/ahmedkhanak1995/sign-language-gesture-images-dataset.
T. Mantecón, C. R. del-Blanco, F. Jaureguizar, and N. García, "Hand Gesture Recognition Using Infrared Imagery Provided by Leap Motion Controller," in Advanced Concepts for Intelligent Vision Systems, Cham, 2016, pp. 47–57. DOI: https://doi.org/10.1007/978-3-319-48680-2_5
C. G. Bastidas, K. A. Pérez, Á. L. V. Caraguay, L. I. B. López, and M. E. Benalcázar, "Comparison of Hand Gesture Recognition Models Combining Supervised Learning and Reinforcement Learning," in 2024 IEEE Latin American Conference on Computational Intelligence (LA-CCI), Aug. 2024, pp. 1–11. DOI: https://doi.org/10.1109/LA-CCI62337.2024.10814786
R. Sebbah and F. Z. Chelali, "Using Machine Learning and Deep Learning for Static and Dynamic Hand Gesture Recognition," in 2024 International Conference on Advances in Electrical and Communication Technologies (ICAECOT), July 2024, pp. 1–6. DOI: https://doi.org/10.1109/ICAECOT62402.2024.10828747
A. Dey, S. Biswas, and L. Abualigah, "Umpire’s Signal Recognition in Cricket Using an Attention based DC-GRU Network," International Journal of Engineering, vol. 37, no. 4, pp. 662–674, 2024. DOI: https://doi.org/10.5829/IJE.2024.37.04A.08
A. Dey, S. Biswas, and D.-N. Le, "Recognition of Wh-Question Sign Gestures in Video Streams using an Attention Driven C3D-BiLSTM Network," Procedia Computer Science, vol. 235, pp. 2920–2931, Jan. 2024. DOI: https://doi.org/10.1016/j.procs.2024.04.276
A. S. M. Miah, Md. A. M. Hasan, and J. Shin, "Dynamic Hand Gesture Recognition Using Multi-Branch Attention Based Graph and General Deep Learning Model," IEEE Access, vol. 11, pp. 4703–4716, 2023. DOI: https://doi.org/10.1109/ACCESS.2023.3235368
Downloads
How to Cite
License
Copyright (c) 2025 Chaovarit Janpirom, Bunthida Chunngam, Tanchanok Phewkham, Aekkarat Suksukont
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
