Video Level Sign Language Recognition with Key Frame Extraction Using Adaptive Convolution Neural Networks with a New Activation Function

Navyasri Mullapudi; G. Jaya Suma

doi:10.48084/etasr.14023

Authors

Navyasri Mullapudi Department of Computer Science and Engineering, JNTUK, Andhra Pradesh, 533003, India
G. Jaya Suma Department of Information Technology, Gurajada University, Andhra Pradesh, 535003, India

Volume: 15 | Issue: 6 | Pages: 30356-30361 | December 2025 | https://doi.org/10.48084/etasr.14023

Received: 12 August 2025 | Revised: 27 August 2025, 16 September 2025, and 18 September 2025 | Accepted: 21 September 2025 | Online: 10 November 2025

Corresponding author: Navyasri Mullapudi

Abstract

This paper proposes a deep learning architecture with a novel activation function in video-level sign language recognition. Samples from a video dataset of deaf-mute people were divided into multiple frames, and a new extraction algorithm is proposed in order to select and extract key frames from the videos. Adaptive Convolution Neural Networks (CNNs) utilizing a novel activation function were trained with the extracted video frames. The the high accuracy of the proposed method was verified in terms of precision, recall, f1-score, and accuracy.

Keywords:

sign language recognition, convolution neural networks, video frame extraction, activation function

Downloads

Download data is not yet available.

References

N. K. Kahlon and W. Singh, "Machine translation from text to sign language: a systematic review," Universal Access in the Information Society, vol. 22, no. 1, pp. 1–35, Mar. 2023. DOI: https://doi.org/10.1007/s10209-021-00823-1

A. H. Kugate et al., "Efficient Key Frame Extraction from Videos Using Convolutional Neural Networks and Clustering Techniques," EAI Endorsed Transactions on Context-aware Systems and Applications, vol. 10, Jul. 2024. DOI: https://doi.org/10.4108/eetcasa.5131

A. Ajit, K. Acharya, and A. Samanta, "A Review of Convolutional Neural Networks," in 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), Vellore, India, Oct. 2020, pp. 1–5. DOI: https://doi.org/10.1109/ic-ETITE47903.2020.049

B. Fang, J. Co, and M. Zhang, "DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation," in Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, New York, NY, USA, Aug. 2017, pp. 1–13. DOI: https://doi.org/10.1145/3131672.3131693

A. Khan et al., "Deep Learning Approaches for Continuous Sign Language Recognition: A Comprehensive Review," IEEE Access, vol. 13, pp. 55524–55544, 2025. DOI: https://doi.org/10.1109/ACCESS.2025.3554046

S. Renjith and R. Manazhy, "Indian Sign Language Recognition: A Comparative Analysis Using CNN and RNN Models," in 2023 International Conference on Circuit Power and Computing Technologies (ICCPCT), Kollam, India, Dec. 2023, pp. 1573–1576. DOI: https://doi.org/10.1109/ICCPCT58313.2023.10245525

K. Nimisha and A. Jacob, "A Brief Review of the Recent Trends in Sign Language Recognition," in 2020 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, Jul. 2020, pp. 186–190. DOI: https://doi.org/10.1109/ICCSP48568.2020.9182351

R. K. Vasanthakumari, R. V. Nair, and V. G. Krishnappa, "Improved learning by using a modified activation function of a Convolutional Neural Network in multi-spectral image classification," Machine Learning with Applications, vol. 14, Dec. 2023, Art. no. 100502. DOI: https://doi.org/10.1016/j.mlwa.2023.100502

B. Khagi and G.-R. Kwon, "A novel scaled-gamma-tanh (SGT) activation function in 3D CNN applied for MRI classification," Scientific Reports, vol. 12, no. 1, Sep. 2022, Art. no. 14978. DOI: https://doi.org/10.1038/s41598-022-19020-y

R. Avenash and P. Viswanath, "Semantic Segmentation of Satellite Images using a Modified CNN with Hard-Swish Activation Function," presented at the 14th International Conference on Computer Vision Theory and Applications, Nov. 2025, pp. 413–420, Nov. 03, 2025.

I. D. Khan, O. Farooq, and Y. U. Khan, "Automatic Seizure Detection Using Modified CNN Architecture and Activation Layer," Journal of Physics: Conference Series, vol. 2318, no. 1, Dec. 2022, Art. no. 012013. DOI: https://doi.org/10.1088/1742-6596/2318/1/012013

R. ZahediNasab and H. Mohseni, "Neuroevolutionary based convolutional neural network with adaptive activation functions," Neurocomputing, vol. 381, pp. 306–313, Mar. 2020. DOI: https://doi.org/10.1016/j.neucom.2019.11.090

V. S. Bawa and V. Kumar, "Linearized sigmoidal activation: A novel activation function with tractable non-linear characteristics to boost representation capability," Expert Systems with Applications, vol. 120, pp. 346–356, Apr. 2019. DOI: https://doi.org/10.1016/j.eswa.2018.11.042

P. Shet, M. Srinivas, C. Madhav, and R. Likhith, "Indian Sign Language Video Dataset." [Online]. Available: https://www.kaggle.com/datasets/prasadshet/indian-sign-language-video-dataset.

M. S. Aiswarya and R. Arockia Xavier Annie, "Keyframe Extraction Algorithm for Continuous Sign-Language Videos Using Angular Displacement and Sequence Check Metrics," International Journal of Intelligent Systems, vol. 2024, no. 1, 2024, Art. no. 4725216. DOI: https://doi.org/10.1155/2024/4725216

N. Devabathini and P. Mathivanan, "Sign Language Recognition Through Video Frame Feature Extraction using Transfer Learning and Neural Networks," in 2023 International Conference on Next Generation Electronics (NEleX), Vellore, India, Sep. 2023, pp. 1–6. DOI: https://doi.org/10.1109/NEleX59773.2023.10421383

M. Navyasri and G. J. Suma, "Digit Recognition of Hand Gesture Images in Sign Language Using Convolution Neural Network Classification Algorithm," in Recent Advances in Electrical and Electronic Engineering, (ICSTE 2023), 2024, pp. 337–345. DOI: https://doi.org/10.1007/978-981-99-4713-3_32

M. Navyasri and G. J. Suma, "A novel key frame extraction using a deep learning model for sign language recognition on videos," Multimedia Tools and Applications, Oct. 2025. DOI: https://doi.org/10.1007/s11042-025-21134-0

L. Xiangyang, Q. Xing, Z. Han, and C. Feng, "A Novel Activation Function of Deep Neural Network," Scientific Programming, vol. 2023, no. 1, 2023, Art. no. 3873561. DOI: https://doi.org/10.1155/2023/3873561

P. Bohra, J. Campos, H. Gupta, S. Aziznejad, and M. Unser, "Learning Activation Functions in Deep (Spline) Neural Networks," IEEE Open Journal of Signal Processing, vol. 1, pp. 295–309, 2020. DOI: https://doi.org/10.1109/OJSP.2020.3039379

M. A. Mohamed, H. A. Hassan, M. H. Essai, H. Esmaiel, A. S. Mubarak, and O. A. Omer, "Modified state activation functions of deep learning-based SC-FDMA channel equalization system," EURASIP Journal on Wireless Communications and Networking, vol. 2023, no. 1, Nov. 2023, Art. no. 115. DOI: https://doi.org/10.1186/s13638-023-02326-4

A. O. Hashi, S. Z. M. Hashim, and A. B. Asamah, "Dynamic Adaptation in Deep Learning for Enhanced Hand Gesture Recognition," Engineering, Technology & Applied Science Research, vol. 14, no. 4, pp. 15836–15841, Aug. 2024. DOI: https://doi.org/10.48084/etasr.7670

W. H. Lee, J. L. Tan, Z. A. A. Salam, H. Y. Teoh, Q. J. Lee, and L. T. S. Suzanne, "Sign language recognition based on CNN with optimized activation function," Journal of Applied Technology and Innovation, vol. 8, no. 1, pp. 9–14, 2024.

L. Pigou, S. Dieleman, P.-J. Kindermans, and B. Schrauwen, "Sign Language Recognition Using Convolutional Neural Networks," in Computer Vision - ECCV 2014 Workshops, 2015, pp. 572–578. DOI: https://doi.org/10.1007/978-3-319-16178-5_40

Md. M. Rahman, Md. S. Islam, Md. H. Rahman, R. Sassi, M. W. Rivolta, and M. Aktaruzzaman, "A New Benchmark on American Sign Language Recognition using Convolutional Neural Network," in 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), Dhaka, Bangladesh, Sep. 2019, pp. 1–6. DOI: https://doi.org/10.1109/STI47673.2019.9067974