A Data Acquisiton System with sEMG Signal and Camera Images for Finger Classification with Machine Learning Algorithms
Received: 8 February 2024 | Revised: 20 February 2024 | Accepted: 23 February 2024 | Online: 6 March 2024
Corresponding author: Ismail Mersinkaya
Abstract
Advances in robotics and biomedical engineering have expanded the possibilities of Human-Computer Interaction (HCI) in the last few years. The identification of hand movements is the accurate and real-time signal acquisition of hand movements through the use of image-based systems and surface electromyography sensors. This study uses multithreading to record motion signals from the forearm muscles in conjunction with a surface electromyography (sEMG) sensor and a camera image. The finger movement information labels were tabulated and analyzed along with the simultaneous acquisition of surface electromyography signals and these gestures through the camera. After the acquisition, signal processing techniques were applied to the sEMG signal markered from the camera. Therefore, once the interface is established, data sets suitable for machine learning can be generated.
Keywords:
sEMG, image processing, signal processing, real-time data acquisitionDownloads
References
S. Husnain and R. Abdulkader, "Fractional Order Modeling and Control of an Articulated Robotic Arm," Engineering, Technology & Applied Science Research, vol. 13, no. 6, pp. 12026–12032, Dec. 2023.
Md. R. Ahsan, M. Ibrahimy, and O. Khalifa, "EMG Signal Classification for Human Computer Interaction A Review," European Journal of Scientific Research, vol. 33, pp. 480–501, Jan. 2009.
R. Zhou, K. Wang, and M. Li, "Design of a sEMG Signal Acquisition Instrument for Physical Rehabilitation Training," in 2017 10th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, Sep. 2017, vol. 2, pp. 136–139.
A. Khan, S. Khusro, and I. Alam, "BlindSense: An Accessibility-inclusive Universal User Interface for Blind People," Engineering, Technology & Applied Science Research, vol. 8, no. 2, pp. 2775–2784, Apr. 2018.
M. Abdul-Niby, M. Farhat, M. Abdullah, and A. Nazzal, "A Low Cost Automated Weather Station for Real Time Local Measurements," Engineering, Technology & Applied Science Research, vol. 7, no. 3, pp. 1615–1618, Jun. 2017.
K. Englehart, B. Hudgin, and P. A. Parker, "A wavelet-based continuous classification scheme for multifunction myoelectric control," IEEE Transactions on Biomedical Engineering, vol. 48, no. 3, pp. 302–311, Mar. 2001.
J. Amezquita-Garcia, M. Bravo-Zanoguera, F. F. Gonzalez-Navarro, R. Lopez-Avitia, and M. A. Reyna, "Applying Machine Learning to Finger Movements Using Electromyography and Visualization in Opensim," Sensors, vol. 22, no. 10, Jan. 2022, Art. no. 3737.
"Gesture Recognition," MediaPipe Studio. https://mediapipe-studio.webapps.google.com/demo/gesture_recognizer.
K. Englehart and B. Hudgins, "A robust, real-time control scheme for multifunction myoelectric control," IEEE Transactions on Biomedical Engineering, vol. 50, no. 7, pp. 848–854, Jul. 2003.
L. H. Smith, L. J. Hargrove, B. A. Lock, and T. A. Kuiken, "Determining the Optimal Window Length for Pattern Recognition-Based Myoelectric Control: Balancing the Competing Effects of Classification Error and Controller Delay," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 19, no. 2, pp. 186–192, Apr. 2011.
K. Xing, P. Yang, J. Huang, Y. Wang, and Q. Zhu, "A real-time EMG pattern recognition method for virtual myoelectric hand control," Neurocomputing, vol. 136, pp. 345–355, Jul. 2014.
M. Ariyanto et al., "Finger movement pattern recognition method using artificial neural network based on electromyography (EMG) sensor," in 2015 International Conference on Automation, Cognitive Science, Optics, Micro Electro-Mechanical System, and Information Technology (ICACOMIT), Bandung, Indonesia, Jul. 2015, pp. 12–17.
K. H. Lee, J. Y. Min, and S. Byun, "Electromyogram-Based Classification of Hand and Finger Gestures Using Artificial Neural Networks," Sensors, vol. 22, no. 1, Jan. 2022, Art. no. 225.
S. Bhagwat and P. Mukherji, "Electromyogram (EMG) based fingers movement recognition using sparse filtering of wavelet packet coefficients," Sādhanā, vol. 45, no. 1, Dec. 2019, Art. no. 3.
S. Kim et al., "Development of an Armband EMG Module and a Pattern Recognition Algorithm for the 5-Finger Myoelectric Hand Prosthesis," International Journal of Precision Engineering and Manufacturing, vol. 20, no. 11, pp. 1997–2006, Nov. 2019.
M. A. Oskoei and H. Hu, "GA-based Feature Subset Selection for Myoelectric Classification," in 2006 IEEE International Conference on Robotics and Biomimetics, Sep. 2006, pp. 1465–1470.
A. Saikia et al., "Combination of EMG Features and Stability Index for Finger Movements Recognition," Procedia Computer Science, vol. 133, pp. 92–98, Jan. 2018.
P. Phukpattaranont, S. Thongpanja, K. Anam, A. Al-Jumaily, and C. Limsakul, "Evaluation of feature extraction techniques and classifiers for finger movement recognition using surface electromyography signal," Medical & Biological Engineering & Computing, vol. 56, no. 12, pp. 2259–2271, Dec. 2018.
H. Su, S. E. Ovur, X. Zhou, W. Qi, G. Ferrigno, and E. De Momi, "Depth vision guided hand gesture recognition using electromyographic signals," Advanced Robotics, vol. 34, no. 15, pp. 985–997, Aug. 2020.
H. Zhou et al., "Toward Hand Pattern Recognition in Assistive and Rehabilitation Robotics Using EMG and Kinematics," Frontiers in Neurorobotics, vol. 15, 2021, Art. no. 659876.
E. Ceolini et al., "Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing," Frontiers in Neuroscience, vol. 14, 2020, Art. no. 00637.
J. Li, J. Zhong, and N. Wang, "A multimodal human-robot sign language interaction framework applied in social robots," Frontiers in Neuroscience, vol. 17, 2023, Art. no. 1168888.
Q. Song, X. Ma, and Y. Liu, "Continuous online prediction of lower limb joints angles based on sEMG signals by deep learning approach," Computers in Biology and Medicine, vol. 163, Sep. 2023, Art. no. 107124.
G. Amprimo, C. Ferraris, G. Masi, G. Pettiti, and L. Priano, "GMH-D: Combining Google MediaPipe and RGB-Depth Cameras for Hand Motor Skills Remote Assessment," in 2022 IEEE International Conference on Digital Health (ICDH), Barcelona, Spain, Jul. 2022, pp. 132–141.
Downloads
How to Cite
License
Copyright (c) 2024 Ismail Mersinkaya, Ahmet Resit Kavsaoglu
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
