Moar: A Swimmer Motion Swimming Style Identification Model using Deep Learning
Received: 17 October 2024 | Revised: 17 November 2024 | Accepted: 21 November 2024 | Online: 2 February 2025
Corresponding author: Manal Alharbi
Abstract
Athletes in various sports, such as swimming, are increasingly using motion capture to identify and optimize their movement techniques. However, traditional motion capture systems tend to be expensive and limited. Computer vision-based methods have emerged as alternatives to identify four swimming styles: freestyle, backstroke, breaststroke, and butterfly. However, previous models did not identify flaws in swimmer movement. A significant challenge is the lack of labeled swimming video datasets that indicate these flaws. To overcome this challenge, this study collected and labeled a dataset of swimmer flaws and integrated them with the publicly available dataset SwimXYZ. Then, YOLO models were trained on the generated data. The YOLOv8s model demonstrated an impressive mean average precision (mAP@0.50) of 98% in the detection of swimming style and 95% in the simultaneous detection of swimming style and the identification of incorrect movements. This model can be used in real-time applications to help swimmers evaluate and improve the accuracy of their techniques.
Keywords:
human motion capture, artificial intelligence, computer vision, YOLOv8, swimming styles, swimming flaws, incorrect movementDownloads
References
A. Menache, Understanding Motion Capture for Computer Animation. Elsevier, 2011.
T. Baker, "The History of Motion Capture Within The Entertainment Industry," B.S. Thesis, Metropolia University of Applied Sciences, Helsini, Finland, 2020.
C. Bregler, "Motion Capture Technology for Entertainment [In the Spotlight]," IEEE Signal Processing Magazine, vol. 24, no. 6, pp. 160–158, Aug. 2007.
S. Sharma, S. Verma, M. Kumar, and L. Sharma, "Use of Motion Capture in 3D Animation: Motion Capture Systems, Challenges, and Recent Trends," in 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India, Feb. 2019, pp. 289–294.
A. A. Alsuwaylimi, R. Alanazi, S. M. Alanazi, S. M. Alenezi, T. Saidani, and R. Ghodhbani, "Improved and Efficient Object Detection Algorithm based on YOLOv5," Engineering, Technology & Applied Science Research, vol. 14, no. 3, pp. 14380–14386, Jun. 2024.
L. Mündermann, S. Corazza, and T. P. Andriacchi, "The evolution of methods for the capture of human movement leading to markerless motion capture for biomechanical applications," Journal of NeuroEngineering and Rehabilitation, vol. 3, no. 1, Mar. 2006, Art. no. 6.
I. Takeda, A. Yamada, and H. Onodera, "Artificial Intelligence-Assisted motion capture for medical applications: a comparative study between markerless and passive marker motion capture," Computer Methods in Biomechanics and Biomedical Engineering, vol. 24, no. 8, pp. 864–873, Jun. 2021.
G. Bleser, B. Taetz, M. Miezal, C. A. Christmann, D. Steffen, and K. Regenspurger, "Development of an Inertial Motion Capture System for Clinical Application: Potentials and challenges from the technology and application perspectives," i-com, vol. 16, no. 2, pp. 113–129, Aug. 2017.
W. W. T. Lam, Y. M. Tang, and K. N. K. Fong, "A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation," Journal of NeuroEngineering and Rehabilitation, vol. 20, no. 1, May 2023, Art. no. 57.
B. Pueo Ortega and J. M. Jiménez Olmedo, "Application of motion capture technology for sport performance analysis," Retos: nuevas tendencias en educación física, deporte y recreación, no. 32 (2o semestre), pp. 241–247, 2017.
E. van der Kruk and M. M. Reijne, "Accuracy of human motion capture systems for sport applications; state-of-the-art review," European Journal of Sport Science, vol. 18, no. 6, pp. 806–819, Jul. 2018.
G. R. D. Bernardina, T. Monnet, H. T. Pinto, R. M. L. de Barros, P. Cerveri, and A. P. Silvatti, "Are Action Sport Cameras Accurate Enough for 3D Motion Analysis? A Comparison With a Commercial Motion Capture System," Journal of Applied Biomechanics, vol. 35, no. 1, pp. 80–86.
S. Barris and C. Button, "A Review of Vision-Based Motion Analysis in Sport," Sports Medicine, vol. 38, no. 12, pp. 1025–1043, Dec. 2008.
S. Noiumkar and S. Tirakoat, "Use of Optical Motion Capture in Sports Science: A Case Study of Golf Swing," in 2013 International Conference on Informatics and Creative Multimedia, Kuala Lumpur, Malaysia, Sep. 2013, pp. 310–313.
S. L. Colyer, M. Evans, D. P. Cosker, and A. I. T. Salo, "A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System," Sports Medicine - Open, vol. 4, no. 1, Jun. 2018, Art. no. 24.
R. J. Aughey et al., "Comparison of a computer vision system against three-dimensional motion capture for tracking football movements in a stadium environment," Sports Engineering, vol. 25, no. 1, Jan. 2022, Art. no. 2.
G. Nagymáté, T. Tuchband, and R. M. Kiss, "A novel validation and calibration method for motion capture systems based on micro-triangulation," Journal of Biomechanics, vol. 74, pp. 16–22, Jun. 2018.
Q. Fathima and R. Fathima, "Motion Capture Technology in Animation," Kristu Jayanti Journal of Computational Sciences (KJCS), pp. 44–57, Dec. 2023.
K. Maduwantha et al., "Accessibility of Motion Capture as a Tool for Sports Performance Enhancement for Beginner and Intermediate Cricket Players," Sensors, vol. 24, no. 11, Jan. 2024, Art. no. 3386.
M. Jacobsson, J. Willén, and M. Swarén, "A Drone-mounted Depth Camera-based Motion Capture System for Sports Performance Analysis," in Artificial Intelligence in HCI, Copenhagen, Denmark, 2023, pp. 489–503.
M. Talha, "Research on the use of 3D modeling and motion capture technologies for making sports training easier," Revista de Psicología del Deporte (Journal of Sport Psychology), vol. 31, no. 3, pp. 1–10, Oct. 2022.
S. A. Rawashdeh, D. A. Rafeldt, T. L. Uhl, and J. E. Lumpp, "Wearable motion capture unit for shoulder injury prevention," in 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Cambridge, MA, USA, Jun. 2015, pp. 1–6.
A. N. Belova et al., "State-of-the-art possibilities of motion capture technologies in sport injuries research (review)," Russian Journal of Biomechanics, vol. 26, no. 2, pp. 74–86, Dec. 2022.
Z. Ang, "Application of IoT technology based on neural networks in basketball training motion capture and injury prevention," Preventive Medicine, vol. 175, Oct. 2023, Art. no. 107660.
Z. Chen and G. Zhang, "CNN sensor based motion capture system application in basketball training and injury prevention," Preventive Medicine, vol. 174, Sep. 2023, Art. no. 107644.
N. Alharbi, "Exploring Advance Approaches for Drowning Detection: A Review," Engineering, Technology & Applied Science Research, vol. 14, no. 4, pp. 16032–16039, Aug. 2024.
G. Fiche, V. Sevestre, C. Gonzalez-Barral, S. Leglaive, and R. Séguier, "SwimXYZ: A large-scale dataset of synthetic swimming motions and videos," in ACM SIGGRAPH Conference on Motion, Interaction and Games, Rennes, France, Nov. 2023, pp. 1–7.
Y. Omae, M. Kobayashi, K. Sakai, T. Akiduki, A. Shionoya, and H. Takahashi, "Detection of swimming stroke start timing by deep learning from an inertial sensor," ICIC Express Letters, Part B: Applications, vol. 11, no. 3, pp. 245–251, 2020.
L. Chen and D. Hu, "An effective swimming stroke recognition system utilizing deep learning based on inertial measurement units," Advanced Robotics, vol. 37, no. 7, pp. 467–479, Apr. 2023.
N. Dirani and G. Pelletier, "Comparaison between the performance of distinct models in classification of swimming types using rolling windows and compression methods," Sep. 2023.
H. Kulhandjian, N. Ramachandran, M. Kulhandjian, and C. D’Amours, "Human Activity Classification in Underwater using Sonar and Deep Learning," in Proceedings of the International Conference on Underwater Networks & Systems, Atlanta, GA, USA, Oct. 2019, pp. 1–5.
L. C. Powell, "The Evaluation of Recognizing Aquatic Activities Through Wearable Sensors and Machine Learning," Ph.D. dissertation, Texas AM University, USA, 2019.
G. Brunner, D. Melnyk, B. Sigfússon, and R. Wattenhofer, "Swimming style recognition and lap counting using a smartwatch and deep learning," in Proceedings of the 23rd International Symposium on Wearable Computers, London, UK, Sep. 2019, pp. 23–31.
J. Costa, C. Silva, M. Santos, T. Fernandes, and S. Faria, "Framework for Intelligent Swimming Analytics with Wearable Sensors for Stroke Classification," Sensors, vol. 21, no. 15, Jan. 2021, Art. no. 5162.
M. Ehab, H. Mohamed, M. Ahmed, M. Hammad, N. ElMasry, and A. Atia, "ISwimCoach: A Smart Coach guiding System for Assisting Swimmers Free Style Strokes: ISwimCoach," in Companion Publication of the 2020 International Conference on Multimodal Interaction, Oct. 2020, pp. 265–269.
H. Fani, A. Mirlohi, H. Hosseini, and R. Herperst, "Swim Stroke Analytic: Front Crawl Pulling Pose Classification," in 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, Oct. 2018, pp. 4068–4072.
A. J. Silva et al., "The Use of Neural Network Technology to Model Swimming Performance," Journal of Sports Science & Medicine, vol. 6, no. 1, pp. 117–125, Mar. 2007.
B. Victor, Z. He, S. Morgan, and D. Miniutti, "Continuous Video to Simple Signals for Swimming Stroke Detection with Convolutional Neural Networks," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, Jul. 2017, pp. 122–131.
E. Delhaye, A. Bouvet, G. Nicolas, J. P. Vilas-Boas, B. Bideau, and N. Bideau, "Automatic Swimming Activity Recognition and Lap Time Assessment Based on a Single IMU: A Deep Learning Approach," Sensors, vol. 22, no. 15, Jan. 2022, Art. no. 5786.
A. Jayawardene and P. Kalansooriya, "Swimming Stroke Analysis and Feedback System using Machine Learning," presented at the 1st International Conference on Advanced Computing Technologies (ICACT 2024), Oct. 2024.
A. Hall et al., "The detection, tracking, and temporal action localisation of swimmers for automated analysis," Neural Computing and Applications, vol. 33, no. 12, pp. 7205–7223, Jun. 2021.
G. Jocher, J. Qiu, and A. Chaurasia, "Ultralytics YOLO." Jan. 2023, [Online]. Available: https://github.com/ultralytics/ultralytics.
J. Terven, D. M. Córdova-Esparza, and J. A. Romero-González, "A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS," Machine Learning and Knowledge Extraction, vol. 5, no. 4, pp. 1680–1716, Nov. 2023.
Ultralytics, "Comprehensive Guide to Ultralytics YOLOv5." https://docs.ultralytics.com/YOLOv5.
Downloads
How to Cite
License
Copyright (c) 2024 Atheer Al-Majnoni, Jumana Al-Sahli, Dana Al-Ahmady, Amani Al-Mutairi, Areej Alsini, Manal Alharbi
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
