Enhanced System Usability Scale using the Software Quality Standard Approach
Received: 28 April 2023 | Revised: 28 June 2023 | Accepted: 7 July 2023 | Online: 13 October 2023
Corresponding author: Raja Rina Raja Ikram
Abstract
The objective of this paper is to improve the current System Usability Scale (SUS) and assess its applicability in the context of Learning Management Systems (LMS). The need to evaluate the usability of systems has become increasingly important in today's market, as it can have a significant impact on the user experience. In light of the COVID-19 pandemic, e-learning has become an essential tool for students, making LMS an appropriate research case study. Through a comprehensive literature review, it was discovered that SUS is the most widely used tool for evaluating system usability. However, SUS fails to satisfy some of the usability criteria outlined in ISO 9126 and ISO 9241-11. Therefore, this paper proposes an enhanced SUS model and its conceptual framework to address these limitations. The proposed model was validated using a case study approach, involving subject matter experts and software testing students, who evaluated the reliability of the enhanced SUS. Additionally, the existing and enhanced SUS models were evaluated based on an LMS case study and the results were used to calculate the enhanced SUS's reliability coefficient using Cronbach's alpha. The validation results show that the enhanced SUS has higher reliability with improved quality coverage compared to the original SUS. The proposed model has the potential to enhance the evaluation of system usability and, consequently, improve user experience.
Keywords:
usability, software quality, System Usability Scale (SAS)Downloads
References
A. Hafeez, S. Javed, A. B. Murtaza, A. Aziz, S. M. Hassan, and I. Hussain, "A Survey on the Adaption of CMS in Pakistani Universities," Engineering, Technology & Applied Science Research, vol. 9, no. 6, pp. 5037–5040, Dec. 2019.
S. Vairamuthu and S. M. Anouncia, "Reconnoitering Students’ Satisfaction of an Online Based Assessment System to Improve Usability using PSO: An Examination into a Problem Solving and Programming Course," Engineering, Technology & Applied Science Research, vol. 6, no. 5, pp. 1207–1211, Oct. 2016.
M. Jordan and N. Duckett, "Universities Confront ‘Tech Disruption’: Perceptions of Student Engagement Online Using Two Learning Management Systems," The Journal of Public and Professional Sociology, vol. 10, no. 1, Jan. 2018.
I. Maslov, S. Nikou, and P. Hansen, "Exploring user experience of learning management system," The International Journal of Information and Learning Technology, vol. 38, no. 4, pp. 344–363, Jan. 2021.
ISO/TC 159/SC 4 Ergonomics of human-system interaction Technical Commitee, ISO 9241-11:1998. Ergonomic requirements for office work with visual display terminals (VDTs) — Part 11: Guidance on usability. Geneva, Switzerland: ISO, 1998.
J. R. Lewis and J. Sauro, "Revisiting the Factor Structure of the System Usability Scale," JUX - The Journal of User Experience, vol. 12, no. 4, pp. 183–192, Aug. 2017.
J. R. Lewis, "The System Usability Scale: Past, Present, and Future," International Journal of Human–Computer Interaction, vol. 34, no. 7, pp. 577–590, Jul. 2018.
ISO/TC 159/SC 4 Technical Committee, ISO 9241-11:2018. Ergonomics of human-system interaction. Geneva, Switzerland: ISO, 2018.
ISO/IEC JTC 1/SC 7 Technical Committee, ISO/IEC 9126-1:2001. Software engineering — Product quality — Part 1: Quality model. Geneva, Switzerland: ISO, 2001.
R. Reh, M. L. Mursidi, and N. A. A. Husin, "Reliability analysis for pilot survey in integrated survey management system," in 2011 Malaysian Conference in Software Engineering, Johor Bahru, Malaysia, Sep. 2011, pp. 220–222.
J. M. Cortina, "What is coefficient alpha? An examination of theory and applications," Journal of Applied Psychology, vol. 78, no. 1, pp. 98–104, 1993.
N. Schmitt, "Uses and abuses of coefficient alpha," Psychological Assessment, vol. 8, no. 4, pp. 350–353, 1996.
J. R. Lewis, "IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use," International Journal of Human–Computer Interaction, vol. 7, no. 1, pp. 57–78, Jan. 1995.
J. Sauro and J. R. Lewis, "Chapter 8 - Standardized Usability Questionnaires," in Quantifying the User Experience, J. Sauro and J. R. Lewis, Eds. Boston, MA, USA: Morgan Kaufmann, 2012, pp. 185–240.
L. Buenaflor, "ISO 9126 Software Quality Characteristics," Medium, Sep. 02, 2017. https://medium.com/@leanardbuenaflor/iso-9126-software-quality-characteristics-a25a26e7d046.
Sholiq, R. A. Auda, A. P. Subriadi, A. Tjahyanto, and A. D. Wulandari, "Measuring software quality with usability, efficiency, and portability characteristics," IOP Conference Series: Earth and Environmental Science, vol. 704, no. 1, Nov. 2021, Art. no. 012039.
O. A. Rotaru, S. Vert, R. Vasiu, and D. Andone, "Standardised Questionnaires in Usability Evaluation. Applying Standardised Usability Questionnaires in Digital Products Evaluation," in Information and Software Technologies, Cham, 2020, pp. 39–48.
J. Kirakowski and M. Corbett, "SUMI: the Software Usability Measurement Inventory," British Journal of Educational Technology, vol. 24, no. 3, pp. 210–212, 1993.
J. Kirakowski and M. Corbett, "SUMI: the Software Usability Measurement Inventory," British Journal of Educational Technology, vol. 24, no. 3, pp. 210–212, 1993.
R. A. Ali and M. R. M. Arshad, "Perspectives of Students’ Behavior Towards Mobile Learning (M-learning) in Egypt: an Extension of the UTAUT Model," Engineering, Technology & Applied Science Research, vol. 6, no. 4, pp. 1109–1114, Aug. 2016.
M. Hemmati and H. Hosseini, "Effect of IT Application on Project Performance Focusing on the Mediating Role of Organizational Innovation, Knowledge Management and Organizational Capabilities," Engineering, Technology & Applied Science Research, vol. 6, no. 6, pp. 1221–1226, Dec. 2016.
N. A. Kamarudin, R. R. Raja Ikram, F. N. Azman, S. S. Syed Ahmad, and D. Bin Zainuddin, "A Study of The Effects of Short-Term AI Coding Course with Gamification Elements on Students’ Cognitive Mental Health," TEM Journal, vol. 11, no. 4, pp. 1854–1862, Nov. 2022.
Y. Thamilarasan and R. R. R. Ikram, "MyMUET: A Design of a Mobile based Crowdsourced Assessment for Malaysia University English Test with Non-Synchronous Participant Interaction," International Journal of Innovative Technology and Exploring Engineering, vol. 8, no. 11, Sep. 2019.
Downloads
How to Cite
License
Copyright (c) 2023 Yarshini Thamilarasan, Raja Rina Raja Ikram, Mashanum Osman, Lizawati Salahuddin, Wan Yaakob Wan Bujeri, Kasturi Kanchymalay
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
