Understanding university instructors’ cumulative user experience with a Learning Management System: A semester-long mixed methods study

Abstract

Educational technologies are rapidly advancing, with LMS being used by most academic institutions to facilitate and support learning. The COVID pandemic catalysed use of technologies to support non-traditional learning setup in the Philippines. While LMS use is widespread and its usability is highly explored, research on long-term use remains limited, particularly regarding how user behavior and system interactions evolve over time, which is crucial to understand the context of use and identification of areas for improvement to better support users. In addition, only a select few examine this from teachers’ viewpoints. This study explores the university instructors’ experiences over a semester with a Moodle-based LMS using a mixed methods approach. The results presented here are part of a broader study, covering the “during semester” experiences with the LMS. Ten (10) instructors participated in the study, accomplishing 3 weekly surveys and a diary every month. The weekly surveys measured their usage and ratings on the communication, course management and system usability, as well as their experiences. The monthly diary enabled participants to reflect on the challenges they faced, opportunities for improvement and the tools they used to achieve their goals. Combining these methods helped users track their UX progression over time. This study employed a post-phenomenological approach, which examines the dynamic human-technology-world relationship, where technological mediation co-shapes human subjectivity and the objectivity of their world, viewing technology as inviting and inhibiting human action and practices in any given situation. This semester-long study found that three components, Course Management, Communication and System Usability, have yielded overall positive results, both quantitatively through the weekly ratings and qualitatively, as reported in weekly and monthly diaries. Instructors perceived the LMS to invite actions related to teaching and learning rather than inhibiting them. Results show that while the UX scores for all three components have been positive, the qualitative reports were not always consistent with these positive ratings, specifically for System Usability, highlighting the importance of the mixed approach to understanding user experiences. The top three recommendations for LMS improvement included training and support, communication tools, and the user interface.

Keywords: cumulative user experience, learning management systems, usability

DOI: 10.54941/ahfe1006667