Measuring Subjective Usability of Medical Devices - Questionnaire Development and Evaluation

Abstract

Safety and effectiveness are major usability concerns for the development process of medical devices. Other relevant factors like user satisfaction or overall user experience (UX) are sometimes neglected as they are not required from a regulatory perspective, nor can they be evaluated well through the classical approach of usability testing. Usability/UX questionnaires can measure these subjective variables, however, only few researchers have addressed the development of standardized questionnaires for medical products. This two-parted research aims to further close this gap. First, numerous attributes of usability/UX were researched and then critically evaluated by usability experts (N=9) with practical experience in healthcare. The constructs relevant and applicable were then divided into clusters and items were newly created or carefully chosen from existing questionnaires and then condensed to a 70-item raw version of the questionnaire. In the second part of this study, nurses (N=106) from South Africa, UK, and USA evaluated a dialysis device, providing responses to the questionnaire statements alongside the System Usability Scale (SUS) for validation purposes. Psychometric analysis showed that the average internal reliability across the eleven subscales was α = .70 and ranged from .48 to .84. Seven items were chosen to be eliminated because of their weak item discrimination and difficulty which would lead to an increase of internal reliability. The initial scores for 9 out of the 11 subscales moderately correlated with the SUS (r = .53 to .60) with a significance of α < .05. Overall, the results indicate that the newly developed questionnaire could be feasible to close the identified gap. Nevertheless, the modified questionnaire ought to be validated with a larger sample size and across a broader range of medical products.

Keywords: Usability, Human Factors, User Experience, Questionnaire Development, Healthcare, Medical Devices, Usability Engineering

DOI: 10.54941/ahfe1003489