Human Factors Methods in Developing AI and Machine Learning High-Risk Prediction Models in Obstetric Care

Abstract: This study investigates the application of human factors (HF) methods to the development of artificial intelligence (AI) and machine learning (ML) models for high-risk obstetric (OB) care, focusing on integration within the Epic electronic health record (EHR) system across three hospital systems. A systematic scoping review of 39 AI/ML techniques revealed that none had achieved an acceptable level of clinician acceptance. To address this issue, we propose a human-centered design approach, emphasizing clinical decision-making, workflow alignment, and potential maternal morbidity. Our multi-phase strategy actively engages stakeholders, including OB care providers, to refine system prototypes while considering usability, explainability, trust, and cultural sensitivity. The research aims to establish a roadmap for the future development of high-risk maternal health prediction models.In the first phase (Aim 1), we identify system requirements through stakeholder engagement and literature review, uncovering significant pain points, including communication delays, poor information flow, and the lack of AI/ML tools in practice. The study highlights the importance of addressing these challenges to optimize clinical outcomes.The second phase (Aim 2) focuses on designing AI/ML architectures, aligning system features with HF principles and ensuring standardized data across clinics. Co-designing tools with OB providers resulted in solutions such as smart-SBAR for shift changes and patient-engagement strategies that enhance cultural sensitivity. Providers expressed a strong preference for decision-support tools that complement rather than replace clinical judgment, with transparency in AI outputs seen as critical to building trust.Aim 3, which is ongoing, involves the prototyping and testing of AI/ML tools within clinical workflows, focusing on usability and alignment with existing practices. Preliminary findings suggest that transparent, interpretable AI outputs, along with streamlined workflows, significantly improve clinician trust and utility.In the final phase (Aim 4), the study evaluates both formative and summative aspects of the AI/ML tools, assessing their impact on clinical decision-making, trust, and maternal health outcomes. The evaluation methodology involves assessing model performance, clinician feedback, and model accuracy in simulated environments.By incorporating human factors methods into the development of AI/ML tools for high-risk OB care, the study aims to improve clinician acceptance, enhance decision-making processes, and streamline workflows. These findings contribute to the broader adoption of AI/ML tools in maternal healthcare, with the potential to improve clinical outcomes and patient care efficiency.

Keywords: Human Factors, Human-centered design, Machine Learning, Artificial Intelligence, Risk, Obstetrics and Gynecology

DOI: 10.54941/ahfe1006635

Cite this paper:

​