A Multimodal, Uncertainty-Aware, and Transparent AI Grading Tool for Scalable Automated Grading in AI and Data Science Higher Education

Abstract

Multimodal assignments- programming, free response, and oral - are difficult for humans to grade consistently at scale. Artificial Intelligence (AI) assisted Automated Grading Tools (AGTs) offer a path forward, but existing systems are evaluated within a single modality and lack a framework for routing assignments to human review based on AI uncertainty and modality. This paper evaluates a multimodal AGT on 95 assignments: Programming (n=35), free response (n=30), and oral (n=30), each scored by two human graders and the AI. This paper presents three novel contributions to the field of AI assisted AGT. First, it contributes a per-modality validity characterization including the first directionally-resolved test on mock-interview grading. Second, it presents the exemplar-anchored harshness hypothesis as a falsifiable mechanism for AI severity on subjective modalities. Finally, it provides an empirical test of semantic-entropy escalation finding no usable routing signal, based on the sample size and experiment settings.

Keywords: Automatic Grading Tool, Ai, Multimodal, Semantic Entropy, Large Language Model

DOI: 10.54941/ahfe1008097