Detecting Information Leakage in Predictive Process Mining: A Three-Layer Evaluation Framework

Open Access
Article
Conference Proceedings
Authors: Fathima Lodi, Amir Schur
Abstract

Predictive process monitoring aims to provide early warnings about process outcomes, yet published benchmarks frequently report near-perfect prediction scores on full process traces. This paper investigates whether those scores reflect genuine predictive ability or information already embedded in the event log. A simulated supply-chain dataset with 818,292 events across 25,000 linked cases was used to evaluate 276 model configurations under controlled conditions where ground truth causal labels were available. Three forms of information leakage were identified- activity names that encode outcomes, conformance features that overlap with prediction targets, and temporal variables that measure downstream effects rather than upstream causes. When these sources were retained, models achieved an AUC of 1.00. After leakage was controlled through prefix-based evaluation, scores fell to between 0.76 and 0.93. A diagnostic procedure with specific detection thresholds is proposed for each layer. The framework was examined on two external benchmark datasets from different domains, and all three leakage patterns were independently observed in both. These findings suggest that near-perfect classification scores in predictive process mining should be interpreted with caution, and that a systematic leakage audit should precede model comparison or deployment.

Keywords: Predictive Process Monitoring, Information Leakage Detection, Process Mining, Prefix-Based Evaluation, Event Log Analysis, Process Intelligence, Supply Chain Analytics

DOI: 10.54941/ahfe1008421

Cite this paper
Downloads
0
Visits
1
Download PDF

More from this volume

← Psychological Foundations of Epistemic Profiles for Misinformation Resilience: Implications for Defence-Oriented Agent-Based ModellingBuilding Sustainable Business Value through Human-Centred AI Transition Pathways →
View all articles in Human Factors in Design, Engineering, and Computing →