Scalable Threat Detection in Customer Interactions Using LLMs and LLM-as-Judge Framework

Abstract

This paper introduces a Customer Threat Detection Model leveraging a pre-trained large language model (LLM) on a major cloud platform to analyze customer service call transcripts and social media posts for potential security threats. The solution was developed in response to a critical need by the corporate security team to proactively identify threats during high-risk periods—such as the Southern California wildfires in January—when call volumes to the Customer Contact Center surged and employees and property faced elevated safety risks. Historically, manual identification of threats was slow and inconsistent, creating potential exposure for the organization. Operating in batch mode, the system processes daily calls and assigns each interaction a threat score (0–100), mapped to five ordinal bins from Low to High. The model combines expert-defined keywords with semantic embedding techniques to expand its threat lexicon, enabling detection of evolving language and context. Each transcript is transformed into a structured prompt and evaluated by the LLM to produce a threat score and category.Manual review sampled calls showed ~93% accuracy but proved resource-intensive and impractical for ongoing monitoring. To address scalability, we applied an “LLM-as-a-Judge” framework, where LLMs act as surrogate evaluators of model outputs. For 10K sampled calls, two summaries per call, overall and threat-focused, were generated and independently assessed by a second LLM to assign ordinal threat categories. Agreement metrics (accuracy, Cohen’s kappa, mean absolute difference), triadic consistency, and keyword sensitivity were computed. A small Keyword Influence Delta indicated strong contextual detection and guided keyword refinement.Results indicate good agreement between the deployed model and independent LLM judges, demonstrating scalability and reduced analyst workload in safety‑critical monitoring contexts.

Keywords: Threat Scoring, Severity Classification, LLM-as-Judge, Semantic Enrichment

DOI: 10.54941/ahfe1007696