Evaluating the Efficacy of Structured Analytic Techniques (SATs) as a Support System to Enhance Decision-Making within ISR Mission Environments

Abstract: U.S. Air Force military operators involved with Intelligence, Surveillance, and Reconnaissance (ISR) missions are required to process, exploit, and disseminate (PED) collected intelligence within friendly and hostile environments in near-real time in order to provide geographical locations and ground movement patterns. Intelligence collected during ISR operations are then implemented into future strategic planning to provide our military an edge in the battlefield. However, the information collected can be vague, incomplete, or ill-defined resulting in operators making poor or inadequate decisions. Therefore, the objective of this study was to evaluate the effectiveness of two structured analytic techniques (SATs) against a control group when interpreting and comprehending narrative content in order to support and facilitate current tool development and future technology transition within the ISR community. Three groups of 25 participants (N=75) were randomly assigned to one of two analytic techniques or a control approach and provided a narrative. The SATs implemented were the Method for Defining Analytical Questions (MDAQ) which was developed in-house by our ISR subject matter experts (SMEs), a Scaffolding approach, or a Control approach. MDAQ is a repeatable process focused on identifying an indicator and its association to a person, place, or event before providing a solution. Scaffolding is founded on determining a problem statement, generating a solution, providing justification, evaluating the hypothesis, and providing a solution. For the Control approach, participants read through the content and provided a solution. The objective of the study was to determine if providing a structured analytic technique would enhance the detection of essential elements of information (EEI) embedded within the narrative leading to improved performance accuracy. The findings provided underlying evidence that implementing a Scaffolding approach significantly improved performance accuracy compared to MDAQ and Control (p<0.01). Moreover, a statistically significant difference was detected within the MDAQ group when participants repeated the process compared to those who only went through the process once (p<0.01). Nevertheless, the findings suggests that providing participants with a structured analytic technique enabled them to identify and interpret critical EEIs that maybe overlooked otherwise resulting in improved performance accuracy. This discovery will support human-computer interactions for future ISR tool development.

Keywords: Intelligence, Surveillance, and Reconnaissance (ISR), Method for Defining Analytical Questions (MDAQ), Human Decision-Making

DOI: 10.54941/ahfe1004458

Cite this paper:

​