Revisiting the Brief Nuclear Usability Measure: A Preliminary Evaluation of its Validity and Reliability using Licensed Operators

Abstract: Global energy consumption is expected to increase through 2050 (EIA, 2023). Global population grown, increased manufacturing, and higher living standards are cited as key drivers to pushing energy consumption beyond energy efficiency advances. In order to circumvent such grim projections, the role of nuclear electricity generation has a pivotal role in providing carbon-free electricity generation across the world. Within the United States, there have been thrusts in extending the operational lifespan of the existing light water reactor fleet through significantly modernizing these existing plants with digital technologies that reduce their operations and maintenance cost. Additionally, despite some setbacks, development and deployment of advanced reactor technologies are continuing to move forward from both developer and regulatory standpoints. Across both pathways, the role of human factors engineering is crucial to ensuring timely completion of major modernization efforts or advanced reactor deployment by addressing human and technology integration challenges. Such challenges range from effective allocation of function with digital technologies and automation to the design of novel human-system interfaces that support effective monitoring and control of advanced reactors. Two important human factors constructs that are relevant for human factors tests and evaluation of these advanced technologies entails situation awareness and workload. For instance, these constructs are referenced in existing regulatory review guidance (e.g., NUREG-0711) as vital to assess during integrated system validation and other testing and evaluation activities. Indeed, a multimethod approach is generally most appropriate for assessing situation awareness and workload as these constructs are not directly observable. A suite of objective and subjective measures is typically a “gold standard” in evaluating situation awareness and workload. Though, the use of self-report through standardized survey instruments offers a practical way of collecting such data, particularly in the “real world.” For instance, when testing licensed operators, availability and time for data collection can be significantly limited (Kovesdi and Joe, 2019). Therefore, approaches to streamline survey instruments that can adequately assess situation awareness and workload have been explored. Kovesdi and Joe (2019) developed an abbreviated survey instrument, the Brief Nuclear Usability Measure, derived from reviewing the National Aeronautics and Space Administration Raw Task Load Index (NASA-TLX; Hart and Staveland, 1988) Single Ease Question (SEQ; Sauro and Lewis, 2016), and Situation Awareness Rating Technique (SART; Taylor, 1990). This work presents an exploratory evaluation of the validity and reliability of the Brief Nuclear Usability Measure using real-world data collected from four separate and independent human factors studies that utilized licensed operators during operator-in-the-loop studies. The intent of this work is to provide an assessment of the utility of this measure as a practical tool to evaluating situation awareness and workload for prospective human factors tests and evaluations for both modernization efforts and advanced reactor development.ReferencesKovesdi, C., & Joe, J. (2019, February). Exploring the Psychometrics of Common Post‑Scenario Human Factors Questionnaires of Workload, Situation Awareness, and Perceived Difficulty. In 11th Nuclear Plant Instrumentation, Control, and Human-Machine Interface Technologies, NPIC and HMIT 2019 (pp. 1631-1643). American Nuclear Society.Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in Psychology (Vol. 52, pp. 139-183). North-Holland.https://www.eia.gov/pressroom/releases/press542.phpSauro, J., & Lewis, J. R. (2016). Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann.Taylor, R.M. (1990). Situational Awareness Rating Technique (SART): The development of a toll for aircrew systems design. In: AGARD Conference Proceedings No 478, Situational Awareness in Aerospace Operations. Aerospace Medical Panel Symposium, Copenhagen, 2nd -6th October 1989.U.S. Nuclear Regulatory Commission. (2012). Human Factors Engineering Program Review Model, NUREG-0711, Rev. 3. Washington, DC: U.S. Nuclear Regulatory Commission.

Keywords: Workload Assessment, Situation Awareness Assessment, Control Room Design and Evaluation

DOI: 10.54941/ahfe1005661

Cite this paper:

​