TY - JOUR
T1 - Multi-source information fusion to assess control room operator performance
AU - Zhang, Xiaoge
AU - Mahadevan, Sankaran
AU - Lau, Nathan
AU - Weinger, Matthew B.
N1 - Funding Information:
This research was partly funded by the U.S. Department of Energy through a NEUP grant No. DENE 0008267 (Principal Investigator: Dr. Matthew Weinger, Vanderbilt University; Monitor: Dr. Bruce Hallbert, Idaho National Laboratory). The authors would like to thank Amy Jungmin Seo (graduate student) and Dennis Deardorff (graduate student, retired licensed operator) at Virginia Tech for their technical assistance and valuable discussions. We are also grateful to the Center for Advanced Engineering and Research (CAER) (Director: Bob Bailey) for access to the experimental data and related software. The simulator experiment at CAER was funded by the Virginia Tobacco Indemnification and Community Revitalization Commission under grant awards #2128 and #1663. Support was also provided by Virginia's Center for Innovative Technology Commonwealth Research and Commercialization Fund under grant awards #MF-004 and #MF14F-020-En.
Publisher Copyright:
© 2018 Elsevier Ltd
PY - 2020/2
Y1 - 2020/2
N2 - Control room operators respond to abnormal situations through a series of cognitively demanding activities, e.g., monitoring, detection, diagnosis, and response. However, variability among operators in terms of prior experience and current operational context affects their response to the malfunction. A machine learning framework was employed to integrate multiple data sources and develop an empirical model of operator performance in responding to malfunction events. A human-in-the-loop within-subjects experiment was performed using a high-fidelity Generic Pressurized Water Reactor simulator. The study recruited nine licensed operators in three-person crews completing ten scenarios, each incorporating two to four malfunction events. Individual operator performance was assessed using eye tracking technology and physiological recordings of skin conductance response and respiratory function. Expert-rated event management performance was the primary study outcome. These heterogeneous data sources were fused using an approach that integrated a support vector machine with bootstrap aggregation to develop a trained quantitative prediction model. While no single variable predicted operator performance, the fused model's predictions using independent verification data was very good (prediction accuracy of 75–83%). The proposed methodology offers a quantitative approach to evaluate the crew performance through fusing the heterogeneous data collected from experiment.
AB - Control room operators respond to abnormal situations through a series of cognitively demanding activities, e.g., monitoring, detection, diagnosis, and response. However, variability among operators in terms of prior experience and current operational context affects their response to the malfunction. A machine learning framework was employed to integrate multiple data sources and develop an empirical model of operator performance in responding to malfunction events. A human-in-the-loop within-subjects experiment was performed using a high-fidelity Generic Pressurized Water Reactor simulator. The study recruited nine licensed operators in three-person crews completing ten scenarios, each incorporating two to four malfunction events. Individual operator performance was assessed using eye tracking technology and physiological recordings of skin conductance response and respiratory function. Expert-rated event management performance was the primary study outcome. These heterogeneous data sources were fused using an approach that integrated a support vector machine with bootstrap aggregation to develop a trained quantitative prediction model. While no single variable predicted operator performance, the fused model's predictions using independent verification data was very good (prediction accuracy of 75–83%). The proposed methodology offers a quantitative approach to evaluate the crew performance through fusing the heterogeneous data collected from experiment.
KW - Bootstrap aggregating
KW - Eye tracking
KW - Human reliability analysis
KW - Physiological response
KW - Support vector machine
UR - http://www.scopus.com/inward/record.url?scp=85055875394&partnerID=8YFLogxK
U2 - 10.1016/j.ress.2018.10.012
DO - 10.1016/j.ress.2018.10.012
M3 - Journal article
AN - SCOPUS:85055875394
SN - 0951-8320
VL - 194
JO - Reliability Engineering and System Safety
JF - Reliability Engineering and System Safety
M1 - 106287
ER -