Uncertainty analysis on support vector machine for measuring organizational factors in probabilistic risk assessment of nuclear power plants

Abstract

This paper is a product of a line of research by the authors to explicitly incorporate organizational factors into the probabilistic risk assessment (PRA) of complex socio-technical systems. This explicit incorporation helps (i) assess the system risk due to organizational and managerial weaknesses, (ii) identify the critical organizational root causes of failure scenarios, aiding in effective corrective action, and (iii) avoid the potential of underestimating the system risk involving human error and organizational factors. To facilitate the measurement of organizational factors contributing to the PRA scenarios, the previous studies by the authors developed the Data-Theoretic methodology, where “data analytics” are guided by a “theory” of underlying causation to prevent misleading results from solely data-oriented approaches. The Data-Theoretic methodology consists of two submodules: (a) DT-BASE that develops a detailed causal model based on a theory-building process and is equipped with a baseline quantification utilizing analyst interpretation of information extracted from relevant references; and (b) DT-SITE that conducts data analytics (text mining using machine learning) to quantify the organizational causal elements based on industry event databases. This article investigates uncertainty analysis for the uncertainties associated with machine learning-based data mining in DT-SITE using a Support Vector Machine (SVM) classifier for industry event databases. This article conducts a literature review and proposes a categorization scheme of the SVM uncertainties to establish a theoretical foundation for identifying uncertainty sources associated with the SVM classifier. The implementation of uncertainty analysis for DT-SITE is then illustrated using a nuclear power plant case study. The potential uncertainty sources related to the SVM classifier used in DT-SITE are identified using the categorization scheme developed based on the literature review. The uncertainty analysis procedure for the SVM classifier in DT-SITE is proposed, and the linkage to the current uncertainty analysis process in PRA is discussed. The proposed uncertainty analysis procedure for the SVM classifier in DT-SITE is applied to the illustrative case study to assess the impact of one of the potential uncertainty sources (i.e., the selection of document sections included in the dataset) on the risk-informed decision-making.