ProMetaUS: A proactive meta-learning uncertainty-based framework to select models for Dynamic Risk Management

Abstract

Safety managers, practitioners, and researchers can employ different models for estimating and assessing hazards, consequences, likelihoods, risks, and/or mitigation measures in the safety field. The selection of a specific model may depend on the uncertainty associated with its estimation and its impact on the safety-related decision-making process. The recognition of this issue as an example of Algorithm Selection Problem (ASP) allows investigating the applicability of meta-learning principles that are scarcely adopted in the risk and safety literature. Consequently, we propose a novel meta-learning inspired framework to proactively rank a set of candidate models for Dynamic Risk Management (DRM) based on desired uncertainty conditions. We denominate this framework ProMetaUS (Proactive Meta-learning and Uncertainty-based Selection for dynamic risk management). To achieve this purpose, our meta-learning system acquires knowledge that relates the characteristics extracted both directly and indirectly from datasets (e.g. data-based, domain-based, simple and fast uncertainty-based, simple and fast sensitivity-based meta-features) to some performance measures of the models. Performance measures include confidence information, shape measurable quantities, safety decision criteria and threshold limits, and sensitivity analysis outputs. We tested the proposed framework in a case study about Oxygen Deficiency Hazard (ODH) assessment by means of @RISK. For each of the five datasets, single-performance measure rankings and a final ranking of the three models are generated. Such rankings are aggregated to obtain the global recommended ranking.