Automatic validation and analysis of predictive models by means of big data and data science

Abstract

Validation is an essential procedure in the development of a predictive model in several engineering fields. In addition, recent data analysis techniques and the increasing availability of data have the potential to provide a deeper understanding of experimental data and simulation models. This work proposes a systematic, objective, and automatic methodology to validate and analyze experiments and models from a high-level perspective. The proposed methodology exploits the opportunities offered by the ‘data ecosystem’ concept, combining data and model evaluation and providing an integrated set of techniques to produce synthetic but comprehensive insights about the experiment and the predictive model. The methodology focuses on data assessment of the experiments used in the process, the use of a trend similarity comparison index to measure the model performance, and data science techniques to systematically extract models’ behavior insight by analyzing a large number of validation results and linking them to the experiment characteristics.The automated proposed approach follows the generality principle and can be extended to different application domains in which predictive models are validated against big data in the chemical engineering domain. As a case study, the proposed methodology is applied with hundreds of experimental datasets to evaluate a kinetic model that describes the pyrolysis and combustion of hydrocarbons.