Predictive Performance of Machine Learning Algorithms Trained with Sparse Data

Abstract

In the field of prognostic health management of aerospace assets and fleets, accurate prediction of the future state of an asset based on current and historical data has always been a challenge. Several different methods of computing that future state are commonly utilized including physics-based simulation and machine learning algorithms. Physics-based simulations have the advantage of true predictability in that they are able to evolve a model of the system using first principles based on causality and predict future states that have yet to be observed on real assets, however they also have the disadvantage of being difficult, time-consuming, and expensive to construct. Machine learning algorithms are the reverse: simple, fast, and cheap to construct but without true predictability since they can only fit input data to the data sets with which they were trained. For some aerospace fleets with a significant amount of historical degradation and failure data to train a model, machine learning algorithms provide very good predictions, but for other fleets with little historical degradation and failure data, machine learning may not provide good predictions of the future states of the assets. This paper explores the accuracy of the predictions made with machine learning algorithms when they are trained with sparse data with varying quality. An empirical study is performed where machine learning models are trained with data representing various historical data set types that are typically encountered in aerospace health management (e.g. low quality dense data, high quality sparse data, etc.), and the results from running those algorithms are compared with an idealized physics-based model. The predictive performance of the machine learning algorithms are quantified, and suggestions on when it is appropriate to utilize machine learning algorithms in aerospace health management systems are presented.