Data gap analysis of ship and maritime data using meta learning

Abstract

With the developments in new Internet of Things (IoT) technology, ship owners and managers can access and analyze vast amounts of real-time onboard data. However, data collected from the ship’s engine and navigation information include various types of errors owing to imprecision, bias, sensor failure, or human error caused by manual recording by the ship operator’s hands. Such abnormal data make accurate ship performance prediction and monitoring difficult; thus, the original data need to be carefully refined through data preprocessing. Machine learning models have been used to detect and remove abnormal data; however, this yields inconsistent predictions in results depending on the types of machine learning models used. In this study, a robust data gap analysis method is proposed to detect abnormal data among ship and marine data collected in real time. Data preprocessing, including rule-based data imputation, clustering, denoising, and dimension reduction, is systematically performed; subsequently, a meta-learning model, which is a combination of various machine learning models, is used to predict ship performance and detect its abnormal data. This study compared the performance of single-machine learning models and meta models through various error analysis methods that utilize actual ship operation data. The single machine leading models have various error values depending on types of data and models, while the meta model consistently has values of less than 5% of mean absolute percentage error (MAPE) and relative root-mean-square error (RRMSE), showing a Nash–Sutcliffe efficiency coefficient(NSE) value of 0.7 or higher. Our proposed data gap analysis framework, including the meta-learning model, can be useful for monitoring the condition of ships, as it can more accurately and robustly classify them as normal or abnormal.