A Real-Time AIS Data Cleaning and Indicator Analysis Algorithm Based on Stream Computing

Abstract

The data quality and real-time analysis of automatic identification system (AIS) are of great significance for water transportation safety and intelligent maritime construction. To improve the AIS data quality and analyze AIS data in real time, a real-time AIS data cleaning and indicator analysis algorithm is proposed. This algorithm performs distributed real-time data cleaning and analysis for massive AIS data based on stream computing technology. It includes data fusion, deduplication, decoding, abnormal data identification, sequencing, prediction, and statistics steps. Abnormal AIS data are repaired by linear regression, multiple trajectory tracking, caching, and other technologies. The AIS status is determined in real-time via multidimensional AIS packet loss analyses, multifactor AIS data statistics, and spatial-temporal data visualization, effectively improving the intelligence level of maritime supervision applications. The proposed algorithm has been running on a production environment, and it monitors AIS data in a certain section of the Yangtze River Basin 24 hours every day without interruption. The operation results show that the proposed algorithm can improve the quality of AIS data, addresses ship trajectory jump issues, and provides timely position updates. The real-time indicator analysis results can provide the data support for ship navigation and maritime supervision.