Reconstruction of incomplete public transportation check-out records by heuristic approaching

Abstract

Smart cards, integrated circuit cards, are frequently used to pay transportation fees in many countries such as the United States, the European Union, Korea and others. Smart card transaction histories in public transportation have been used in making diverse traffic policies. Unfortunately, data preprocessing is required to organize records to extract useful information since the raw data contain artifacts and incomplete information. The paired data, i.e. a boarding and its associated alighting, are one of the key pieces of information required to yield the city traffic demand. However, it is difficult to gather the paired data mainly because the card tagging policies and data collecting methods vary from city to city and from institution to institution. The broken trip links between the boarding and its associated alighting information are the most frequent incomplete data in the smart card histories. Although huge budgets are spent on annual traffic surveys in many countries, there are no accurate Smart card data analysis methods that cover all city cases due to high data complexity and limited base information. Among these two restrictions, limited information is more difficult to resolve because it has a large codomain, thousands of stops, for inference under tiny samples. It gives handicaps of applying well-known high-performance approaches such as deep learning and probability analysis. We propose a reconstruction method by a customized clustering algorithm based on coordinates of transactions-occurred stops, user trip histories and actual vehicle movement trajectories for incomplete trip pairs. The proposed method is derived from evaluating each passenger’s historical clusters and trip patterns. In a blind test 79.1% accuracy is achieved in predicting missing alighting stops while conventional methods reach only 42.3% accuracy.