Urban Traffic Data Imputation With Detrending and Tensor Decomposition

Chuanfei Gong,Yaying Zhang

doi:10.1109/access.2020.2964299

Chuanfei Gong, Yaying Zhang

Open Access

https://doi.org/10.1109/access.2020.2964299

Copy DOI

Abstract

Due to various uncontrollable factors (such as random faulty acquisition equipment and data distortion), urban traffic flow data inevitably suffers from some form of data loss. Finding an effective filling method to estimate the missing data is of great help to the study of transportation networks. Traffic flow during a day are likely to have its regular peak period and off-peak period. For most regions of the urban road network, normally there is a certain trend in the traffic flow data. In this paper, we propose a data imputation method that employs a tensor decomposition approach, which fully considers the characteristics of the traffic flow in both time and space. The proposed method is based on high order singular value decomposition with soft thresholding core. In this method, traffic data are divided into its main trend part and the residual part, which is called detrending. And tensor decomposition is performed on these two parts separately. For each part, dynamic rank method is used to adjust the rank of tensor decomposition. With the actual 214 anonymous road segments with 10 minutes interval data in Guangdong, China, the highway data with 15 minutes interval in Madrid, Spain, and the traffic flow data from PeMS with 5 minutes interval in California, USA. The results of the different models are discussed in the case of continuous data missing and random data missing by different time intervals. In addition, by comparing with other data imputation methods, our method can fill the missing data with better performance.

Highlights

With the rapid development of modern cities and the rising travel needs of humans, the problem of road traffic congestion has become increasingly serious
(2) We analyze the correlation of traffic flow data, and divide the traffic flow tensor into the main trend tensor part and the residual tensor part by singular value decomposition instead of just considering it as a whole, which may lead to improved accuracy of data filling
MODEL AND METHOD we propose a method which combines detrending and tensor decomposition to deal with the traffic data imputation

Summary

INTRODUCTION

With the rapid development of modern cities and the rising travel needs of humans, the problem of road traffic congestion has become increasingly serious. When using the tensor decomposition method to fill missing entries based on existing data, the filling accuracy depends on the degree of correlation between the observed data. With the observations that we mentioned above, an improved tensor decomposition method is proposed to fill the missing traffic data. (1) Tensor decomposition method is employed to fill the traffic flow data, which makes full use of all available observed data to fill the missing data, hoping to avoid the weakness that the prediction-based method may rise because of considering historical data only. (2) We analyze the correlation of traffic flow data, and divide the traffic flow tensor into the main trend tensor part and the residual tensor part by singular value decomposition instead of just considering it as a whole, which may lead to improved accuracy of data filling.

MODEL AND METHOD

IMPROVED TUCKER DECOMPOSITION WITH SOFT THRESHOLDING CORE AND DYNAMIC RANKS

TEST DATA DESCRIPTIO

DATA CORRELATION ANALYSIS

Findings

CONCLUSION