Towards Semantic SLAM: 3D Position and Velocity Estimation by Fusing Image Semantic Information with Camera Motion Parameters for Traffic Scene Analysis

Robert Piché,Mostafa Mansour,Pavel Davidson,Oleg Stepanov

doi:10.3390/rs13030388

Robert Piché, Mostafa Mansour + Show 2 more

Open Access

https://doi.org/10.3390/rs13030388

Copy DOI

Abstract

In this paper, an EKF (Extended Kalman Filter)-based algorithm is proposed to estimate 3D position and velocity components of different cars in a scene by fusing the semantic information and car model, extracted from successive frames with camera motion parameters. First, a 2D virtual image of the scene is made using a prior knowledge of the 3D Computer Aided Design (CAD) models of the detected cars and their predicted positions. Then, a discrepancy, i.e., distance, between the actual image and the virtual image is calculated. The 3D position and the velocity components are recursively estimated by minimizing the discrepancy using EKF. The experiments on the KiTTi dataset show a good performance of the proposed algorithm with a position estimation error up to 3–5% at 30 m and velocity estimation error up to 1 m/s.

Highlights

In recent years, significant progress has been made in vision-based SimultaneousLocalization and Mapping (SLAM) to allow a robot to map its unknown environment and localize itself in it [1]
The filter can be initialized by a direct 3D position measurement from a stereo camera or from a monocular camera [31,32] depending on the distance of the object [33]
We proposed an algorithm to estimate 3D position and velocity components of different cars in a scene

Summary

Introduction

Localization and Mapping (SLAM) to allow a robot to map its unknown environment and localize itself in it [1]. Many works have been dedicated to the use of geometric entities such as corners and edges to produce a dense feature map in the form of a 3D point cloud. The geometric aspect of SLAM has reached a level of maturity allowing it to be implemented in real time with high accuracy [2,3] and with an outcome consisting of a camera pose and sparse map in the form of a point cloud. Despite the maturity and accuracy of geometric SLAM, it is inadequate when it comes to any interaction between a robot and its environment. To interact with an environment, a robot should have a meaningful map with object-based entities instead of geometric ones. The robot should reach a level of semantic understanding allowing it to distinguish between different objects and their properties and to distinguish between different instances of the same object

Results

Discussion

Conclusion