Design of a Multi-modal Sensor Fusion Unmanned Vehicle System based on Computer Vision

Abstract

With the development of artificial intelligence and Internet of Things technology, more and more intelligent products are appearing around us, such as sweeping robots, library service robots and drug delivery robots, etc. In the practical application of these intelligent robots, the core technology of spatial positioning is inseparable, and considering the cost, signal interference and positioning accuracy, it is necessary to study the positioning technology with low cost, small size and strong anti-interference factors. To solve the problem of poor performance of positioning accuracy when using low-cost sensors in the physical environment, we use a four-wheel drive vehicle model as a carrier to build an unmanned vehicle system based on multi-modal sensor fusion, binocular vision localization and other technologies. The core of the system is the MM32F3277G9P chip from MindMotion and the Raspberry Pi embedded development board. The proposed vision information is based on the Intel Realsense T265 camera, which is fused with the data from the nine-axis inertial measurement unit (IMU) and the dual-frequency global positioning system (GPS), so that the positioning algorithm can continuously provide robust and accurate state estimation results in the physical environment through the complementary advantages.