P‐2.26: Research and Implementation of Augmented Reality System Based on Improved ORB Features

Abstract

The ORB-SLAM3 algorithm operates around the matching relationship of feature points. Only when the extracted matching points are sufficient and accurate can the camera pose and the world coordinates of map points be calculated correctly and quickly, then sufficient effective data can be extracted in the environment. It is particularly important. ORB-SLAM3 can extract images up to 30ms/frame. The real-time performance on the PC is very good, but the performance on the embedded side is not good enough. This paper optimizes the problems existing in the feature point extraction and map construction of the monocular ORB-SLAM3 system to improve the illumination invariance, uniformity and human-computer interaction of the algorithm, improve the threshold and improve the extraction accuracy with the help of the IMU sensor. The radius algorithm and YOLOv7 algorithm filter the best points to improve the matching accuracy of feature points. In order to achieve the final effect of augmented reality, a set of augmented reality development system was constructed based on the Unity3D cross-platform game engine. The system guarantees synchronous rendering, particle effects, background video rendering and real-time shadow rendering. At the same time, the interface is packaged and can be connected to the perception positioning algorithm library under different platforms. The augmented reality system is completed by using HoloLens. The system can autonomously detect the environment, Track and record the position of relevant objects in the environment, and at the same time use the visual SLAM perception positioning algorithm to ensure the safety of users, providing a new idea for the application of augmented reality devices.