This paper is based on multi-sensor fusion technology of unmanned vehicle position to research and design a kind of scheme could choose to use adaptive visual data, GPS, INS, solved the problem of poor application effect of multiple scenarios on unmanned vehicles. This paper will integrate ORB_SLAM2 monocular vision and INS, compared with the traditional method to IMU and visual data separated filtering, we designed a general nonlinear optimization framework could bring visual feature points and the noise of the IMU data into it to optimize the camera position. Not only could scale and sensor bias be accurately estimated, could also improve the robustness and position precision of the unmanned vehicle. When the GPS signal is better, the algorithm combines the GPS data with the kalman filtering algorithm, which could be used to estimate the position of the GPS data more accurately, and can also assist the construction of visual map. In addition, it also shows good performance in data delay handling. By test in Euroc datasets and real vehicle, effect is good, in large scale outdoor scenarios also has higher position accuracy and robustness, the error is less than 0.1 meters, and has a strong practical value.