Principle Of Binocular Stereo Vision Research Articles

A lightweight detection method of pavement potholes based on binocular stereo vision and deep learning

The identification of pavement defects is crucial and must be done efficiently, accurately, and cost-effectively. A commercial-grade sports camera and mobile vehicle work together to detect pavement potholes in a lightweight manner. A dataset of 6,186 images with a resolution of 1500 by 1500 pixels has been constructed based on vehicle-mounted tilted images. Object detection and image segmentation are performed using a single-model MASK R-CNN, and the principle of binocular stereo vision is employed to reconstruct the three-dimensional (3D) structures of potholes. A novel methodology is introduced to calculate the 3D feature parameters of potholes, enabling the precise determination of the pavement pothole damage ratio. An average accuracy of 98% for detection and 94% for segmentation is achieved with the Mask R-CNN model. The proposed algorithms achieve an average DR calculation accuracy of 82%, facilitating precise identification of surface irregularities. The paper presents an intelligent approach for detecting pavement potholes.

Three-dimensional system modeling and design of ecological garden landscape based on the interlaced spatial pattern of light and shadow

The layout and design of ecological landscaping is an important part of the construction and development of modern cities. In the 3D reconstruction of the spatial pattern of the light and shadow interlaced zone of the ecological landscape, the complexity and particularity of the ecological landscape structure make it difficult for the three-dimensional reconstruction stereo matching set to meet the accuracy requirements, and the quality 3D image construction cannot meet the requirements of landscape planning. Based on the principle of binocular stereo vision, a regional feature stereo matching algorithm (rsurf) is used to improve the accuracy of feature matching. Considering that the algorithm is easy to filter out the detailed features of the image, the improved RANSAC algorithm is used to filter the matching results. The experimental results show that in the matching cost test of the optimal matching window, the 15 × window neighborhood has the lowest matching cost, and the generated value in the 100 × 100 source window is 0.824. In the test after matching and fusion, the rsurf algorithm is superior to the surf algorithm in both RMS and PMS error performance, and can better meet the requirements of 3D reconstruction of the binocular vision system. The research content has an important reference for the application of landscape visualization 3D technology, and improves the overall layout effect of landscape landscape.

Impact factors on rockfall movement using high-speed camera binocular stereo vision technology

Rockfall is a frequent and serious geological hazard in mountainous areas worldwide. It is of great significance to fully understand rockfall movement for disaster assessment and prevention. In this paper, based on the principle of binocular stereo vision, a filed experimental monitoring technology was developed to conduct the rockfall movement experiment research. Through the orthogonal experimental method, taking the block velocities obtained by binocular stereo vision system as a reference index, the impact factors such as the slope angles and the falling heights, shapes and masses of blocks on rockfall movement are studied. The lateral deviations and jumping heights of different blocks when they are released from the same falling height and moving along the same slope are presented. The results show that slope angle is the most key factor affecting block movement, while the falling heights, shapes and masses of blocks are the secondary factors. Moreover, the experimental methods and results of this paper can lay a foundation for the further study of rockfall movement characteristics.

Wave Surface Identification Based on Stereo Vision and Wave Theory: an Initial Attempt

Wave height is the most basic element of wave information, and accurate measurement of wave height plays a crucial role in both understanding wave theory and verifying numerical models. In this paper, an improved optical measurement system was proposed based on the principle of binocular stereo vision, which breaks through the limitations of traditional measurement equipment such as wave gauges and other single-point measurements, and improves the commonly used optical measurement method that relies solely on images by involving water wave theory into the data post-processing procedure. By identifying and reconstructing the unidirectional regular wave surface in a towing tank and comparing the results with the wave gauges and theoretical incoming wave parameters for validation, the results show that the measurement system has an error of about 1% in the measurement of a wide area. In the end of the paper, a preliminary attempt is made on the irregular wave surface identification.

A Stereo Matching Method for 3D Image Measurement of Long-Distance Sea Surface

Tsunamis are some of the most destructive natural disasters. Some proposed tsunami measurement and arrival prediction systems use a limited number of instruments, then judge the occurrence of the tsunami, forecast its arrival time, location and scale. Since there are a limited number of measurement instruments, there is a possibility that large prediction errors will occur. In order to solve this problem, a long-distance tsunami measurement system based on the binocular stereo vision principle is proposed in this paper. The measuring range is 4–20 km away from the system deployment site. In this paper, we will focus on describing the stereo matching method for the proposed system. This paper proposes a two-step matching method. It first performs fast sparse matching, and then complete high precision dense matching based on the results of the sparse matching. A matching descriptor based on the physical features of sea waves is proposed to solve the matching difficulty caused by the similarity of sea surface image textures. The relationship between disparity and the y coordinate is built to reduce the matching search range. Experiments were conducted on sea surface images with different shooting times and distances; the results verify the effectiveness of the presented method.

New method and system of automatically moving a CMM probe tip into holes with submillimeter diameter and high aspect ratio

The demand of measuring the internal topography of holes with submillimeter diameter and high aspect ratio is increasing. However, moving a coordinate measuring machine (CMM) probe tip into a submillimeter hole manually is extremely difficult because the diameters of the probe tip and the hole are extremely small to be seen clearly with human eyes. An automatic measuring method and system is presented in this study to simplify the operation and improve the measuring efficiency. A micro CMM system was built. Two cameras were used to image the probe tip and the holes to be measured. The obtained images were magnified, and the edges of the probe tip and the holes were extracted. The image coordinates of the center points of the probe tip and the hole were obtained. The center coordinate values of the probe tip and the hole in the world coordinate system were achieved using the principle of binocular stereo vision. The probe tip was moved into the holes by the differences in the obtained coordinate values. Experiments were conducted. The effectiveness of the proposed method and system was verified by moving the probe tip into a hole with 1 mm diameter. This method can be used to assist in the measurement of deep submillimeter holes, especially in nonbatch measurement situations.

Application of integrated binocular stereo vision measurement and wireless sensor system in athlete displacement test

Based on binocular stereo vision measurement and wireless sensing theory, this paper studies the measurement and analysis methods of athletes' movement displacement parameters. The main purpose is to explore the relationship between sports biomechanical parameters. The analysis reveals the laws of athletes 'movements, and provides theoretical guidance on athletes' attitude recognition and physical training. Based on the principle of binocular stereo vision, a non-contact displacement measurement system was developed. The displacement measurement system uses template matching to extract the image coordinates of the measurement points and restore the spatial coordinates of the measurement points through European 3D reconstruction, which improves the practicality of the measurement system. In the cantilever beam vibration test, the traditional displacement sensor and binocular image displacement measurement system are used to simultaneously measure the cantilever beam displacement. The experimental results show that the integrated displacement measurement system in this paper has higher accuracy and better stability.

Panoramic Stereo Imaging of a Bionic Compound-Eye Based on Binocular Vision.

With the rapid development of the virtual reality industry, one of the bottlenecks is the scarcity of video resources. How to capture high-definition panoramic video with depth information and real-time stereo display has become a key technical problem to be solved. In this paper, the optical optimization design scheme of panoramic imaging based on binocular stereo vision is proposed. Combined with the real-time processing algorithm of multi detector mosaic panoramic stereo imaging image, a panoramic stereo real-time imaging system is developed. Firstly, the optical optimization design scheme of panoramic imaging based on binocular stereo vision is proposed, and the space coordinate calibration platform of ultra-high precision panoramic camera based on theodolite angle compensation function is constructed. The projection matrix of adjacent cameras is obtained by solving the imaging principle of binocular stereo vision. Then, a real-time registration algorithm of multi-detector mosaic image and Lucas-Kanade optical flow method based on image segmentation are proposed to realize stereo matching and depth information estimation of panoramic imaging, and the estimation results are analyzed effectively. Experimental results show that the stereo matching time of panoramic imaging is 30 ms, the registration accuracy is 0.1 pixel, the edge information of depth map is clearer, and it can meet the imaging requirements of different lighting conditions.

Three-dimensional spine morphology measuring technology for daily surface monitoring

In order to conduct surface monitoring of the three-dimensional spine morphology of the human body in daily life, a spine morphology measuring method using "single camera, multi-view" to construct stereo vision is proposed. The images of the back of the human body with landmarks of spinous process are captured from multiple angles by moving a single camera, and based on the "Zhang Zhengyou calibration method" and the triangulation principle of binocular stereo vision, the spatial conversion matrices corresponding to each other between all images and the 3D coordinates of the landmarks are calculated. Then the spine evaluation angle used to evaluate the spine morphology is further calculated. The tests' results showed that the spine evaluation angle error of this method is within ±3°, and the correlation between the results and the X-ray film Cobb angles is 0.871. The visual detection algorithm used in this paper is non-radioactive, and because only one camera is used in the measurement process and there is no need to pre-set the camera's shooting pose, the operation is simple. The research results of this article can be used in a mobile phone-based intelligent detection system, which will be suitable for the group survey of scoliosis in communities, schools, families and other occasions, as well as for the long-term follow-up of confirmed patients. This will provide a reference for doctors to diagnose the condition, predict the development trend of the condition, and formulate treatment plans.

Novel and Automatic Rice Thickness Extraction Based on Photogrammetry Using Rice Edge Features.

The dimensions of phenotyping parameters such as the thickness of rice play an important role in rice quality assessment and phenotyping research. The objective of this study was to propose an automatic method for extracting rice thickness. This method was based on the principle of binocular stereovision but avoiding the problem that it was difficult to directly match the corresponding points for 3D reconstruction due to the lack of texture of rice. Firstly, the shape features of edge, instead of texture, was used to match the corresponding points of the rice edge. Secondly, the height of the rice edge was obtained by way of space intersection. Finally, the thickness of rice was extracted based on the assumption that the average height of the edges of multiple rice is half of the thickness of rice. According to the results of the experiments on six kinds of rice or grain, errors of thickness extraction were no more than the upper limit of 0.1 mm specified in the national industry standard. The results proved that edge features could be used to extract rice thickness and validated the effectiveness of the thickness extraction algorithm we proposed, which provided technical support for the extraction of phenotyping parameters for crop researchers.

Experimental study on joint stiffness with vision-based system and geometric imperfections of temporary member structure

In this paper, tests of plug-pin joints are conducted in order to obtain their mechanical parameters, including semi-rigid property. To solve the difficulties of multi-point displacement measurements for small joints, this investigation proposes a vision-based measurement system based on the principle of binocular stereo vision to improve measurement accuracy. Accurate sub-pixel location is achieved according to a template-matching algorithm based on grayscale. Joint performance, including horizontal bar joint tension and compression, semi-rigidity between horizontal bars and upright rods and bracing tension and compression, is investigated in order to acquire joint failure modes as well as load and displacement (or moment and rotation angle) curves. Through data fitting, multi-linear simplified models are proposed to illustrate the joints’ mechanical performance. This paper also investigates geometric imperfection of temporary member structure with plug-pin joints based on several substructure models and temporary grandstand units using a total station theodolite. The probabilistic models of initial member out-of-straightness and story frame out-of-plumb have been acquired, which can be used into Monte Carlo simulation to create stochastic model of the temporary member structure.

The Application of Vision Principle in 3D Reconstruction

3D reconstruction technology provides digital model of the actual product for the CAD/CAM system of the reverse engineering, which has practical significance for promoting the development of manufacturing automation. Meanwhile, the digital models through 3D reconstruction have the important applications in the fields of virtual reality, video conferencing, criminal investigation and cosmetic surgery. Especially the research for 3D reconstruction of freedom surface is an important research topic in the computer graphics. In this paper, the technology of 3D reconstruction based on stereo vision principle is presented, which can effectively realize reconstruction of 3D digital model of freedom surface. Firstly, the experimental system of stereo vision is established consisting of two CCD cameras in accordance with the principle of binocular stereo vision. Then, the calibration of stereo vision system is completed by selecting the BP neural network for simulating the mapping relation between the images in left and right imaging planes and 3D spatial objects after comparing the two calibration methods of genetic algorithm and BP neural network. Finally, the vision system is experimentally verified taking the face of plaster and human foot as examples, and very good results are achieved.

Panoramic camera on the Yutu lunar rover of the Chang'e-3 mission

The Chang'e-3 panoramic camera, which is composed of two cameras with identical functions, performances and interfaces, is installed on the lunar rover mast. It can acquire 3D images of the lunar surface based on the principle of binocular stereo vision. By rotating and pitching the mast, it can take several photographs of the patrol area. After stitching these images, panoramic images of the scenes will be obtained. Thus the topography and geomorphology of the patrol area and the impact crater, as well as the geological structure of the lunar surface, will be analyzed and studied. In addition, it can take color photographs of the lander using the Bayer color coding principle. It can observe the working status of the lander by switching between static image mode and dynamic video mode with automatic exposure time. The focal length of the lens on the panoramic camera is 50 mm and the field of view is 19.7° × 14.5°. Under the best illumination and viewing conditions, the largest signal-to-noise ratio of the panoramic camera is 44 dB. Its static modulation transfer function is 0.33. A large number of ground testing experiments and on-orbit imaging results show that the functional interface of the panoramic camera works normally. The image quality of the panoramic camera is satisfactory. All the performance parameters of the panoramic camera satisfy the design requirements.

Surface Reconstruction of Stamping Parts Based on Binocular Stereo Vision

In order to realize the 3D reconstruction of stamping parts surface, this paper based on binocular stereo vision principle firstly introduces the model of the binocular cameras. Internal and external parameters of camera can be obtained by binocular calibration, taking the printed circle grid centers which are on the stamping parts as feature points, and then using the disparity image obtained by HALCON to reconstruct 3D information of the feature points. Finally, use Matlab to plot out the scatter diagram of feature points and the fitting curved surface diagram.

A Method of Line Structured Light Vision System Calibration Based on Stereo Vision

It is a difficult task to get enough numbers of highly accurate control points for projector calibration in line structured light vision system. A new projector calibration method based on binocular stereo vision is proposed in this paper. Two cameras calibration can be done usingtraditional camera calibration method and they composed a binocular stereo vision. In projector calibration procedure, a planar template was located at several different positions in front of the stereo vision system. Two cameras captured the stripe images in each position simultaneously. Every center points of the laser stripe can be used as control points of the projector plane. The 3D coordinate of the stripe center points can be obtained through binocular stereo vision principle easily. So the light plane can be calculated quickly. Experiments were carried out and the result shows that the proposed method is flexible and stable.

System Design of Crane Robot Based on Binocular Stereo Vision

In order to improve the safety, high efficiency and stability in the process of the crane hoisting, this paper applied binocular stereo vision technology which has developed into a rather mature stage recently. Binocular stereo vision system could efficiently identify and accurately position various targets, so that it could decrease the crane driver’s working intensity, reduce security risks and improve the production efficiency. In order to experiments was convenient, developing the crane robot system could simulate the processes of crane work, establishing the relevant system structure model for the motion. According to the three-dimensional measuring principle of binocular stereo vision, this paper is proposed to identify and position for the targets, and experimental results show the error is ±10mm.

基于双目立体视觉的图像面积测量算法与技术<br>Algorithm and Technology of Image Area Measurement Based on Binocular Stereo Vision

本文基于双目立体视觉测距原理，用普通的USB摄像头搭建双目立体视觉系统，提出了图像面积的非接触在线测量方法。首先对摄像头进行标定，得到摄像头的内参数，然后根据OpenCV的模板匹配算法，求出两幅图像中对应匹配点的水平视差，得到距离参数，最后根据平行双目视觉的成像原理，求得实际图像与其在成像平面上的图像之间的空间几何关系，进而求出图像的实际面积。该方法实现简单、成本低、实时性好。经过实验验证，该算法在一定测量范围内达到精度要求。 A binocular stereo vision system with a common USB camera is developed based on the distance measurement principle of binocular stereo vision, and the non-contact and on-line inspection method for the area of image is studied. First, we calibrate the camera and obtain the internal parameters of the camera. Second, horizontal parallax of the cor- responding points in two images is calculated by template matching based on OpenCV, and then the distance parameter is obtained. Finally, we find the spatial geometrical relationship between the image of the imaging plane and the actual image, and then the actual area of image is gotten. This method is easy to realize with low cost and real-time perform- ance. The experiment indicates that the algorithm meets the accuracy requirements within certain range of measurement.

Research on Localization of Vehicle Backwindow for Binocular Distance Measurement

The scene distance information is obtained by disparity based on binocular stereo vision. It is widely used in scientific research and practical engineering. Two cameras which fixed on ahead of the vehicle are used to capture stereo image pairs. Then the backwindows of the two images are located using Hough transform. The top left corner of the windows can be treated as a pair of matching points to calculate the distance of the front vehicle according to the principle of binocular stereo vision. The distance measurement scheme is presented. The experimental results demonstrated that the algorithm provides accurate localization and can help driver to deal with traffic problem to some extent.