Binocular Stereo Vision Research Articles

Recognition and Depth Estimation of Ships Based on Binocular Stereo Vision

To improve the navigation safety of inland river ships and enrich the methods of environmental perception, this paper studies the recognition and depth estimation of inland river ships based on binocular stereo vision (BSV). In the stage of ship recognition, considering the computational pressure brought by the huge network parameters of the classic YOLOv4 model, the MobileNetV1 network was proposed as the feature extraction module of the YOLOv4 model. The results indicate that the mAP value of the MobileNetV1-YOLOv4 model reaches 89.25%, the weight size of the backbone network was only 47.6 M, which greatly reduced the amount of computation while ensuring the recognition accuracy. In the stage of depth estimation, this paper proposes a feature point detection and matching algorithm based on the ORB algorithm at sub-pixel level, that is, firstly, the FSRCNN algorithm was used to perform super-resolution reconstruction of the original image, to further increase the density of image feature points and detection accuracy, which was more conducive to the calculation of the image parallax value. The ships’ depth estimation results indicate that when the distance to the target is about 300 m, the depth estimation error is less than 3%, which meets the depth estimation needs of inland ships. The ship target recognition and depth estimation technology based on BSV proposed in this paper makes up for the shortcomings of the existing environmental perception methods, improves the navigation safety of ships to a certain extent, and greatly promotes the development of intelligent ships in the future.

Robot Obstacle Recognition and Target Tracking Based on Binocular Vision

To further improve the perception ability of binocular vision sensor in getting rich environment information and scene depth information, a research method of a robot obstacle recognition and target tracking based on binocular vision was proposed. The method focused on target recognition and obstacle recognition of binocular stereo vision. The system based on obstacles of visual identification was set up. Through the analysis of the Bouguet mathematical model algorithm based on OpenCV, the binocular stereo correction was carried out and the obstacle recognition system was calibrated and corrected. Through the experimental data, it was found that the average error of the obstacle recognition and target tracking algorithm based on binocular vision could be controlled within 50 mm within the range of 2100 mm. The average time of obstacle recognition was 0.096 s and the average time consumption of the whole system was 0.466 s, indicating that the robot obstacle recognition and target tracking system based on binocular vision could meet the accuracy and real-timeness requirements of obstacle recognition and detection.

Design of binocular stereo vision optical system based on a single lens and a single sensor.

To reduce the number of detectors used in conventional binocular stereo cameras, while improving the measurement accuracy and compactness of the system, this paper proposes a design method for a binocular stereo vision optical system based on a single lens and a single sensor. First, based on the design principle of the traditional binocular optical system, to the best of our knowledge, a novel method of designing a framing lens array at the optical stop of the optical system is proposed to image two images on one detector simultaneously. Second, we propose a dual-frame lens array design method at the aperture stop position of the optical system that can image two images on one detector simultaneously. Then, the design principle of the method is analyzed theoretically, as well as a detailed analysis of the imaging position layout and the stray light elimination method of the dual-channel optical system. Finally, a single-lens binocular optical system with a focal length of 20 mm and a full field of view of 30° is designed using the method in this paper, and the analysis results demonstrate that the system has the advantages of good imaging quality and compact construction and provides a design idea for the design of a binocular stereo vision optical system.

Autonomous environment perception and obstacle avoidancetechnologies of Zhurong Mars rover

<p indent="0mm">Mars surface exploration has the characteristics and constraints of harsh and complex environment, lack of prior knowledge, and strict limitation of computing resources. Autonomous environment perception and obstacle avoidance technology is the key to the ability of autonomous exploration, and also one of the key technologies in the field of deep space exploration at home and abroad. In China Tianwen-1 Mars exploration mission, the Zhurong Mars rover, China’s first Mars surface mobile exploration, has realized efficient autonomous environmental perception, obstacle avoidance planning and control under the constraints of limited resources, significantly improved the efficiency of mobile exploration, and the average moving speed is nearly 5 times higher than that of Yutu-2 lunar rover. This paper briefly introduces the sensor configuration, movement mode function and process of Zhurong Mars rover supporting autonomous perception and obstacle avoidance movement, and focuses on the breakthroughs in key technologies of efficient perception and obstacle avoidance planning, including binocular stereo vision efficient perception method based on fast parallel processing, in order to realize the efficiency and safety of autonomous perception and obstacle avoidance movement under severe resource constraints Based on the environmental modeling and obstacle avoidance planning method of trafficability analysis. Finally, the autonomous perception and obstacle avoidance mobile detection of Zhurong Mars rover during the mission is summarized, and the technical development thoughts on further enhancing the autonomous perception and obstacle avoidance ability of Mars rover in the future are put forward in the end.

Image Recognition of Sports Athletes' High-Intensity Sports Injuries Based on Binocular Stereo Vision.

Sports athletes are prone to certain injuries during high-intensity exercise training. In the process of treating an injury, images of the injury site need to be collected and identified. However, the traditional recognition method cannot effectively extract the features of the image. At the same time, it ignores the optimization of the damage image recognition results, resulting in low recognition accuracy and poor efficiency. Binocular stereo vision technology can quickly and accurately detect moving objects. Therefore, in order to more accurately identify high-intensity sports injury images, this study takes the high-intensity sports injury images as the basic research object. Several processes of image processing based on binocular stereo vision are analyzed, and the vulnerable parts of the body in high-intensity sports are also studied. Finally, the method in this study is verified. The experimental results show that the method proposed in this study reduces the average error rate by 0.19% compared with the traditional recognition method. It can effectively identify and detect injury images, thereby improving the accuracy and stability of sports injury image identification. The identification time is also shortened accordingly, which has certain practicability and feasibility. In addition, the binocular camera used in this study has high accuracy, and the obtained images of sports injuries are of good quality, which lays a foundation for image detection and recognition.

Underwater fish mass estimation using pattern matching based on binocular system

Fish mass is the main information for judging growth status, regulating water quality environment, and precision feeding and grading in the process of intelligent aquaculture management activities. However, the occlusion, bending, and poor imaging angle of fish body image are still serious challenges for underwater fully automatic mass measurement. The aim of this study was to develop underwater non-contact method to automatically estimate the free-swimming fish mass based on binocular stereo vision technology. The fish body images were automatically selected and obtained by using pattern recognition method based on LabVIEW development platform during the experimental period. All the fish samples were divided into three groups according to mass (200–500 g, 500–800 g, and 800–1200 g), and then subdivided into three groups by imaging angle (orthogonal angles, greater than 45°, and less than 45°). The experiment indicated that the fish mass could be estimated using fish body area with a high coefficient of determination (R2) based on linear model. The mean relative errors between estimated and measured value were 3.37% (orthogonal angles), 4.95% (greater than 45° angles), and 16.59% (less than 45° angles). Significant difference was found in less than 45° group with p < 0.01. These findings showed that the approach put forward in this research could realize fully automatic mass estimation for underwater free-swimming fish and effectively improve the estimation robustness and efficiency.

Pineapple (Ananas comosus) fruit detection and localization in natural environment based on binocular stereo vision and improved YOLOv3 model

Pineapple (Ananas comosus) fruit detection and localization in natural environment based on binocular stereo vision and improved YOLOv3 model

Deployment of a deep-learning based multi-view stereo approach for measurement of ship shell plates

In this work, a measurement system integrated with a deep-learning based multi-view stereo (MVS) approach is developed to measure ship shell plates. Specifically, a deep learning architecture of CasMVSNet is deployed for depth map inference from multi-view images, which can remarkably decrease the dense reconstruction time and GPU memory consumption. This is the first report that a learned MVS architecture is deployed to three-dimensional (3D) reconstruction of ship shell plates. Measurement experiments are then performed for three typical hull plates to evaluate the accuracy, efficiency and completeness of the proposed measurement method. The results suggest that the complete point cloud data of the curved hull plates can be reconstructed in about 3 min with the average of errors less than 1 mm, which fulfills the requirements of precision and efficiency in shipbuilding production. Compared with traditional wooden templates, the proposed measurement method is more accurate, efficient and inexpensive. The developed measurement system with quantitative data can also be readily integrated with the 3D computer numerical control (CNC) plate bending machine. Moreover, the robustness and flexibility of the proposed measurement method have been verified by comparison with the measurement method based on active binocular stereovision.

A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

The human face is an important channel for human interaction and is the most expressive part of the human body with personalized and diverse characteristics. To improve the modeling speed as well as recognition speed and accuracy of 3D faces, this paper proposes a laser scanning binocular stereo vision imaging method based on binocular stereo vision and depth-sensing detection method. Existing matching methods have a weak anti-interference capability, cannot identify the spacing of objects quickly and efficiently, and have too large errors. Using the technique mentioned in this paper can remedy these shortcomings by mainly using the laser lines scanned onto the object as strong feature cues for left and right views for binocular vision matching and thus for depth measurement perception of the object. The experiments in this paper first explore the effect of parameter variation of the laser scanning binocular vision imaging system on the accuracy of object measurement depth values in an indoor environment; the experimental data are selected from 68 face feature points for modeling, cameras that photograph faces, stereo calibration of the cameras using calibration methods to obtain the parameters of the binocular vision imaging system, and after stereo correction of the left and right camera images, the laser line scan light is used as a pixel matching strong features for binocular camera matching, which can achieve fast recognition and high accuracy and then select the system parameters with the highest accuracy to perform 3D reconstruction experiments on actual target objects in the indoor environment to achieve faster recognition.

Method for improving the measurement accuracy of binocular stereo vision in a scattering environment.

In the scattering environment, binocular stereo vision measurement technology produces large errors due to the change of refractive index of the imaging light path and the decrease in target image contrast. To address this problem, this paper proposes a method for improving the measurement accuracy of binocular stereo vision in a scattering environment combined with polarization imaging theory. First, scattering images with different polarization directions are obtained and filtered by a Gaussian low-pass filter to calculate the degree of polarization and angle of polarization. Then, the scattered light intensity is calculated by using polarization information to obtain images after removing the scattering. Second, feature extraction and matching are carried out for the images after scattering removal. Finally, the target is measured based on the binocular stereo vision measurement model. The experimental results show that when the scattering concentration is high enough, the conventional method can no longer perform measurement, but the method proposed in this paper can still obtain the target parameters at this time, and can also improve measurement accuracy by at least 46.30%. In conclusion, the proposed method provides a valuable reference to improve the accuracy of binocular stereo vision measurement in a scattering environment by reducing the interference of scattering light.

Body size measurement based on deep learning for image segmentation by binocular stereovision system

Body size measurement based on deep learning for image segmentation by binocular stereovision system

4D dual-mode staring hyperspectral-depth imager for simultaneous spectral sensing and surface shape measurement.

A 4D dual-mode staring hyperspectral-depth imager (DSHI), which acquire reflectance spectra, fluorescence spectra, and 3D structural information by combining a staring hyperspectral scanner and a binocular line laser stereo vision system, is introduced. A 405 nm laser line generated by a focal laser line generation module is used for both fluorescence excitation and binocular stereo matching of the irradiated line region. Under the configuration, the two kinds of hyperspectral data collected by the hyperspectral scanner can be merged into the corresponding points in the 3D model, forming a dual-mode 4D model. The DSHI shows excellent performance with spectral resolution of 3 nm, depth accuracy of 26.2 µm. Sample experiments on a fluorescent figurine, real and plastic sunflowers and a clam are presented to demonstrate system's with potential within a broad range of applications such as, e.g., digital documentation, plant phenotyping, and biological analysis.

Detection of bulk feed volume based on binocular stereo vision

The volume detection of medical mice feed is crucial to understand the food intake requirements of mice at different growth stages and to grasp their growth, development, and health status. Aiming at the problem of volume calculation in the way of feed bulk in mice, a method for detecting the bulk volume of feed in mice based on binocular stereo vision was proposed. Firstly, the three-dimensional point coordinates of the feed's surface were calculated using the binocular stereo vision three-dimensional reconstruction technology. The coordinates of these dense points formed a point cloud, and then the projection method was used to calculate the volume of the point cloud; and finally, the volume of the mice feed was obtained. We use the stereo matching data set provided by the Middlebury evaluation platform to conduct experimental verification. The results show that our method effectively improves the matching degree of stereo matching and makes the three-dimensional point coordinates of the obtained feed's surface more accurate. The point cloud is then denoised and Delaunay triangulated, and the volume of the tetrahedron obtained after the triangulation is calculated and summed to obtain the total volume. We used different sizes of wood instead of feed for multiple volume calculations, and the average error between the calculated volume and the real volume was 7.12%. The experimental results show that the volume of the remaining feed of mice can be calculated by binocular stereo vision.

Research and Implementation of Sports Entity Simulation Based on Heterogeneous Binocular Vision

To study the value of heterogeneous binocular vision in the detection of sports targets, a Zynq-based joint software and hardware design method is proposed, and a mobile object detection system for binocular vision is developed based on it. This paper introduces the relevant technologies and theories used in the system design. First, the basic principles and various modules of the binocular stereo vision process are introduced, including camera tuning, stereo correction, stereo tuning, and telescope stereo vision. Based on the depth information, a method to detect and measure motor targets was developed. Finally, combined with the improved mobile target detection algorithm, the real-time methodology research and algorithm design to determine the target range are completed. The performance of the moving target detection system based on binocular stereo vision and the moving target detection and ranging algorithm are tested and analyzed. We know that the front and rear width of the moving target (the person tested in this paper) is at least 10 cm, while the error value of the ranging algorithm in this paper is within 6 meters, and the average error is less than 10 cm. According to the characteristics of binocular stereo vision, the farther the viewing angle is, the smaller the parallax value of the left and right image pixels will be. Therefore, the insignificant change of depth information will bring some errors to the ranging algorithm. The greater the measurement error is, the lower the percentage deviation of the measurement result is within 2%. It also shows that the target ranging algorithm in this paper can ensure good accuracy within a certain distance. Through this method, we can obtain the accurate data of moving objects in sports competition, so as to improve the training method of athletes and improve the competition results.

Research on Interior Design and Space Layout Optimization Based on Multi-Intelligent Decision-Making

In order to meet people’s diverse, complex, and changeable living requirements and aesthetic requirements, it is necessary to rationalize the design of indoor three-dimensional space. The optimization of the indoor environmental space layout is affected by the energy consumption and rationality of functional areas, resulting in indoor environmental space The performance of the layout optimization system deteriorates. In order to improve the performance of the indoor environment spatial layout and meet the living needs of more people, this paper proposes an optimal design of a spatial layout based on multi-intelligence decision-making. The research results of the article show that (1) the average confidence and the average recognition time of logo set B are both smaller than those of logo set A , indicating that logo set B is easier to be recognized by users in the same commercial building environment and can be more quickly recognized. In terms of discrimination, the time used to correct the misrecognition of identification set B is shorter than that of identification set A , which means that the different types of identifications in identification set B are easier to distinguish. (2) The spatial positioning algorithm proposed in this paper has good reliability and practicability, high positioning accuracy, and small error for indoor spatial coordinate positioning. Compared with the traditional method, the projection method of this paper has a significantly smaller error, and the maximum error is only 0.272, which makes the indoor space design more rational. (3) Comparing the energy consumption coefficients of the three spatial layout optimization systems, it can be seen that the indoor environmental spatial layout optimization system based on multi-intelligence decision-making has the lowest energy consumption when optimizing the indoor environmental spatial layout, because the system is in the design process. The indoor environment space layout model is designed, which effectively reduces the energy consumption of the indoor environment space layout optimization. When the indoor environment space layout optimization system based on multi-intelligence decision-making is used to optimize the indoor environment space layout, the average optimization accuracy is 98.32%, while the average indoor environment space layout optimization accuracy of the space layout optimization system with curved shading space layout optimization is 42.2% %, the layout optimization of binocular stereo vision space. When optimizing the indoor space layout, the average optimization accuracy is 70.87%.

Motion Track Enhancement Method of Sports Video Image Based on OTSU Algorithm

Current sports video image tracking methods cannot get the average gray information of pixels, which leads to the low segmentation accuracy of target trajectory. Therefore, this paper proposes a method of enhancing the motion track of sports video object based on OTSU algorithm. The OTSU algorithm is introduced to segment the unsupervised image automatically based on threshold calculation. Combined with 3D histogram, the OTSU algorithm is optimized to overcome the influence of noise and nonuniform illumination and to improve the segmentation performance. Using the optimized OTSU algorithm, the preprocessing of sports video image is realized by gray level transform, median filter, and binarization. Based on this, the object motion model of sports video image is constructed, and the edge information of sports video image is extracted. Using binocular stereo vision technology, according to the image target motion model and the current space coordinates, calculate the motion track, and obtain the parameters of sports video image. Experimental results show that the proposed method can segment the target accurately in a shorter time, and the enhancement effect of target trajectory is obvious, and the error is small.

Spatiotemporal Matching Cost Function Based on Differential Evolutionary Algorithm for Random Speckle 3D Reconstruction

Random speckle structured light can increase the texture information of the object surface, so it is added in the binocular stereo vision system to solve the matching ambiguity problem caused by the surface with repetitive pattern or no texture. To improve the reconstruction quality, many current researches utilize multiple speckle patterns for projection and use stereo matching methods based on spatiotemporal correlation. This paper presents a novel random speckle 3D reconstruction scheme, in which multiple speckle patterns are used and a weighted-fusion-based spatiotemporal matching cost function (STMCF) is proposed to find the corresponding points in speckle stereo image pairs. Furthermore, a parameter optimization method based on differential evolutionary (DE) algorithm is designed for automatically determining the values of all parameters included in STMCF. In this method, since there is no suitable training data with ground truth, we explore a training strategy where a passive stereo vision dataset with ground truth is used as training data and then apply the learned parameter value to the stereo matching of speckle stereo image pairs. Various experimental results verify that our scheme can realize accurate and high-quality 3D reconstruction efficiently and the proposed STMCF exhibits superior performance in terms of accuracy, computation time and reconstruction quality than the state-of-the-art method based on spatiotemporal correlation.

High Precision Calibration Algorithm for Binocular Stereo Vision Camera using Deep Reinforcement Learning.

Camera calibration is the most important aspect of computer vision research. To address the issue of insufficient precision, therefore, a high precision calibration algorithm for binocular stereo vision camera using deep reinforcement learning is proposed. Firstly, a binocular stereo camera model is established. Camera calibration is mainly divided into internal and external parameter calibration. Secondly, the internal parameter calibration is completed by solving the antihidden point of the camera light center and the camera distortion value of the camera plane. The deep learning fitting value function is used based on the internal parameters. The target network is established to adjust the parameters of the value function, and the convergence of the value function is calculated to optimize reinforcement learning. The deep reinforcement learning fitting structure is built, the camera data is entered, and the external parameter calibration is finished by continuous updating and convergence. Finally, the high precision calibration of the binocular stereo vision camera is completed. The results show that the calibration error of the proposed algorithm under different sizes of checkerboard calibration board test is only 0.36% and 0.35%, respectively, the calibration accuracy is high, the value function converges quickly, and the parameter calculation accuracy is high, the overall time consumption of the proposed algorithm is short, and the calibration results have strong stability.

Dental Implant Navigation System Based on Trinocular Stereo Vision.

Traditional dental implant navigation systems (DINS) based on binocular stereo vision (BSV) have limitations, for example, weak anti-occlusion abilities, as well as problems with feature point mismatching. These shortcomings limit the operators’ operation scope, and the instruments may even cause damage to the adjacent important blood vessels, nerves, and other anatomical structures. Trinocular stereo vision (TSV) is introduced to DINS to improve the accuracy and safety of dental implants in this study. High positioning accuracy is provided by adding cameras. When one of the cameras is blocked, spatial positioning can still be achieved, and doctors can adjust to system tips; thus, the continuity and safety of the surgery is significantly improved. Some key technologies of DINS have also been updated. A bipolar line constraint algorithm based on TSV is proposed to eliminate the feature point mismatching problem. A reference template with active optical markers attached to the jaw measures head movement. A T-type template with active optical markers is used to obtain the position and direction of surgery instruments. The calibration algorithms of endpoint, axis, and drill are proposed for 3D display of the surgical instrument in real time. With the preoperative path planning of implant navigation software, implant surgery can be carried out. Phantom experiments are carried out based on the system to assess the feasibility and accuracy. The results show that the mean entry deviation, exit deviation, and angle deviation are 0.55 mm, 0.88 mm, and 2.23 degrees, respectively.

Relative instability compensation method for hand–eye collaborative system in large-scale components measurement

In a hand–eye collaborative measurement system, the measurement error caused by the position change of an industrial robot end can create challenges during the high-precision measurement of large-scale components. The relative instability between binocular stereo vision and robot end is discussed for the first time, and a relative instability compensation (RIC) method is proposed. Firstly, using a watershed-model based on the Gaussian mixture model clustering, and locking of the region of interest are achieved. The correction of ellipse distortion combined with camera internal parameters is employed, and the 3D coordinate is reconstructed. Subsequently, the common and link points are constructed to perceive the updating of the positions. Based on the assumption of small-angle deviation, a novel model is established for compensating the relative instability. Ultimately, the experiment results indicate that the proposed method is effective for improving the measurement precision, which is superior to 0.075 mm.