Binocular Stereo Vision System Research Articles

Robust distance measurement using illumination map estimation and MAHNet in underground coal mines

An effective binocular stereo distance measurement method is proposed to address challenges posed by low brightness and weak texture of images captured in underground coal mines for the machine vision method. This approach is based on illumination map estimation and the MobileNetV3 attention hourglass stereo matching network (MAHNet) model. First, a binocular stereo vision system is established in which infrared LEDs are uniformly distributed on both sides of the belt conveyor bracket as visual feature points. Second, images are preprocessed using illumination map estimation, and the optimization of inhomogeneous brightness image enhancement is achieved by adopting adaptive Gamma correction. Third, the YOLOv5 target detection network and Gaussian fitting fusion algorithm are utilized to detect infrared LED feature points. Fourth, the MAHNet model is employed to generate the cost volume and perform disparity regression, resulting in the acquisition of accurate disparity images. Finally, triangulation is applied to determine the depth of feature points. The experimental results of distance measurement demonstrate that an average relative ranging accuracy of 1.52% within the range of 50.0 cm to 250.0 cm can be achieved by the optimized method, thereby validating the effectiveness of this binocular distance measurement method in underground coal mines.

A system and method for person identification and positioning incorporating object edge detection and scale-invariant feature transformation

During the overhead crane, due to the complex background and large image size, the direct processing algorithm usually leads to time-consuming operation and difficult to achieve accurate personnel positioning. To address these issues, we propose the feature points recognition and matching algorithm incorporating object edge detection and scale-invariant feature transform (SIFT), which is applied to the designed object detection and positioning algorithm for binocular stereo vision. The proposed method applies the Canny operator to extract contours from the locally cropped regions obtained using YOLOv5. The contour points are used as the key points of SIFT, and corresponding feature points are created descriptors, and feature points matching are performed. Experimental results indicate that the proposed method can filter feature points successfully. Moreover, the binocular stereo vision system with the binocular positioning algorithm designed has good performance, with the overall depth direction positioning error within 5% in a range of 12 m.

GelStereo Palm: A Novel Curved Visuotactile Sensor for 3-D Geometry Sensing

Recently, visuotactile sensors have shown promising potential in robotics due to their high-resolution sensing ability. Unfortunately, the majority of available visuotactile sensors are limited to flat shapes, which severely limits their application possibilities. In this paper, we propose a novel curved visuotactile sensor, the GelStereo Palm, which senses the 3D contact geometry on a curved surface using a binocular vision system. Meanwhile, to solve the light refraction problem in the binocular stereo vision system under a curved medium, a Refractive Stereo Ray Tracing model for GelStereo Palm (GP-RSRT) is presented. Moreover, a 3D tactile point cloud sensing pipeline is introduced to reconstruct the 3D contact geometry in real-time. Finally, extensive experiments are conducted to verify the accuracy and robustness of the 3D contact geometry sensing of our GelStereo Palm sensor.

Calibration method for monocular laser speckle projection system.

This paper proposes a novel calibration method for the monocular laser speckle projection system. By capturing images of a calibration board with speckles under different poses, projector's optical axis is fitted and utilized to calibrate the rotation between the camera and projector. The translation is solved in closed form subsequently and projector's virtual image is recovered via homography. After calibration, the system can be regarded and operated as a binocular stereo vision system with speckle pattern. The proposed method is efficient and convenient, without need of reference image or high-precision auxiliary equipment. Validated by experiments on Astra-s and Astra-pro, it presents significant improvement in depth-estimation compared to the traditional method.

Tomato flower pollination features recognition based on binocular gray value-deformation coupled template matching

It is necessary to recognize the tomato pollination features for the designing demand of intelligent and precise tomato supplementary pollination equipment. Mentioned pollination features include flower opening state and the three-dimensional position and pose of flower anther. Tomato flower pollination features recognition method is designed in this paper based on the deep learning full opened flower recognition model and binocular template matching three-dimensional information recognition method. First of all, a binocular stereo vision system is built to acquire the tomato flower images in greenhouse with natural lighting. Which can help vision system avoid the impact of inconsistent light intensity on image recognition. The acquired images were equalized with three groups parameters to labeled images. And the improved MC-AlexNet deep learning model is established to recognize the full opened tomato flowers in the left image of image pair acquired with binocular vision system. Then, the template is created with the full opened flower recognition result of deep learning model in left image based on gray value and deformation template matching method. And the template matching is established to recognize the corresponding full opened flowers in the right image of image pair. Finally, with the template matching result, the anther segmentation is conducted to calculate three-dimensional position and pose of anther. The experimental results show that the accuracy of full opened tomato flowers recognition model is 96.23%. The average position recognition deviation of anther is 5.94 mm. And the average anther pose recognition deviation in three plane is 6.24° for the images that anthers can be observed. And the average time consuming is about 143.18 ms per image pair. It can be concluded that the method of binocular template matching established in this paper can fulfill the demand of supplementary pollination equipment design, and the research result lays a foundation for designing and improvement of tomato supplementary pollination equipment in greenhouse.

Detection and Positioning of Camellia oleifera Fruit Based on LBP Image Texture Matching and Binocular Stereo Vision

To achieve the rapid recognition and accurate picking of Camellia oleifera fruits, a binocular vision system composed of two industrial cameras was used to collect images of Camellia oleifera fruits in natural environments. The YOLOv7 convolutional neural network model was used for iterative training, and the optimal weight model was selected to recognize the images and obtain the anchor frame region of the Camellia oleifera fruits. The local binary pattern (LBP) maps of the anchor frame region were extracted and matched by using the normalized correlation coefficient template matching algorithm to obtain the positions of the center point in the left and right images. The recognition experimental results showed that the accuracy rate, recall rate, mAP and F1 of the model were 97.3%, 97.6%, 97.7% and 97.4%. The recognition rate of the Camellia oleifera fruit with slight shading was 93.13%, and the recognition rate with severe shading was 75.21%. The recognition rate of the Camellia oleifera fruit was 90.64% under sunlight condition, and the recognition rate was 91.34% under shading condition. The orchard experiment results showed that, in the depth range of 400–600 mm, the maximum error value of the binocular stereo vision system in the depth direction was 4.279 mm, and the standard deviation was 1.142 mm. The detection and three-dimensional positioning accuracy of the binocular stereo vision system for Camellia oleifera fruits could basically meet the working requirements of the Camellia oleifera fruit-picking robot.

Analysis of the current situation and teaching strategies of university table tennis teaching in the context of big data

Abstract As an important part of university physical education, table tennis teaching occupies a crucial position in physical education reform and requires continuous innovation and reform. This paper first starts from the current situation of table tennis teaching and proposes the application of a binocular stereo vision system based on a binocular stereo vision system for university table tennis teaching, which lays the foundation for three-dimensional reconstruction and skill learning. Secondly, the mathematical model of the 3D reconstruction algorithm is constructed by the tangential distortion of least squares, which avoids the complex algorithm of iteratively solving the 3D world coordinates of table tennis and improves the speed of the 3D reconstruction algorithm while the accuracy meets the subsequent prediction requirements. Finally, students in the experimental and control groups were compared in the technical evaluation of table tennis backhand pushing, attacking techniques, and left pushing and right attacking to verify the effectiveness of the table tennis binocular stereo vision teaching system. The study results showed that the experimental group students trained by the binocular stereo vision system for table tennis had 41.77% higher accuracy, 23.13% higher coordination, 12.31% higher return speed, and 14.84% higher hitting power compared to the control group. Applying a binocular stereo-vision system to table tennis teaching is novel and is conducive to stimulating students’ curiosity and challenge, enhancing their interest in learning, and effectively improving their understanding and knowledge of table tennis techniques.

Modeling of the Three-Dimensional Point Cloud Based on the Binocular Stereovision and Structure Identification

Abstract The existence of weak planes affects the stability of rock engineering, especially the failure modes of the surrounding rocks. Therefore, investigation of the structural planes of rock mass and the extraction of discontinuities is important for rock mechanics. In this work, the LenaCV binocular stereovision system was studied and applied to establish three-dimensional (3D) surface models. Calibration of the binocular camera parameters, optimal distance discussion, and 3D modeling optimization were conducted. The optimal measuring distance and the intrinsic parameters suitable for the LenaCV stereovision were discussed. Finally, the 3D reconstruction of two case studies was carried out and the 3D point cloud modeling was established. The structural discontinuity mapping in rock faces was also carried out. The feasibility of the binocular stereovision system, as well as the optimum baseline distance, was discussed and verified. The study results can provide a new platform for the economic investigation of 3D rock mass structures and provide a new approach for the efficient extraction of rock mass discontinuities.

A Novel Ship-Height Measurement System Based on Optical Axes Collinear Stereo Vision

Overheight ships pose serious danger to bridges when sailing along inland rivers. Measuring ship height from a long distance with high precision is a very challenging problem. This article proposes a novel optical axes colinear binocular stereo vision system to meet this goal. The system is equipped with two arrays of cameras, each consisting of a long-focus camera (LFC) and a short-focus camera (SFC), located face-to-face on each side of a river. The SFC, i.e., wide-angle-lens camera, is used for ship detection, while the LFC, i.e., tele lens camera, is used for precise ship-height measurement. Different from conventional binoculars, the optical axis of the two LFCs are arranged to be colinear in our setup. The advantage of this configuration is in twofold. First, it is simple and precise to calibrate the measuring apparatus and the system error since the two LFCs are mutually visible to each other. Second, due to its face-to-face configuration across the river, the entire navigation waterway is under constant monitoring without blind spot; hence, the shortcoming of limited field of view (FOV) of tele lens is overcome with conventional parallel optical axis stereo vision setup. In this work, the detailed error analysis was performed and in-field experiments were well designed. The experimental data matched very well with the theoretical prediction. The proposed system can achieve a ship-height measuring error below 0.1 m for a distance of 1.46 km.

Extraction of 3D distribution of potato plant CWSI based on thermal infrared image and binocular stereovision system.

As the largest component of crops, water has an important impact on the growth and development of crops. Timely, rapid, continuous, and non-destructive detection of crop water stress status is crucial for crop water-saving irrigation, production, and breeding. Indices based on leaf or canopy temperature acquired by thermal imaging are widely used for crop water stress diagnosis. However, most studies fail to achieve high-throughput, continuous water stress detection and mostly focus on two-dimension measurements. This study developed a low-cost three-dimension (3D) motion robotic system, which is equipped with a designed 3D imaging system to automatically collect potato plant data, including thermal and binocular RGB data. A method is developed to obtain 3D plant fusion point cloud with depth, temperature, and RGB color information using the acquired thermal and binocular RGB data. Firstly, the developed system is used to automatically collect the data of the potato plants in the scene. Secondly, the collected data was processed, and the green canopy was extracted from the color image, which is convenient for the speeded-up robust features algorithm to detect more effective matching features. Photogrammetry combined with structural similarity index was applied to calculate the optimal homography transform matrix between thermal and color images and used for image registration. Thirdly, based on the registration of the two images, 3D reconstruction was carried out using binocular stereo vision technology to generate the original 3D point cloud with temperature information. The original 3D point cloud data were further processed through canopy extraction, denoising, and k-means based temperature clustering steps to optimize the data. Finally, the crop water stress index (CWSI) of each point and average CWSI in the canopy were calculated, and its daily variation and influencing factors were analyzed in combination with environmental parameters. The developed system and the proposed method can effectively detect the water stress status of potato plants in 3D, which can provide support for analyzing the differences in the three-dimensional distribution and spatial and temporal variation patterns of CWSI in potato.

Impact factors on rockfall movement using high-speed camera binocular stereo vision technology

Rockfall is a frequent and serious geological hazard in mountainous areas worldwide. It is of great significance to fully understand rockfall movement for disaster assessment and prevention. In this paper, based on the principle of binocular stereo vision, a filed experimental monitoring technology was developed to conduct the rockfall movement experiment research. Through the orthogonal experimental method, taking the block velocities obtained by binocular stereo vision system as a reference index, the impact factors such as the slope angles and the falling heights, shapes and masses of blocks on rockfall movement are studied. The lateral deviations and jumping heights of different blocks when they are released from the same falling height and moving along the same slope are presented. The results show that slope angle is the most key factor affecting block movement, while the falling heights, shapes and masses of blocks are the secondary factors. Moreover, the experimental methods and results of this paper can lay a foundation for the further study of rockfall movement characteristics.

Automatic mileage positioning for road inspection using binocular stereo vision system and global navigation satellite system

Road inspections ensure an adequate infrastructure operation and traffic safety, but the widely used positioning modes of road inspection have large deviations with road mileage. This paper describes an automatic method of road mileage positioning for road inspection. It integrates binocular stereo vision system, Global Navigation Satellite System (GNSS), objection detection algorithm, and Optical Character Recognition into road mileage positioning. A validation test result shows the average deviation of the mileage positioning is 4.06 m, which meets the requirement of road inspection. The significance of this paper consists of improving the automatic method of mileage positioning with the rapid disposal of abnormal targets on roads. In future research, the targets of road inspection, such as pavement potholes, patch, wearing lane markings, will be detected by objection detection algorithms and be positioned by the mileage positioning method.

A non-iterative calibration method for the extrinsic parameters of binocular stereo vision considering the line constraints

The calibration accuracy of the binocular stereo vision system is important for precision measurement. The extrinsic parameters (structure parameters), which describe the relative position of the two cameras, need to be optimized before measurement. The optimization of extrinsic parameters is accomplished through iterations of minimizing an objective function. It is hard to decide the optimum objective function between various objective functions. The calibration accuracy is decreased when the objective function is incorrect, which can result in excessive optimization or fall into a local minimum (ambiguous solution), notably for planar targets. In this paper, we proposed a non-iterative calibration method for the extrinsic parameters of binocular stereo vision considering the line constraints of a planar target. Each line formed by every two calibration points can provide two independent constraints. Additional equations for solving the extrinsic parameters can be obtained using these constraints. We added these equations to the equations system established by the homography of the target plane and image plane. The extrinsic parameters of the binocular stereo vision system can be solved non-iteratively. Using the parameters calibrated by our method, greater improvements in the accuracy and stability of measurements occurred, compared to other existing methods. In the −45° to +45° measuring range, the maximum measurement error is no more than 0.037° and the average measurement error is no more than 0.022°. In the 0 to 28 mm measuring range, the maximum measurement error is no more than 0.039 mm and the average measurement error is no more than 0.012 mm. The maximum length measurement error is no more than 0.038 mm and the average length measurement error is no more than 0.011 mm while measuring a 240 × 180 mm2 target.

Body size measurement based on deep learning for image segmentation by binocular stereovision system

Body size measurement based on deep learning for image segmentation by binocular stereovision system

4D dual-mode staring hyperspectral-depth imager for simultaneous spectral sensing and surface shape measurement.

A 4D dual-mode staring hyperspectral-depth imager (DSHI), which acquire reflectance spectra, fluorescence spectra, and 3D structural information by combining a staring hyperspectral scanner and a binocular line laser stereo vision system, is introduced. A 405 nm laser line generated by a focal laser line generation module is used for both fluorescence excitation and binocular stereo matching of the irradiated line region. Under the configuration, the two kinds of hyperspectral data collected by the hyperspectral scanner can be merged into the corresponding points in the 3D model, forming a dual-mode 4D model. The DSHI shows excellent performance with spectral resolution of 3 nm, depth accuracy of 26.2 µm. Sample experiments on a fluorescent figurine, real and plastic sunflowers and a clam are presented to demonstrate system's with potential within a broad range of applications such as, e.g., digital documentation, plant phenotyping, and biological analysis.

Spatiotemporal Matching Cost Function Based on Differential Evolutionary Algorithm for Random Speckle 3D Reconstruction

Random speckle structured light can increase the texture information of the object surface, so it is added in the binocular stereo vision system to solve the matching ambiguity problem caused by the surface with repetitive pattern or no texture. To improve the reconstruction quality, many current researches utilize multiple speckle patterns for projection and use stereo matching methods based on spatiotemporal correlation. This paper presents a novel random speckle 3D reconstruction scheme, in which multiple speckle patterns are used and a weighted-fusion-based spatiotemporal matching cost function (STMCF) is proposed to find the corresponding points in speckle stereo image pairs. Furthermore, a parameter optimization method based on differential evolutionary (DE) algorithm is designed for automatically determining the values of all parameters included in STMCF. In this method, since there is no suitable training data with ground truth, we explore a training strategy where a passive stereo vision dataset with ground truth is used as training data and then apply the learned parameter value to the stereo matching of speckle stereo image pairs. Various experimental results verify that our scheme can realize accurate and high-quality 3D reconstruction efficiently and the proposed STMCF exhibits superior performance in terms of accuracy, computation time and reconstruction quality than the state-of-the-art method based on spatiotemporal correlation.

Calibration method for structure parameters of a binocular stereo vision system equipped with polarizers.

For structure parameter calibration of a binocular stereo vision system equipped with polarizers, the optimal calibration polarization angle needs to be determined. There are no corresponding solutions for the determination of the optimal polarization angle of structure parameter calibration. Furthermore, existing research considers the polarization angle that causes the image to possess the lowest brightness (gray value) as the optimal polarization angle. This reduces image contrast, and eventually the texture information of the image is lost, which also affects the accuracy of feature extraction. In this paper, we propose a new calibration method for the structure parameters of a binocular stereo vision system equipped with polarizers. We calculated the pose of the target relative to each camera for different polarization angles. The sum of the object-space errors corresponding to each polarization angle was considered as the evaluation criterion to determine the optimal calibration polarization angle. The calibration of structure parameters was finished using images captured on the premise of the optimal calibration polarization angle. This angle can also be considered as the reference of the polarization angle for measurement. Experiment results show that using the calibration results of our method, the reconstructed length error of a 275×200mm target was less than ±0.052mm, the reconstructed linear displacement error was less than ±0.048mm for the range of 0-30 mm, and the reconstructed rotary angle error was less than ±0.048∘ for the range of-30∘-30∘.

Optimization Design of Camera Pose for Network Base on Multi-view Stereo Vision

When designing a multi-vision stereo vision network, the camera’s positional information has a critical impact on the measurement accuracy. In order to solve the problem of optimizing the camera pose during network design, in this paper, we split the multi-vision stereo vision system into binocular stereo vision system. Based on the mathematical model of binocular stereo vision system, the error function of the positional parameters is constructed. By simulation analyzing the function models the field of view angle, the angle between the optical axis and the baseline, the baseline length, the law of the influence of each parameter on the measurement accuracy and the optimal range of values are obtained. Then we compare the actual measurement results with the simulation analysis results. The results show that as the field of view angle, optical axis and baseline angle, baseline length increases, the measurement error first decreases and then increases, and the increase is gradually accelerated. When each parameter is located in the optimal range, the measurement error meets the precision engineering measurement accuracy requirement of 0.05mm, and the minimum error is reduced by 0.114mm, 0.120mm and 0.061mm compared to the maximum error. In the network design, the optimization of the pose parameters can effectively obtain better pose information of the camera and thus improve the accuracy of the measurement.

Rutting Trajectory Measurement of Curved Ramps Using 3d Line Scanning Laser System and Binocular Stereo Vision System

Rutting Trajectory Measurement of Curved Ramps Using 3d Line Scanning Laser System and Binocular Stereo Vision System

Pose Estimation under Visual Sensing Technology and Its Application in Art Design

The study is aimed at solving the problem of large measurement errors caused by the binocular camera in traditional 3D art design, which leads to inaccurate 3D information of the target. The contour information extraction in the process of human motion pose reconstruction is easily affected by the noise in the image. Therefore, a binocular stereo vision system is built first and it integrates image acquisition, camera calibration, and image processing. The dedistortion method is used to process the image because it can reduce errors. Second, a three-dimensional human motion pose reconstruction model is implemented, the Gaussian template is used to remove the noise in the image frame, and the change detection template (CDM) is used to solve the problem of background “exposure” and “occlusion.” Finally, simulation experiments are designed to verify the system and model designed. Since the research on the application of pose estimation based on visual sensing technology in art design is still blank, such research has great significance and provides a reference for the research in the field. The literature analysis is used to expound and analyze the application of pose estimation based on visual sensing technology in visual communication design and environmental art design: (1) although the binocular stereo vision system causes some errors in the measurement, the overall error is controlled within 2% and the accuracy is high, which proves that it can be applied to the acquisition of three-dimensional information of the target in art design; (2) there is a high degree of fitting between the video sequence data created by the three-dimensional human motion pose reconstruction model designed and the real motion data, which indicates that this method has high accuracy in processing image sequences and the feasibility of applying it to human pose reconstruction in three-dimensional art design is high; (3) through the analysis of the existing literature, it is found that most of the current visual-based attitude assessment studies are carried out by using network cameras combined with computers, and the quality of the obtained images is low. The combination of binocular stereo sensor and attitude estimation technology can be applied to the design of advertising, animation, games, and packaging, making the behavior of virtual characters in animation and games more vivid. The combination provides convenience for the collection of environmental spatial information and object attitude information, the formulation of a design scheme, and real-time monitoring of construction in environmental art design. The purpose of this study is to provide an important theoretical basis for the technical upgrading of art design.