Binocular Stereo Vision Technology Research Articles

Enhanced precision in greenhouse tomato recognition and localization

The core challenge in realizing automatic tomato harvesting in greenhouse environments lies in the precise identification and localization of the fruits. This paper introduces a comprehensive approach based on an improved YOLOv5 detection algorithm and optimized binocular stereo vision technology. Firstly, by introducing the C3-Transformer Encoder (CTM) structure and Bidirectional Feature Pyramid Network (Bi-FPN), this study enhanced the model’s ability to recognize tomatoes, especially under complex backgrounds and occlusion conditions. After field testing, the mAP50 accuracy reached 97.1%, an increase of 1.2 percentage points, enhancing detection precision. In addition, the ZED binocular camera was used, and the census stereo matching algorithm was optimized, significantly reducing disparity errors, thereby improving the accuracy of depth information. This allows the model to accurately calculate the three-dimensional spatial position of tomatoes obscured by branches and leaves, greatly improving the efficiency of the harvesting robot. Through field debugging verification with the harvesting robot, the method proposed in this study has shown high accuracy and reliability in the recognition and localization of tomatoes in complex greenhouse environments.

Non-Contact Tilapia Mass Estimation Method Based on Underwater Binocular Vision

The non-destructive measurement of fish is an important link in intelligent aquaculture, and realizing the accurate estimation of fish mass is the key to the stable operation of this link. Taking tilapia as the object, this study proposes an underwater tilapia mass estimation method, which can accurately estimate the mass of free-swimming tilapia under non-contact conditions. First, image enhancement is performed on the original image, and the depth image is obtained by correcting and stereo matching the enhanced image using binocular stereo vision technology. And the fish body is segmented by an SAM model. Then, the segmented fish body is labeled with key points, thus realizing the 3D reconstruction of tilapia. Five mass estimation models are established based on the relationship between the body length and the mass of tilapia, so as to realize the mass estimation of tilapia. The results showed that the average relative errors of the method models were 5.34%~7.25%. The coefficient of determination of the final tilapia mass estimation with manual measurement was 0.99, and the average relative error was 5.90%. The improvement over existing deep learning methods is about 1.54%. This study will provide key technical support for the non-destructive measurement of tilapia, which is of great significance to the information management of aquaculture, the assessment of fish growth condition, and baiting control.

Headland Identification and Ranging Method for Autonomous Agricultural Machines

Headland boundary identification and ranging are the key supporting technologies for the automatic driving of intelligent agricultural machinery, and they are also the basis for controlling operational behaviors such as autonomous turning and machine lifting. The complex, unstructured environments of farmland headlands render traditional image feature extraction methods less accurate and adaptable. This study utilizes deep learning and binocular vision technologies to develop a headland boundary identification and ranging system built upon the existing automatic guided tractor test platform. A headland image annotation dataset was constructed, and the MobileNetV3 network, notable for its compact model structure, was employed to achieve binary classification recognition of farmland and headland images. An improved MV3-DeeplabV3+ image segmentation network model, leveraging an attention mechanism, was constructed, achieving a high mean intersection over union (MIoU) value of 92.08% and enabling fast and accurate detection of headland boundaries. Following the detection of headland boundaries, binocular stereo vision technology was employed to measure the boundary distances. Field experiment results indicate that the system’s average relative errors of distance in ranging at distances of 25 m, 20 m, and 15 m are 6.72%, 4.80%, and 4.35%, respectively. This system is capable of meeting the real-time detection requirements for headland boundaries.

Fully automatic system for fish biomass estimation based on deep neural network

The approach for estimating biomass in non-contact, free-swimming fish has encountered difficulties such as fish body occlusion, bending, non-orthogonal angles, and low efficiency. To address these issues, this study had combined fish posture recognition (using deep learning technology) with biomass estimation (utilizing stereo vision technology) for the first time, and developed a fast, precise, and fully automatic fish biomass estimation system. The improved single-stage target detection algorithm significantly improved the direct detection and extraction of high-quality images of moving fish in real-time, eliminating the need for manual processing of images that may have imperfect posture. Fish body length and height were measured in the real world using binocular stereo vision technology. Finally, the fish body weight can be estimated by considering the relationship among their body length, height, and weight. The experiment confirmed that the system had successfully avoided influencing factors that could affect fully automatic estimation. The results demonstrated a strong linear relationship between the estimated and measured fish body weights, with a mean relative error (MRE) of 2.87%. There were no significant differences between the estimated and measured weights (p = 0.94). The MRE of the multi-factor model was much lower than that of the single-factor model (length-weight of 8.86% and height-weight of 7.41%). The results indicate that the system developed is a highly effective approach to fully automated biomass estimation. This can be used to guide actual production and further study of the mechanism of fish growth.

DIC measurement method based on binocular stereo vision for image 3D displacement detection

The deformation detection of large machinery is usually achieved using three-dimensional displacement measurement. Binocular stereo vision measurement technology, as a commonly used digital image correlation method, has received widespread attention in the academic community. Binocular stereo vision achieves the goal of three-dimensional displacement measurement by simulating the working mode of the human eyes, but the measurement is easily affected by light refraction. Based on this, the study introduces particle swarm optimization algorithm for target displacement measurement on Canon imaging dataset, and introduces backpropagation neural network for mutation processing of particles in particle swarm algorithm to generate fusion algorithm. It combines the four coordinate systems of world, pixel, physics, and camera to establish connections. Taking into account environmental factors and lens errors, the camera parameters and deformation coefficients were revised by shooting a black and white checkerboard. Finally, the study first conducted error analysis on binocular stereo vision technology in three dimensions, and the relative error remained stable at 1 % within about 60 seconds. At the same time, three algorithms, including the spotted hyena algorithm, were introduced to conduct performance comparison experiments using particle swarm optimization and backpropagation network algorithms. The experiment shows that the three-dimensional error of the fusion algorithm gradually stabilizes within the range of [–0.5 %, 0.5 %] over time, while the two-dimensional error generally hovers around 0 value. Its performance is significantly superior to other algorithms, so the binocular stereo vision of this fusion algorithm can achieve good measurement results.

Digital twin technology for wind turbine towers based on joint load–response estimation: A laboratory experimental study

An accurate estimation of dynamic loads and structural dynamic responses is deemed an indispensable prerequisite for developing trustworthy digital twin (DT) models of dynamically excited structures. Traditional joint load–response estimation (JLRE) algorithms necessitate a full-rank feedthrough matrix, implying that accelerometers are required at all degrees of freedom with input loads. If an unknown input is a bending moment or torque, the rotational acceleration must be measured, which is usually difficult, if not impractical, in real applications. Therefore, the applications of the traditional JLRE in wind turbines (WTs) are rather limited due to the complex excitation mechanism. However, the unified linear input and state estimator (ULISE) algorithm presented in this paper eliminates this constraint. This paper experimentally tested this algorithm on an operating 1:50 scaled WT model. The influences from blade rotation were considered. A comprehensive structural health monitoring (SHM) sensing system was deployed on the WT tower to measure the tower's dynamic responses. A camera system, which integrated digital image correlation (DIC) technology and binocular stereo vision technology, was also adopted. The unknown excitations and unmonitored dynamic responses of the WT tower were estimated simultaneously using the ULISE algorithm. The reconstructed strain and acceleration responses achieved high accuracy, with a normalized root mean square error less than 9%. The blade rotation had a notable impact on the WT tower, primarily manifested as a bending moment whose dominant frequency was corresponding to the blade's rotational speed. Such joint estimations could function as the virtual sensing of unknown inputs and responses in the DT model of the WT tower, which will enable DT-based online remote structural condition assessment and preventive maintenance in the future.

Extraction of 3D distribution of potato plant CWSI based on thermal infrared image and binocular stereovision system.

As the largest component of crops, water has an important impact on the growth and development of crops. Timely, rapid, continuous, and non-destructive detection of crop water stress status is crucial for crop water-saving irrigation, production, and breeding. Indices based on leaf or canopy temperature acquired by thermal imaging are widely used for crop water stress diagnosis. However, most studies fail to achieve high-throughput, continuous water stress detection and mostly focus on two-dimension measurements. This study developed a low-cost three-dimension (3D) motion robotic system, which is equipped with a designed 3D imaging system to automatically collect potato plant data, including thermal and binocular RGB data. A method is developed to obtain 3D plant fusion point cloud with depth, temperature, and RGB color information using the acquired thermal and binocular RGB data. Firstly, the developed system is used to automatically collect the data of the potato plants in the scene. Secondly, the collected data was processed, and the green canopy was extracted from the color image, which is convenient for the speeded-up robust features algorithm to detect more effective matching features. Photogrammetry combined with structural similarity index was applied to calculate the optimal homography transform matrix between thermal and color images and used for image registration. Thirdly, based on the registration of the two images, 3D reconstruction was carried out using binocular stereo vision technology to generate the original 3D point cloud with temperature information. The original 3D point cloud data were further processed through canopy extraction, denoising, and k-means based temperature clustering steps to optimize the data. Finally, the crop water stress index (CWSI) of each point and average CWSI in the canopy were calculated, and its daily variation and influencing factors were analyzed in combination with environmental parameters. The developed system and the proposed method can effectively detect the water stress status of potato plants in 3D, which can provide support for analyzing the differences in the three-dimensional distribution and spatial and temporal variation patterns of CWSI in potato.

Screen-monitored stitching deflectometry based on binocular stereo vision

The measurement accuracy of the current phase measuring deflectometry (PMD) methods for in-situ measurement of the surface shape of large-aperture optics, which are placed at different inclination angles, is affected by the varied pose and deformed shape of screen due to manufacturing, external force and temperature variation, etc. This paper proposes a novel PMD method, the screen-monitored stitching deflectometry (SMSD), by combining binocular stereo vision with stitching deflectometry. And a four-camera PMD measurement system is constructed accordingly. The coordinates of the point source are determined using binocular stereo vision technology; based on the acquired coordinates, the stereo-iterative reconstruction and stitching algorithm are used to achieve the surface shape measurement. Simulations and experiments are accomplished to validate the advantages of the proposed method in reducing measurement errors caused by the varied pose and deformed shape of screen.

Impact factors on rockfall movement using high-speed camera binocular stereo vision technology

Rockfall is a frequent and serious geological hazard in mountainous areas worldwide. It is of great significance to fully understand rockfall movement for disaster assessment and prevention. In this paper, based on the principle of binocular stereo vision, a filed experimental monitoring technology was developed to conduct the rockfall movement experiment research. Through the orthogonal experimental method, taking the block velocities obtained by binocular stereo vision system as a reference index, the impact factors such as the slope angles and the falling heights, shapes and masses of blocks on rockfall movement are studied. The lateral deviations and jumping heights of different blocks when they are released from the same falling height and moving along the same slope are presented. The results show that slope angle is the most key factor affecting block movement, while the falling heights, shapes and masses of blocks are the secondary factors. Moreover, the experimental methods and results of this paper can lay a foundation for the further study of rockfall movement characteristics.

Petroleum Pipeline Interface Recognition and Pose Detection Based on Binocular Stereo Vision

Liquified natural gas (LNG) manipulator arms have been widely used in natural gas transportation. However, the automatic docking technology of LNG manipulator arms has not yet been realized. The first step of automatic docking is to identify and locate the target and estimate its pose. This work proposes a petroleum pipeline interface recognition and pose judgment method based on binocular stereo vision technology for the automatic docking of LNG manipulator arms. The proposed method has three main steps, including target detection, 3D information acquisition, and plane fitting. First, the target petroleum pipeline interface is segmented by using a color mask. Then, color space and Hu moment are used to obtain the pixel coordinates of the contour and center of the target petroleum pipeline interface. The semi-global block matching (SGBM) algorithm is used for stereo matching to obtain the depth information of an image. Finally, a plane fitting and center point estimation method based on a random sample consensus (RANSAC) algorithm is proposed. This work performs a measurement accuracy verification experiment to verify the accuracy of the proposed method. The experimental results show that the distance measurement error is not more than 1% and the angle measurement error is less than one degree. The measurement accuracy of the method meets the requirements of subsequent automatic docking, which proves the feasibility of the proposed method and provides data support for the subsequent automatic docking of manipulator arms.

Image Recognition of Sports Athletes' High-Intensity Sports Injuries Based on Binocular Stereo Vision.

Sports athletes are prone to certain injuries during high-intensity exercise training. In the process of treating an injury, images of the injury site need to be collected and identified. However, the traditional recognition method cannot effectively extract the features of the image. At the same time, it ignores the optimization of the damage image recognition results, resulting in low recognition accuracy and poor efficiency. Binocular stereo vision technology can quickly and accurately detect moving objects. Therefore, in order to more accurately identify high-intensity sports injury images, this study takes the high-intensity sports injury images as the basic research object. Several processes of image processing based on binocular stereo vision are analyzed, and the vulnerable parts of the body in high-intensity sports are also studied. Finally, the method in this study is verified. The experimental results show that the method proposed in this study reduces the average error rate by 0.19% compared with the traditional recognition method. It can effectively identify and detect injury images, thereby improving the accuracy and stability of sports injury image identification. The identification time is also shortened accordingly, which has certain practicability and feasibility. In addition, the binocular camera used in this study has high accuracy, and the obtained images of sports injuries are of good quality, which lays a foundation for image detection and recognition.

Underwater fish mass estimation using pattern matching based on binocular system

Fish mass is the main information for judging growth status, regulating water quality environment, and precision feeding and grading in the process of intelligent aquaculture management activities. However, the occlusion, bending, and poor imaging angle of fish body image are still serious challenges for underwater fully automatic mass measurement. The aim of this study was to develop underwater non-contact method to automatically estimate the free-swimming fish mass based on binocular stereo vision technology. The fish body images were automatically selected and obtained by using pattern recognition method based on LabVIEW development platform during the experimental period. All the fish samples were divided into three groups according to mass (200–500 g, 500–800 g, and 800–1200 g), and then subdivided into three groups by imaging angle (orthogonal angles, greater than 45°, and less than 45°). The experiment indicated that the fish mass could be estimated using fish body area with a high coefficient of determination (R2) based on linear model. The mean relative errors between estimated and measured value were 3.37% (orthogonal angles), 4.95% (greater than 45° angles), and 16.59% (less than 45° angles). Significant difference was found in less than 45° group with p < 0.01. These findings showed that the approach put forward in this research could realize fully automatic mass estimation for underwater free-swimming fish and effectively improve the estimation robustness and efficiency.

Motion Track Enhancement Method of Sports Video Image Based on OTSU Algorithm

Current sports video image tracking methods cannot get the average gray information of pixels, which leads to the low segmentation accuracy of target trajectory. Therefore, this paper proposes a method of enhancing the motion track of sports video object based on OTSU algorithm. The OTSU algorithm is introduced to segment the unsupervised image automatically based on threshold calculation. Combined with 3D histogram, the OTSU algorithm is optimized to overcome the influence of noise and nonuniform illumination and to improve the segmentation performance. Using the optimized OTSU algorithm, the preprocessing of sports video image is realized by gray level transform, median filter, and binarization. Based on this, the object motion model of sports video image is constructed, and the edge information of sports video image is extracted. Using binocular stereo vision technology, according to the image target motion model and the current space coordinates, calculate the motion track, and obtain the parameters of sports video image. Experimental results show that the proposed method can segment the target accurately in a shorter time, and the enhancement effect of target trajectory is obvious, and the error is small.

Research on the influence of light with different wavelength on the motion behavior of carp robots

In order to study the effect of light with different wavelengths on the motion behavior of carp robots, phototaxis experiment, anatomical experiment, light control experiment and speed measurement experiment were carried out in this study. Blue, green, yellow and red light with different wavelength were used to conduct phototaxis experiments on carp to observe their movement behavior. By dissecting the skull bones of the carp to determine the appropriate location to carry the light control device, we independently developed a light control carrying device which was suitable for any illumination intensity environment. The experiment of the light-controlled carp robots was carried out. The motion behavior of the carp robot was checked by using computer binocular stereo vision technology. The motion trajectory of the carp robot was tracked and obtained by applying kernel correlation filter (KCF) algorithm. The motion velocity of the carp robot at different wavelengths was calculated according to their motion trajectory. The results showed that carps' sensitivity to different light changed from strong to weak in the order of blue, red, yellow and green, so that using light with different wavelengths to control the speed of the carp robot has certain laws to follow. A new method to avoid brain damage in carp robots control can be provided in this study.

Research on UAV Visual Recognition and Positioning Method BasedonGIS

Regarding the positioning and tracking of the target object by the UAV, the GPS signal in the actual flight is unstable and inaccurate. From this, the GIS visual identification and positioning method is found; the binocular stereo vision technology is proposed This optical physics theory, as well as binocular cameras and specific calculation methods for positioning; found a way to accurately locate and track UAVs through GIS; accurate enough to calculate the UAV’s positioning operation process and calculations through the RSS positioning and tracking method Method: The wording simulation experiment proves that this visual recognition positioning method has the advantages of higher accuracy and strong adaptability.

Design of 3D Scene Visual Communication Modeling Based on Virtual Reality Graphics Rendering Framework

Virtual reality is a kind of computer simulation system which can create virtual environment. People can interact with the three-dimensional environment generated by computer. In this paper, the real-time computer graphics technology, three-dimensional modeling technology and binocular stereo vision technology in virtual reality technology are studied, and the binocular stereo vision optical system is designed. This paper studies the visual communication modeling design of 3D scene under the framework of virtual reality graphics rendering. In this paper, we design and make a three-dimensional model, and develop a virtual indoor scene application. In this paper, the 3D model is used to build the scene, and several commonly used rendering technologies such as texture mapping, material, lighting calculation, transparent effect and shadow calculation in unity engine are used to render the scene. The experimental results show that this method can improve the design efficiency and the accuracy of design matching.

A design of intelligent delivery drone based on BeiDou navigation and visual processing

This article used automatic control, 5G real-time communication, visual processing and other technologies to design a carbon fiber intelligent delivery drone, based on BeiDou satellite navigation and face recognition. And we proposed a semi-autonomous delivery mode to achieve real-time control of the drone. This mode can reduce flight risks and ensure the safety and accuracy of distribution. The drone can be powered by microcrystalline silicon thin film solar cells. The BeiDou satellite navigation system is used for autonomous positioning, combined with binocular stereo vision technology to observe the surroundings, automatically plan the flight path. On the way, personnel can intervene in real time as needed. Finally, when the drone arrives at the destination, facial recognition technology is used to confirm the identity of the recipient to complete the delivery.

Binocular stereo vision based illuminance measurement used for intelligent lighting with LED

Intelligent lighting is responsible for providing an appropriate illuminance and balancing the environment and resources. An appropriate illuminance can improve visual effect of machine recognition, while it is always difficult to be measured and controlled. An illuminance measurement method by using depth measurement technology of binocular stereo vision is proposed in this study. Illuminance analysis models of single LED and combined LEDs are established, and theoretical calculation equations for analyzing the relationship between power, illuminance, and depth are proposed. The illuminance distribution of single LED and combined LEDs are measured. The difference between single LED and combined LEDs are obtained in terms of illuminance, power, energy efficiency, effective illuminance area, etc. The results show that the illuminance is proportional to power of LED consumed and inversely proportional to the square of depth of image. The higher the energy conversion efficiency under the high illuminance for both single LED and combined LEDs. The depth of image is measured by binocular stereo vision technology, which has a good measurement performance. The binocular stereo vision based illuminance measurement method proposed in this study is efficient, reliable and robust, which is in good agreement with the experimental results, and the relative error is less than 3.8 %. The illuminance measurement is realized without increasing the equipment and image acquisition workload, which lays the foundation for dynamic monitoring and controlling of illuminance. The research results can also be applied to related fields as well, such as the intelligent lighting of street lamp, high beam in automatic driving and supplement lamps in video monitoring.

Research on Application of Machine Vision in Robots for Live-Power Lines

The electric fire-exchanging operation is basically used to implement operation such as Wire stripping, drainage wire through the fastening wire clip, clamp bolt fastening, etc. Manual operation is time-consuming and laborious and has very high safety risks, so the automatic electrified ignition robot emerges as The Times require; The automatic operation method should accurately identify the position of wire and drainage line, so as to guide the robot to move and carry out the corresponding operation. Based on this, an intelligent binocular vision guidance method is proposed in this paper. The image data in the working space is collected by the binocular camera in real time, the wire is identified by the image segmentation technology, the wire is separated from the background, and the spatial pose of the wire is solved by the PNP and binocular stereo vision technology. Through YOLOV5 target detection, the position identification and positioning of the thread and clamp of the drainage line are realized, and the automatic fire ignition operation is finally guided by the manipulator. Experiments show that this method has the advantages of high accuracy, good stability and fast speed.

Retracted] Multivisual Animation Character 3D Model Design Method Based on VR Technology

For VR systems, one of its core parts is to present people with a real and immersive 3D simulation environment. This paper uses real‐time computer graphics technology, three‐dimensional modeling technology, and binocular stereo vision technology to study the multivisual animation character objects in virtual reality technology; designs a binocular stereo vision animation system; designs and produces a three‐dimensional model; and develops a virtual multivisual animation scene application. The main research content and work performed in the text include the research of the basic graphics rendering pipeline process and the analysis and research of each stage of the rendering pipeline. It mainly analyzes the 3D graphics algorithm used in the three‐dimensional geometric transformation of computer graphics and studies the basic texture technology, basic lighting model, and other image output processes used in the fragment processing stage. Combined with the development needs of the subject, the principles of 3D animation rendering production software and 3D graphics modeling are studied, and the solid 3D model displayed in the virtual reality scene is designed and produced. This article also reflects the application of virtual reality in multivisual animation character design from the side, so it has realistic value and application prospects.