Binocular Vision Measurement Research Articles

System Structural Error Analysis in Binocular Vision Measurement Systems

A binocular stereo vision measurement system is widely used in fields such as industrial inspection and marine engineering due to its high accuracy, low cost, and ease of deployment. An unreasonable structural design can lead to difficulties in image matching and inaccuracies in depth computation during subsequent processing, thereby limiting the system’s performance and applicability. This paper establishes a systemic error analysis model to enable the validation of changes in structural parameters on the performance of the binocular vision measurement. Specifically, the impact of structural parameters such as baseline distance and object distance on measurement error is analyzed. Extensive experiments reveal that when the ratio of baseline length to object distance is between 1 and 1.5, and the angle between the baseline and the optical axis is between 30 and 40 degrees, the system measurement error is minimized. The experimental conclusions provide guidance for subsequent measurement system research and parameter design.

High-Precision Drilling by Anchor-Drilling Robot Based on Hybrid Visual Servo Control in Coal Mine

Rock bolting is a commonly used method for stabilizing the surrounding rock in coal-mine roadways. It involves installing rock bolts after drilling, which penetrate unstable rock layers, binding loose rocks together, enhancing the stability of the surrounding rock, and controlling its deformation. Although recent progress in drilling and anchoring equipment has significantly enhanced the efficiency of roof support in coal mines and improved safety measures, how to deal with drilling rigs’ misalignment with the through-hole center remains a big issue, which may potentially compromise the quality of drilling and consequently affect the effectiveness of bolt support or even result in failure. To address this challenge, this article presents a robotic teleoperation system alongside a hybrid visual servo control strategy. Addressing the demand for high precision and efficiency in aligning the drilling rigs with the center of the drilling hole, a hybrid control strategy is introduced combining position-based and image-based visual servo control. The former facilitates an effective approach to the target area, while the latter ensures high-precision alignment with the center of the drilling hole. The robot teleoperation system employs the binocular vision measurement system to accurately determine the position and orientation of the drilling-hole center, which serves as the designated target position for the drilling rig. Leveraging the displacement and angle sensor information installed on each joint of the manipulator, the system utilizes the kinematic model of the manipulator to compute the spatial position of the end-effector. It dynamically adjusts the spatial pose of the end-effector in real time, aligning it with the target position relative to its current location. Additionally, it utilizes monocular vision information to fine-tune the movement speed and direction of the end-effector, ensuring rapid and precise alignment with the target drilling-hole center. Experimental results demonstrate that this method can control the maximum alignment error within 7 mm, significantly enhancing the alignment accuracy compared to manual control. Compared with the manual control method, the average error of this method is reduced by 41.2%, and the average duration is reduced by 4.3 s. This study paves a new path for high-precision drilling and anchoring of tunnel roofs, thereby improving the quality and efficiency of roof support while mitigating the challenges associated with significant errors and compromised safety during manual control processes.

Experimental study on the punching failure model of soft red rock subgrade by BPT-BVM methods and it’s assessment of the load bearing capacity

The recommended bearing capacity of medium weathering mudstone foundation is less than the capacity of the rock structure to withstand loads in Southwest China. A comprehensive failure characterization of medium weathering mudstone in Chengdu has been performed including bearing plate test (BPT), binocular vision measurement (BVM) test, uniaxial compressive strength test, trial trench test of shallow rock surface and 3D imaging in this paper. Failure behavior of rock has been modeled with 3D imaging algorithm that utilizes Zhang’s calibration method in BVM system combination with trial trench test of shallow rock surface. The bearing capacity of medium weathering mudstone foundation were extracted from uniaxial experiments and BPT-BVM test by fitting relevant material properties to the data. The results revealed that: Bearing capacity of medium weathering mudstone of layered isotropic in Chengdu is undervalued. Specifically, the characteristic load carrying value is in the range 1500–2500 kP, that is 50% higher than in the local standard system. Failure process is different from Hoek–Brown Failure Criterion, presenting a wave peak transfer phenomenon of the increment displacement into the distance. Thus, it can be reduced to that of punching failures for thin bedded structures of Moudstone foundations. Compressive strength of soft rock proves to be main factor limiting the bearing capacity, a clear correlation between the uniaxial compressive strength reduction coefficient and the bearing capacity has been used to establish, leading to the proposal of a load bearing capacity prediction model.

Automated Binocular Vision Measurement of Food Dimensions and Volume for Dietary Evaluation

We propose a novel binocular vision technique to obtain food dimensions for dietary evaluation. A wearable device acquired images of food samples, which were segmented using the improved Chan-Vese model and compared with virtual objects. Results showed this method to be robust, achieving higher accuracy than conventional correspondence techniques.

Binocular vision measurement system for geometric error of 3D printers at high temperature

Binocular vision measurement system for geometric error of 3D printers at high temperature

DIC measurement method based on binocular stereo vision for image 3D displacement detection

The deformation detection of large machinery is usually achieved using three-dimensional displacement measurement. Binocular stereo vision measurement technology, as a commonly used digital image correlation method, has received widespread attention in the academic community. Binocular stereo vision achieves the goal of three-dimensional displacement measurement by simulating the working mode of the human eyes, but the measurement is easily affected by light refraction. Based on this, the study introduces particle swarm optimization algorithm for target displacement measurement on Canon imaging dataset, and introduces backpropagation neural network for mutation processing of particles in particle swarm algorithm to generate fusion algorithm. It combines the four coordinate systems of world, pixel, physics, and camera to establish connections. Taking into account environmental factors and lens errors, the camera parameters and deformation coefficients were revised by shooting a black and white checkerboard. Finally, the study first conducted error analysis on binocular stereo vision technology in three dimensions, and the relative error remained stable at 1 % within about 60 seconds. At the same time, three algorithms, including the spotted hyena algorithm, were introduced to conduct performance comparison experiments using particle swarm optimization and backpropagation network algorithms. The experiment shows that the three-dimensional error of the fusion algorithm gradually stabilizes within the range of [–0.5 %, 0.5 %] over time, while the two-dimensional error generally hovers around 0 value. Its performance is significantly superior to other algorithms, so the binocular stereo vision of this fusion algorithm can achieve good measurement results.

Binocular vision vibration measurement based on pixel coordinate matching of inner corner points in a chequerboard

A binocular vision measurement system provides a simple method for obtaining three-dimensional vibration data from moving objects, which is suitable for vibration monitoring of large structures such as bridges. Aiming to address the problem that the feature selection process for binocular visual inspection affects the measurement accuracy, chequerboard feature points are selected in this paper for carrying out a visual displacement measurement method. Firstly, pixel coordinate matching of the inner corner points in the chequerboard is completed and then a binocular vision measurement system is established. The measurement results are compared with using circular feature points. Secondly, the binocular vision measurement model is applied to the vibration measurement of a cantilever beam. Using comparisons with a three-axis acceleration sensor, the effectiveness and accuracy of this method are evaluated. Finally, the method is applied to measure the vibration of the cantilever beam under different load conditions and its vibration characteristics are analysed. The results show that the accuracy of the binocular vision measurement method based on pixel coordinate matching of the inner corner points in the chequerboard is higher than that using circular feature points. From comparisons with the acceleration sensor, the measurement error of this method is found to be small. In addition, the method can effectively analyse the vibration performance of a cantilever beam under different load conditions. Therefore, this measurement method is effective and provides a theoretical basis for the identification of vibration characteristics in large engineering structures.

Binocular measurement method for a continuous casting slab based on the one-dimensional probabilistic Hough transform and local sub-pixel sifting.

Due to the low measurement accuracy of the continuous casting slab model caused by difficulty in detecting ideal corners, a binocular measurement method based on the one-dimensional probabilistic Hough transform and local sub-pixel sifting is proposed. First, the one-dimensional probabilistic Hough transform based on inclination angle voting and the Freeman chain code is used to detect the line segments of the exterior outline. Next, sub-pixel points are extracted in each region of interest (ROI) by using Zernike moments, and sifted in the overlapping area of adjacent ROIs. Then the orthogonal total least squares (TLS) method is applied to fitting sub-pixel edges. Finally, after the key points are matched, three-dimensional localization and measurement are completed according to the binocular vision measurement principle. The experimental results show that the minimum relative error and average relative error of length reach 0.3401% and 0.3945%, respectively, satisfying the measurement requirement. Compared with scale-invariant feature transform (SIFT) and the oriented FAST and rotated BRIEF (ORB), the measurement error of the proposed algorithm is reduced by 80.01% and 74.63%, respectively. Compared with another edge fitting method based on k-means clustering and least squares fitting, its measurement error is reduced by 34.11%, and the time consumption is shortened by 39.07%, verifying its excellent performance in accuracy and efficiency.

Binocular vision measurement for large-scale weakly textured ship hull plates using feature points encoding method

The accurate forming measurement for curved hull plates is vital to ensure the quality of shipbuilding. However, the weak texture, large scale and complex shapes of curved hull plates present challenges for shape inspections. To enrich the texture of plates, checkerboard patterns are combined with the binocular vision system. Moreover, calculating the sub-pixel coordinates of corner points enhances reconstruction accuracy. A novel encoding method for the feature points on the complex shaped plates with repeatable texture is proposed to match the homologous points correctly. Finally, three dimensional coordinates are obtained by triangulation. Experimental results show that the dimensional accuracy is 300 ± 0.636 mm and the average accuracy is ±0.219 mm by registration. Inspection processes are completed within 1 min. This method is more automatic and convenient than contact measurement. Moreover, the efficiency is improved by about 2 times than multi-vision measuring method, enhancing the accuracy and efficiency of measurement for curved hull plates.

Research on a measurement method for spatio-temporal distribution of fragments based on a high-speed camera network

To reliably measure multiple spatio-temporal distribution parameters of fragments in the blast field, a measurement method for spatio-temporal distribution of fragments based on a high-speed camera network is proposed. The method is based on the principle of binocular stereo vision measurement and consists of four steps: fragment image acquisition, target detection of multi-view images, fitting of motion trajectory, and calculation of the spatio-temporal distribution parameters. The test results demonstrate that the method can simultaneously measure multiple parameters in the intersecting field of view. Compared with the measurement results of the array target, the absolute value of the measurement relative error of velocity was less than 1.5%, and the measurement results of scatter angle and distribution density were consistent. It indicates that the measurement results of the proposed method are reliable, and the method is a feasible and effective measurement method for spatio-temporal distribution parameters of fragments in the explosion filed.

Geometric Error Calibration of XYZ-3RPS Hybrid Kinematic Machine via Binocular Vision

Abstract To improve the motion accuracy of an XYZ-3RPS hybrid kinematic machine (HKM), a geometric error calibration method via binocular vision measurement is studied. First, to separately calibrate the series kinematic mechanisms (SKMs) and parallel kinematic mechanisms (PKMs), the geometric error identification equations (GEIEs) of the XYZ SKM and 3RPS PKM are derived, respectively. By analyzing the different influence principles of the geometric errors on the position and attitude of the 3RPS PKM, a constraint function is added to the GEIE of the PKM to improve the calculation accuracy. Moreover, the geometric error compensation strategy is based on the structural characteristics of the XYZ-3RPS HKM. In addition, based on the principle of binocular vision measurement, two calibration plates, called dynamic and static calibration plates, are designed as markers to define the coordinate systems, enabling the acquisition of full positions and attitudes. Furthermore, a marker transformation method and an in-situ adjustment method are designed to determine the positions and attitudes of the HKM required for calibration such that the marker is always at the center of the field of view of the camera to improve measurement accuracy. Finally, the effectiveness of the calibration method is verified through prototype experiments.

Experimental Aeroelastic Investigation of an All-Movable Horizontal Tail Model with Bending and Torsion Free-Plays

In this study, an experimental investigation is performed on a scaled, all-movable horizontal tail to study the aeroelastic behaviors induced by multiple free-plays. The dynamic response in wind tunnel tests is measured by strain gauges, an accelerometer, and a binocular vision measurement system. The obtained results indicate that the present aeroelastic system exhibits highly nonlinear characteristics and undergoes two independent limit cycle oscillations (LCOs) induced by bending free-play and torsion free-play, respectively. Further, various parametric studies are conducted to evaluate the effects of the free-play angles, angle of attack, flow velocity, and gust excitation on the LCOs. It is found that the value of free-play angle has no significant effect on the critical flow velocity which leads to the occurrence of LCOs. The amplitude and frequency of LCOs increase with the increasing free-play angle and flow velocity. Moreover, the horizontal tail experiences high-order harmonic resonances when LCOs appear. Finally, the stability of limit cycles is analyzed based on the gust excitation experiment. Overall, compared to an all-movable horizontal tail with single free-play, the multiple free-plays system exhibits more complex dynamic behaviors. In this paper, the measured results of the scaled model, which has a similar mass distribution and stiffness distribution as actual aircraft, may be valuable for predicting such LCOs induced by multiple free-plays, and providing a reference for the design of all-movable horizontal tail to prevent LCOs.

Three-Dimensional Mapping Technology for Structural Deformation during Aircraft Assembly Process

Owing to the assembly state changes during aircraft assembly processes, assembly force-deformation problem occurs. To obtain the structure shape in the product assembly process efficiently and accurately, a three-dimensional (3D) mapping technology for the structural deformation during the aircraft assembly process is proposed combined with a fiber Bragg grating (FBG) optical fiber sensor and binocular vision measurement system. First, this study established a curvature transformation model using optical fiber monitoring data, obtained the 3D spatial deformation of the product, and completed the unification of the optical fiber wavelength change and spatial 3D point coordinate heterogeneous data. Second, a mesh deformation optimization algorithm based on point-cloud optimization was established. Subsequently, the deformation effects of four mesh deformation models were compared to verify the feasibility and accuracy of HEC-Laplace, and the 3D mapping of the product structure shape in the assembly process was realized. Finally, a cantilever wing model was used to verify the deformation of different loading modes. The results show that the product structure changes can be accurately obtained through the proposed technology, thereby improving the accuracy control and overall assembly quality in the aircraft assembly process and providing a theoretical basis for intelligent aircraft assembly.

Tire deformation measurement technology based on digital image correlation method

The contact between the tire and the road surface causes tread deformation, and using traditional methods to measure tire tread deformation is difficult and time-consuming. Therefore, based on the principle of the digital image correlation method, this paper develops a binocular vision measurement system and realizes the measurement of tire tread deformation through the test bench for tire ground pressure distribution measurement. The related programs of the measurement system, including camera calibration, image matching, and strain fitting, are proposed. The deformation measurement accuracy of the test bench is validated by the sprayed speckle rubber sheet test, and the relative error of the theoretical value and measurement value is within 2%. The tire tread deformation test under static loading conditions was carried out using the test bench, and the tread strain distribution in the contact area was obtained. The analysis of the test results shows that the strain distribution in the contact patch is axisymmetric under static loading conditions. The longitudinal (tire rolling direction) deformation of the tread at the leading edge and the trailing edge area of the contact patch has the largest strain, while the deformation of the shoulder area on both sides of the contact patch is relatively small. For lateral (tire axial direction) deformation, the leading edge and the trailing edge areas of the contact patch are less deformed, and the strain distribution is approximately axisymmetric. The shoulder area of the contact surface has a larger deformation. The tire’s outer shoulder area has a larger deformation than the inside shoulder area, which may be due to the asymmetric tread structure of the tire. For the vertical deformation of the tread, the central area of the tread has the smallest strain, while the surrounding area has a much larger vertical strain.

BSV System with MSC and Its Application in the Measurement of Surface Cracks

In this paper, we have proposed a flexible noncontact crack‐size measurement method that can realize binocular stereo vision measurement with only a single camera. On the premise that the camera’s intrinsic parameters have been accurately calibrated, we use a camera to collect the image of the crack from two directions. Then, we calculate the motion parameters using the collected images from the camera in different positions. In addition, Canny algorithm is used to extract the edge pixels of crack images. Finally, we establish the binocular stereo vision model for crack measurement according to the camera parameters, the motion parameters, and the edge information of crack images. Thus, we can measure the crack size through this model. Experimental results show that the measurement error is less than 5% under a distance of 2 meters, which can effectively prove the precision of the proposed method. In addition, our method only uses a single camera. Compared with the traditional binocular stereo vision method, this method is not only flexible but also more economical.

Binocular vision measurement method incorporating one-dimensional probabilistic Hough transform and local Zernike moment

Binocular vision measurement method incorporating one-dimensional probabilistic Hough transform and local Zernike moment

Binocular visual dimension measurement method for rectangular workpiece with a precise stereoscopic matching algorithm

Binocular vision measurement benefits from high prediction robustness and low structural complexity. However, there are still significant flaws in its accuracy. In this paper, binocular vision measurement of a rectangular workpiece is investigated. A new precise measurement method based on binocular vision is designed to achieve precise measurement of rectangular workpiece dimensions. Firstly, an algorithm for workpiece location based on Zernike moments and corner matching is proposed and employed to precisely locate the workpiece and extract the sub-pixel coordinates of discrete points on an image’s edge. Then, a novel stereoscopic matching algorithm combined with epipolar-geometry and cross-ratio invariance (CMEC) is proposed to improve the accuracy of binocular vision stereoscopic matching. Finally, a projection plane is introduced after the 3D reconstruction of discrete points in the workpiece contours by fitting the plane with least squares. The projection plane limits the coordinate fluctuations of discrete points. Furthermore, the data screening is used to further improve the accuracy of size calculation. The experimental results of the standard checkerboard and actual workpiece show that CMEC’s matching accuracy reached 99%, and the proposed method’s measurement accuracy reached 0.018 mm. This work presents a novel algorithm for stereoscopic matching in binocular vision and machine vision measurement.

An automatic alignment method for discharge arm of mobile crushing station based on binocular vision and fuzzy control

The mobile crushing station is one of the main equipment of the semi-continuous open-pit mining system. The discharge arm and the receiving equipment are manually aligned, which has the problems of long alignment time and low alignment accuracy, which affects the working efficiency of the mining system. According to the development and application of space rendezvous and docking technology at home and abroad, the advantages and disadvantages of different measurement methods are compared and analysed, and the method of applying binocular measurement technology to system positioning in the automatic alignment system of the discharge arm is determined. There are three movements in the mechanical part of the designed discharge arm alignment control system, including the rotary motion of the visual measurement mechanism and the horizontal rotation and telescopic motion of the discharge arm. According to the kinematic analysis and binocular vision measurement theory, the deviation model of the alignment control of the discharge arm is established. A binocular vision measurement and localization method based on the combination of stereo calibration and template matching is proposed, which achieves surprising measurement accuracy. An automatic alignment method of the discharge arm of the mobile crushing station is proposed based on the binocular vision and fuzzy control method. Its validity is verified by simulation and experiment. The strategy of motion decomposition is applied to the alignment system to avoid unnecessary motion of the discharge arm. The research results all show that the alignment method can achieve the angle deviation within ±0.5 degrees, the distance deviation within ±15 mm, and the test alignment time is about 5 minutes, which is better than other alignment control models; the alignment accuracy and the alignment time are improved by more than 50%. The method can control the discharge arm to complete the alignment task quickly and smoothly, which lays a foundation for the further automatic research of the discharge arm.

Automatic Detection of Chip Pin Defect in Semiconductor Assembly Using Vision Measurement

Abstract With the development of semiconductor assembly technology, the continuous requirement for the improvement of chip quality caused an increasing pressure on the assembly manufacturing process. The defects of chip pin had been mostly verified by manual inspection, which has low efficiency, high cost, and low reliability. In this paper, we propose a vision measurement method to detect the chip pin defects, such as the pin warping and collapse that heavily influence the quality of chip assembly. This task is performed by extracting the corner feature of the chip pins, computing the corresponding point pairs in the binocular sequence images, and reconstructing the target features of the chip. In the corner feature step, the corner detection of the pins using the gradient correlation matrices (GCM), and the feature point extraction of the chip package body surface using the crossing points of the fitting lines are introduced, respectively. After obtaining the corresponding point pairs, the feature points are utilized to reconstruct the three dimensional (3D) coordinate information in the binocular vision measurement system, and the key geometry dimension of the pins is computed, which reflects whether the quality of the chip pins is up to the standard. The proposed method is evaluated on the chip data, and the effectiveness is also verified by the comparison experiments.

Method for improving the measurement accuracy of binocular stereo vision in a scattering environment.

In the scattering environment, binocular stereo vision measurement technology produces large errors due to the change of refractive index of the imaging light path and the decrease in target image contrast. To address this problem, this paper proposes a method for improving the measurement accuracy of binocular stereo vision in a scattering environment combined with polarization imaging theory. First, scattering images with different polarization directions are obtained and filtered by a Gaussian low-pass filter to calculate the degree of polarization and angle of polarization. Then, the scattered light intensity is calculated by using polarization information to obtain images after removing the scattering. Second, feature extraction and matching are carried out for the images after scattering removal. Finally, the target is measured based on the binocular stereo vision measurement model. The experimental results show that when the scattering concentration is high enough, the conventional method can no longer perform measurement, but the method proposed in this paper can still obtain the target parameters at this time, and can also improve measurement accuracy by at least 46.30%. In conclusion, the proposed method provides a valuable reference to improve the accuracy of binocular stereo vision measurement in a scattering environment by reducing the interference of scattering light.