Binocular Vision System Research Articles

A non-iterative calibration method for the extrinsic parameters of binocular stereo vision considering the line constraints

The calibration accuracy of the binocular stereo vision system is important for precision measurement. The extrinsic parameters (structure parameters), which describe the relative position of the two cameras, need to be optimized before measurement. The optimization of extrinsic parameters is accomplished through iterations of minimizing an objective function. It is hard to decide the optimum objective function between various objective functions. The calibration accuracy is decreased when the objective function is incorrect, which can result in excessive optimization or fall into a local minimum (ambiguous solution), notably for planar targets. In this paper, we proposed a non-iterative calibration method for the extrinsic parameters of binocular stereo vision considering the line constraints of a planar target. Each line formed by every two calibration points can provide two independent constraints. Additional equations for solving the extrinsic parameters can be obtained using these constraints. We added these equations to the equations system established by the homography of the target plane and image plane. The extrinsic parameters of the binocular stereo vision system can be solved non-iteratively. Using the parameters calibrated by our method, greater improvements in the accuracy and stability of measurements occurred, compared to other existing methods. In the −45° to +45° measuring range, the maximum measurement error is no more than 0.037° and the average measurement error is no more than 0.022°. In the 0 to 28 mm measuring range, the maximum measurement error is no more than 0.039 mm and the average measurement error is no more than 0.012 mm. The maximum length measurement error is no more than 0.038 mm and the average length measurement error is no more than 0.011 mm while measuring a 240 × 180 mm2 target.

Target-free 3D tiny structural vibration measurement based on deep learning and motion magnification

• A novel target-free tiny vibration displacement measurement approach is proposed. • It is based on deep learning and motion magnification techniques. • Binocular vision system is used for measuring 3D vibration measurement. • Deep learning techniques are used for key-points detection, matching and tracking. • Experimental and in-field studies are conducted to validate the accuracy. In the field of structural health monitoring (SHM), computer vision based methods have been usually developed for 2-dimensional (2D) vibration displacement measurement and crack identification of civil engineering structures. However, the accurate measurement of tiny 3-dimensional (3D) vibrations for real civil engineering structures remains a very difficult task. To overcome this challenge, this paper proposes a target-free full-field 3D tiny vibration measurement approach for civil engineering structures by using a binocular vision system. The proposed approach is based on deep learning and motion magnification. A phase-based video motion magnification algorithm is employed to achieve a high measurement accuracy of tiny vibrations at the submillimeter level. The advanced key point detection, matching and tracking algorithms via deep learning techniques are employed to achieve target-free tiny vibration displacement measurement. The accuracy and performance of the proposed approach are evaluated through experimental tests on a steel cantilever beam in the laboratory. In-field experimental tests are conducted on a pedestrian bridge on a university campus to investigate the accuracy of the proposed approach in practical applications. The measured 3D tiny vibration displacement from the proposed approach is compared with those measured by laser displacement sensors, and the derived acceleration responses are compared with those measured from the installed accelerometers on the testing structures. The results demonstrate that the 3D tiny vibration measurements are obtained accurately by the proposed approach.

Refraction calibration of housing parameters for a flat-port underwater camera

In underwater computer vision systems, the camera is usually housed in a watertight case with a glass interface between water and air, which causes the light rays to refract, affecting image formation geometrically. In this paper, we propose an approach to calibrate the housing parameters of a flat-port underwater camera for a reasonable underwater refractive camera model. Our approach is specifically designed to deal with the refraction effect caused by the light traveling in different mediums. In this camera model, the calibrated housing parameters include the distance between the camera and glass interface and the normal of the glass interface in the camera coordinate system. We employ an underwater binocular vision system to calibrate the housing parameters in two steps. First, the distance to the glass interface is estimated by capturing a checkerboard image and computing the distance from the camera center to a known flat plane. Then, the normal of the glass interface is calculated by a multiobjective optimization method using geometric constraints in the underwater binocular stereo system. Extensive experiments have been conducted on our underwater dataset and the results verify the effectiveness of the proposed scheme for underwater camera calibration and three-dimensional measurements.

Optical three-dimensional shape measurement based on structured light and a binocular vision system.

Three-dimensional shape measurement based on structured light is affected by two factors: the number of fringe patterns and the phase unwrapping process. Although one-shot technology can get the wrapped phase, it is not suitable for measuring complex surface. Moreover, phase unwrapping also affects measurement speed and accuracy. To overcome these problems, a two-dimensional wavelet transform with binocular vision system is proposed. Wavelet transform is used to get the wrapped phase based on the Morlet wavelet. In order to get a three-dimensional shape without phase unwrapping, a binocular vision system is used. The increase matching accuracy, the preliminary disparity, and the sub-pixel optimization are calculated, respectively. Based on the calibration parameters, three-dimensional information can be obtained directly from the wrapped phase. In addition, the average phase is calculated based on ambient pixels to confirm wrapped phase boundary. Experimental results demonstrate the feasibility and advantage of the proposed method. Compared with traditional methods, both measurement accuracy and measurement speed can be increased.

Three‐dimensional deformation measurement of aero‐engine high‐speed rotating blade surface based on stroboscopic structure digital image correlation method

Under the working conditions of aero-engine, the three-dimensional composite deformation of the blade is caused by the centrifugal force generated by high-speed rotation and the load generated by gas scouring. Aiming at the problem of three-dimensional composite deformation on the surface of aero-engine blades under high-speed rotation, a three-dimensional deformation measurement method based on stroboscopic structure digital image correlation method is proposed, and a method for eliminating rigid body displacement induced by the trigger error of the stroboscope is proposed. First, the laser trigger-stroboscopic irradiation system is used to freeze the high-speed rotating blade under a specific phase. Then, the binocular vision system is used to collect the surface image of the rotating blade. Finally, the digital image correlation method is used to measure the surface deformation field of the high-speed rotating blade. In this paper, the experimental platform is built and the displacement field and main strain field of the blade surface under high-speed rotating state are measured. The experimental results show that the stroboscopic structure digital image correlation measurement system and method have realized displacement measurement accuracy of 2 μm and strain measurement accuracy of 100 μɛ and have realized the surface deformation measurement of the ducted turbine fan blade under the condition of 6000 rpm (CFM56-7 engine N1 speed). The results show that the method is effective and can provide a reliable measurement method for the deformation measurement of high-speed rotating blades in large engine manufacturing enterprises such as Aero Engine Corporation of China.

Transcranial random noise stimulation and exercise do not modulate ocular dominance plasticity in adults with normal vision.

Short-term deprivation of one eye by monocular patching causes a temporary increase in the contribution of that eye to binocular vision when the eye patch is removed. This effect, known as ocular dominance plasticity, provides a model of neuroplasticity within the human binocular visual system. We investigated whether physical exercise and the non-invasive brain stimulation technique transcranial random noise stimulation (tRNS), two interventions that may increase visual cortex neuroplasticity, enhance ocular dominance plasticity when delivered individually or in combination. Ocular dominance was measured using a grating rivalry test and a dichoptic letter contrast polarity judgment test. We observed robust ocular dominance changes for both outcome measures following 2-hour monocular deprivation; however, the magnitude of the effect was not influenced by exercise or tRNS. Ocular dominance plasticity may already be maximal after 2 hours of monocular deprivation in those with normal vision and therefore cannot be augmented by interventions designed to enhance neuroplasticity.

Deflection monitoring of morphing winglet by binocular vision system with environment adaptability

In this paper, a method with environment adaptability for measuring the deflection angle of morphing winglet based on binocular vision system is proposed. The measurement accuracy of the binocular vision system under the different imaging environment is studied. The circular luminous marks are arranged on the surface of the winglet as detection targets. The edges of luminous marks are detected by the Anisotropic Gaussian Kernel function. Ellipse fitting is carried out on the edges to obtain the center points of the circular marks. According to the coordinates of the luminous mark center, the winglet angles which is changed from −14° to 20° can be obtained. Experimental results of winglet deflection show that the three-dimensional coordinate error of luminous marks is less than 1.2 mm. The angular deviation under different light conditions is less than 0.1°. The fixed angular deflection test show that the maximum repetition error is less than 0.50 mm. The method proposed in this paper can accurately get the angular deflection of the winglet under different light environments.

MoFiM: A morphable fish modeling method for underwater binocular vision system

AbstractFish morphology is an essential basis for fishery management, as it can reflect the growth status of fishes. Noncontact 3D reconstruction of underwater fish is a new way to obtain fish morphology. While it is difficult to reconstruct fish on account of the inadequate information caused by fish swimming and poor underwater imaging. This article introduces a morphable fish modeling method for the underwater binocular vision system. First, we define a fish representation based on selected landmarks. Then, we propose a chirality‐supervision incorporated hourglass network to estimate fish orientation and fish 2D landmarks simultaneously, and calculate fish 3D landmarks by triangulation. Next, we propose a fish modeling method which is based on 3D landmarks and introduce the optimization procedure of fish modeling. Finally, we obtain the complete 3D fish model corresponding to the input images. To train our network and build a parametric model, we constructed an underwater vision dataset and fish instance dataset respectively. We conducted experiments with grass carp as an example, and the experimental results show that our method can achieve effective fish modeling and is useful for noncontact measurement of underwater fish.

A Driving Assistance System for Solving A-Pillar Blind Spots Through Gaze Detection and Field-of-View Estimation

The blind spots brought by a car’s A-pillar are main reasons of most of accidents. In this work, a vision system that focuses on eliminating A-pillar blind spot of a car without any affection of driver’s operation is investigated. The driver’s facial features are captured by a binocular vision system that is mounted on A-pillar, head poses and gaze line directions are reconstructed. The generated blind spot by A-pillar is then simultaneously calculated according to the position of driver’s gaze. A field-of-view of the blind spots is displayed in a screen system mounted on the A-pillar. The screen conjointly with front and side windows thus provides a full view field for the driver, which can effectively reduce the occurrence of accidents.

Computer vision-based system for early diagnosis of stereoscopic vision alterations

ABSTRACT Stereopsis is the three-dimensional perception capability, which is possible when binocular vision is present. Development of binocular vision system ends around 7-year-old, and after this time brain connections are already set, therefore, it will be difficult to recover stereopsis. Early diagnosis of Stereoscopic Visual Alterations (SVA) in the childhood is paramount to receive an appropriate treatment as soon as possible. Currently, to detect SVA, ophthalmologists commonly carry out tests based on cards. All these tests are based on the random-dot-stereogram principle, with different seconds of arc images, which allows doctors to graduate stereopsis. Some limits of this tests have been identified, such as monocular clues (e.g., the contours of the objects), or the non-standardized range levels depending on the test. This paper presents a novel concept of measuring stereopsis based on computer vision techniques. The system detects SVA in patients and calculates the degree of the perceived depth. As early diagnosis of SVA may suppose a therapeutic possibility, this platform is aimed at children by using stereoscopic models with varied and attractive designs. To validate the proposal, an early-stage prototype has been implemented and an objective evaluation of the measurement accuracy and reliability has been carried out with satisfactory results.

A study on length measurement method of hot large forgings based on binocular vision system

A length measurement method of hot large forgings is proposed in this paper. Firstly, the images of hot forgings are acquired by two monochrome Charge Coupled Device (CCD) cameras, and the light strips are detected according to the characteristics that the value of the pixels in the vertical direction of the light strips is approximate the Gaussian distribution. Secondly, the sub-pixel centers of each light strips in the left and right images are calculated and used for matching and three-dimensional (3D). Finally, the curve represented by quintic polynomial is employed to fit the two-dimensional (2D) points which are projected by 3D points on the fitted plane, and the edge of the forging is calculated by the curvature of the fitted curve. The experimental results in laboratory show that the measurement accuracy can reach 0.547%. The length measurement time of hot forging is 3.7 s.

4D dual-mode staring hyperspectral-depth imager for simultaneous spectral sensing and surface shape measurement.

A 4D dual-mode staring hyperspectral-depth imager (DSHI), which acquire reflectance spectra, fluorescence spectra, and 3D structural information by combining a staring hyperspectral scanner and a binocular line laser stereo vision system, is introduced. A 405 nm laser line generated by a focal laser line generation module is used for both fluorescence excitation and binocular stereo matching of the irradiated line region. Under the configuration, the two kinds of hyperspectral data collected by the hyperspectral scanner can be merged into the corresponding points in the 3D model, forming a dual-mode 4D model. The DSHI shows excellent performance with spectral resolution of 3 nm, depth accuracy of 26.2 µm. Sample experiments on a fluorescent figurine, real and plastic sunflowers and a clam are presented to demonstrate system's with potential within a broad range of applications such as, e.g., digital documentation, plant phenotyping, and biological analysis.

UAV Landing Aid Hexapod Robot based on ArUco Marker and Sparse Optical Flow

A UAV (Unmanned Aerial Vehicle) landing robot system based on a six-degree-of-freedom hexapod and computer vision with ArUco marker and sparse optical flow is implemented in this paper. By using binocular camera measuring the real-time position and orientation of UAV, a dynamic target tracking method based on ArUco marker and sparse optical flow is proposed. In this method, the ArUco marker is combined with sparse optical flow to improve the accuracy of target detection and the matching error of binocular stereo matching. The binocular vision system with ArUco marker is used to measure and calculate the relative coordinates of markers in the camera coordinate system. Then the UAV contour fitting is conducted in order to obtain the real-time position and orientation of this mobile target. The effectiveness of the novel UAV landing hexapod robot system based on ArUco marker combined with sparse optical flow is verified through the UAV capture experiment.

Linear Laser Scanning Measurement Method Tracking by a Binocular Vision

The 3D scanning of a freeform structure relies on the laser probe and the localization system. The localization system, determining the effect of the point cloud reconstruction, will generate positioning errors when the laser probe works in complex paths with a fast speed. To reduce the errors, in this paper, a linear laser scanning measurement method is proposed based on binocular vision calibration. A simple and effective eight-point positioning marker attached to the scanner is proposed to complete the positioning and tracking procedure. Based on this, the method of marked point detection based on image moment and the principle of global coordinate system calibration are introduced in detail. According to the invariance principle of space distance, the corresponding points matching method between different coordinate systems is designed. The experimental results show that the binocular vision system can complete localization under different light intensities and complex environments, and that the repeated translation error of the binocular vision system is less than 0.22 mm, while the rotation error is less than 0.15°. The repeated error of the measurement system is less than 0.36 mm, which can meet the requirements of the 3D shape measurement of the complex workpiece.

A Shape Reconstruction and Measurement Method for Spherical Hedges Using Binocular Vision.

The center coordinate and radius of the spherical hedges are the basic phenotypic features for automatic pruning. A binocular vision-based shape reconstruction and measurement system for front-end vision information gaining are built in this paper. Parallel binocular cameras are used as the detectors. The 2D coordinate sequence of target spherical hedges is obtained by region segmentation and object extraction process. Then, a stereo correcting algorithm is conducted to keep two cameras to be parallel. Also, an improved semi-global block matching (SGBM) algorithm is studied to get a disparity map. According to the disparity map and parallel structure of the binocular vision system, the 3D point cloud of the target is obtained. Based on this, the center coordinate and radius of the spherical hedges can be measured. Laboratory and outdoor tests on shape reconstruction and measurement are conducted. In the detection range of 2,000–2,600 mm, laboratory test shows that the average error and average relative error of standard spherical hedges radius are 1.58 mm and 0.53%, respectively; the average location deviation of the center coordinate of spherical hedges is 15.92 mm. The outdoor test shows that the average error and average relative error of spherical hedges radius by the proposed system are 4.02 mm and 0.44%, respectively; the average location deviation of the center coordinate of spherical hedges is 18.29 mm. This study provides important technical support for phenotypic feature detection in the study of automatic trimming.

Recent understanding of binocular vision in the natural environment with clinical implications.

Technological advances in recent decades have allowed us to measure both the information available to the visual system in the natural environment and the rich array of behaviors that the visual system supports. This review highlights the tasks undertaken by the binocular visual system in particular and how, for much of human activity, these tasks differ from those considered when an observer fixates a static target on the midline. The everyday motor and perceptual challenges involved in generating a stable, useful binocular percept of the environment are discussed, together with how these challenges are but minimally addressed by much of current clinical interpretation of binocular function. The implications for new technology, such as virtual reality, are also highlighted in terms of clinical and basic research application.

Spatiotemporal Matching Cost Function Based on Differential Evolutionary Algorithm for Random Speckle 3D Reconstruction

Random speckle structured light can increase the texture information of the object surface, so it is added in the binocular stereo vision system to solve the matching ambiguity problem caused by the surface with repetitive pattern or no texture. To improve the reconstruction quality, many current researches utilize multiple speckle patterns for projection and use stereo matching methods based on spatiotemporal correlation. This paper presents a novel random speckle 3D reconstruction scheme, in which multiple speckle patterns are used and a weighted-fusion-based spatiotemporal matching cost function (STMCF) is proposed to find the corresponding points in speckle stereo image pairs. Furthermore, a parameter optimization method based on differential evolutionary (DE) algorithm is designed for automatically determining the values of all parameters included in STMCF. In this method, since there is no suitable training data with ground truth, we explore a training strategy where a passive stereo vision dataset with ground truth is used as training data and then apply the learned parameter value to the stereo matching of speckle stereo image pairs. Various experimental results verify that our scheme can realize accurate and high-quality 3D reconstruction efficiently and the proposed STMCF exhibits superior performance in terms of accuracy, computation time and reconstruction quality than the state-of-the-art method based on spatiotemporal correlation.

A Hybrid Calibration Method for the Binocular Omnidirectional Vision System

Omnidirectional three-dimensional (3D) measurement based on the binocular omnidirectional vision system attracts increasing attention in computer vision. A 360° horizontal field of view (FOV) and 180° vertical FOV are obtained by the binocular omnidirectional vision system with only a pair of images. However, a nonlinear relationship exists between points on image planes and in 3D space for the binocular omnidirectional vision system, and traditional calibration methods are no longer applicable. This study proposes a hybrid calibration approach, which effectively fuses the unified camera model and the backpropagation neural network with a virtual 3D target, in which the neural network improved with genetic algorithm compensates all kinds of distortions and errors. First, the unified camera model is used to calibrate the intrinsic parameters, extrinsic parameters, and a transformation matrix. Second, the pixel coordinates on image planes are converted to 3D coordinates by 3D reconstruction through binocular vision. The errors between actual and calculated 3D coordinates are obtained. In addition, the neural network improved with genetic algorithm is established and trained with pixel coordinates on image planes and errors. Experimental results indicate that the average measurement errors in the X and Y directions of the central area are respectively reduced to 0.1057 and 0.1548 mm, and those of the edge area are reduced to 0.1992 and 0.2748 mm. Compared with mathematical and neural network methods, the hybrid calibration method provides more stable, robust, and accurate results. It is proven suitable for complex calibration scenes with large FOV, high distortion, and strong nonlinearity.

Pose Testing Method for Medical Service Robots Based on Vision System

Aiming at the medical service robot used in the indoor environment, this study proposes a method to test the pose accuracy of its working surface based on the binocular vision system. This method uses a binocular vision coordinate system to measure targets fixed on the working surface of the service robot, aligns the measurement system with the robot base coordinate system through the nonlinear least squares method, and integrates the multi-eye image data to achieve the accuracy test of the working surface. Finally, the vision test program was tested and verified on a mobile service robot model ABIR X8. According to the accuracy index given in GB/T 38124 Service Robot Performance Test Method, the test results of its pose accuracy were obtained.

Монокулярная и бинокулярная визоконтрастная чувствительность у детей младшего школьного возраста

Purpose.. To evaluate monocular and binocular contrast sensitivity to achromatic and monochromatic colors in children of primary school age from the specific regions in the North of Russian Federation. Material and methods. 36 children of the city of Tyumen and 422 children, students at the secondary school of the village of Beloyarsk of the Priuralsky district and the village of Kharampur of the Purovsky district of the Yamalo-Nenets Autonomous District, were examined. Contrast perimetry using an atlas of test images was performed to assess the frequency-contrast characteristics (FCC) of the visual system of the examined individuals. The results of the study underwent statistical analysis using the IBM SPSS Statistics ver. 22.0 software package. Results. The monocular spatial contrast sensitivity of the children from the specific regions in the North of Russian Federation in comparison with the residents of Tyumen is significantly lower (p<0.001). The formation of the binocular visual system in children from the specific regions in the North of Russian Federation. is faster. In the area of high spatial frequencies in children, the formation of perception of blue color falls behind, which is not observed in low spatial frequencies. At the same time, in the area of medium frequencies, the perception of all monochromatic colors stays far behind, especially green. Conclusion. Normative indicators of frequency-contrast sensitivity to achromatic color of primary school children have been formed. It is established that binocular spatial contrast sensitivity in children of primary school age from the specific regions in the North of Russian Federation is formed faster than monocular spatial-frequency components, in comparison with schoolchildren of Tyumen. Key words: spatial contrast sensitivity, visual analyzer, primary school age, the specific regions in the North of Russian Federation