Stereo Vision Method Research Articles

Profilometry: a non-intrusive active stereo-vision technique for wave-profile measurements in large hydrodynamic laboratories

Profilometry is proposed as a novel non-intrusive image-based technique to capture the profile of the air–water interface as a dense point cloud. It can be classified as an active stereo-vision method applied to the study of gravity-driven water waves and specifically developed to be used in large hydrodynamic laboratories. As an active vision technique, it relies on the use of light sources, and as a stereo technique, it requires one or more high-speed camera pairs for imaging the same scene synchronously. To enhance the visibility of the laser lights on the wave profile, the water surface is sprayed with water droplets. Profilometry, compared to standard wave probes, can be considered as an alternative source of information that can augment spatial resolution to the identification of the air–water interface to capture nonlinear wave-evolution mechanisms and violent wave–body interactions. Its feasibility and accuracy are examined preliminarily in a small-scale flume and then in a large-scale towing tank using long-crested wave scenarios, including regular, irregular, and focused gravity-driven waves, without the presence of a structure. The values of the wave steepness examined were various and included also quite steep cases with nearly vertical wave fronts. Role played by parameters of the technique, as well as of its setup in capturing the wave features are also analysed, with the aim to provide a useful guidance for other researchers that intent to use and develop further this approach.

Flexible technique for global calibration of multi-view stereo vision with a non-overlapping FOV using linear structured light projector

A flexible global calibration method for multi-view stereo vision with non-overlapping field of view (FOV) using linear structured light projector is presented in this paper. At least three groups of linear structured light plane equations are utilized to calculate the rotation between any pair of cameras with non-overlapping FOV, and then the closed solution of the translation is determined based on the constraint that the normal vector is perpendicular to the lines on linear structured light plane. Once the rotation and translation between all pairs of cameras with non-overlapping FOV are obtained, the rotation and translation of each camera relative to the reference position is eventually computed through global optimization. The experimental data reflects that the calibration accuracy is improved by 35.2% after global optimization, and the global calibration results of the proposed method deviates less from that of Zhang’s method.

COMPUTER VISION TECHNOLOGIES FOR DETECTING DEFECTS IN CYLINDRICAL OBJECTS

Currently, the development of computer vision makes it possible to solve many problems of detecting defects at various industrial facilities. One of the promising areas of application of these technologies is to identify inconsistencies in geometric parameters on cylindrical products. The purpose of this work is to review and systematize modern computer vision methods used to solve the problem of detecting defects on vertical cylindrical surfaces. The study analyzed existing approaches to the extraction of spatial characteristics of objects, including methods of stereo vision, spatial filtering and 3D reconstruction. Algorithms for identifying the main landmarks on a cylindrical surface were considered, which makes it possible to bind the coordinate system and localize areas of possible defects. Methods for estimating geometric deviations on the surface, which can act as criteria for detecting defects, have also been studied. As a result of the analysis, a classification of computer vision methods applicable to the problem of detecting defects on cylindrical objects was proposed. Promising directions for further research in the field of improving the accuracy of defect detection through a combination of various image processing algorithms have been identified.

Estimating cost of pothole repair from digital images using Stereo Vision and Artificial Neural Network

A significant amount of road maintenance cost goes into pothole repairs. The primary cost factors related to potholes are their size and depth, as larger and thicker potholes incur higher repair costs. However, existing methods for estimating pothole repair in developing countries rely on manual size measurements, which is time consuming, labor intensive, subjective and can lead to poor estimation of repair cost. This paper presents a system that can automatically determine the size of potholes from digital images and estimate the cost of repair. In this study, the stereo vision method was used to automatically estimate the depths of potholes from digital camera images. A feed-forward backward propagation Artificial Neural Network (ANN) was trained using pothole images acquired using mobile phones. The predicted depths and sizes of the potholes were then used to estimate the quantity of materials required to fill the potholes and subsequently, the cumulative cost of repair. Marking out and manual size measurements were performed for twenty randomly selected potholes in the Ugbowo Campus of the University of Benin, Nigeria. These measurements were compared against the estimated sizes of potholes predicted by the ANN model. A system was developed to automatically compute these material costs and considering other cost components such as transportation, labor, and equipment. Results obtained showed that the mean errors for depth, width and height estimates were 3.403%, 3.789% and 5.2617% respectively. Consequently, the developed system correctly estimated the cost of repair of the potholes considered in this study. A significant contribution of the paper is the speed and convenience of acquiring pothole data using a mobile phones without the need for on spot assessment of potholes or use of relatively more expensive stereoscopic camera setup.

Measurement Method for Contact Wire Wear Based on Stereovision.

The contact wire wear is an important parameter to ensure the safety operation of electric railways. The contact wire may break if the wear is serious, which leads to transportation interruptions. This study proposes an optical measurement method of contact wire wear, using stereovision technology. The matching method of stereovision based on line-scan cameras is proposed. A lookup-table method is developed to exactly determine the image resolution caused by the contact wire being in different spatial positions. The wear width of the contact wire is extracted from catenaries' images, and the residual thickness of the contact wire is calculated. The method was verified by field tests. The round-robin tests of the residual thickness at the same location present excellent measurement repetitiveness. The maximum difference value between dynamic test results and ground measurement results is 0.13 mm. This research represents a potential way to implement condition-based maintenance for contact wire wear in the future in order to improve the maintenance efficiency and ensure the safety of catenary infrastructure.

NEURAL NETWORK ARCHITECTURE FOR TEXT DECODING BASED ON SPEAKER'S LIP MOVEMENTS

In this paper, we tested a command recognition system using the SSI approach and conducted a series of experiments on modern solutions based on ALR interfaces. The main goal was to improve the accuracy of speech recognition in cases where it is not possible to use the speaker's non-noisy audio sequence, for example, at a great distance from the speaker or in a noisy environment. The obtained results showed that training the neural network on a GPU accelerator allowed to reduce the training time by 26.2 times using a high-resolution training sample with a size of the selected mouth area of 150 × 100 pixels. The results of the analysis of the selected speech recognition quality assessment metrics (word recognition rate (WRR), word error rate (WER), and character error rate (CER)) showed that the maximum word recognition rate of the speaker's speech is 96.71% and is achieved after 18 epochs of training. If we evaluate the character regonition rate of viseme recognition, the highest rate can be obtained after 13 epochs of training. Future research will focus on the use of depth cameras and stereo vision methods with increased frame rates to further improve the accuracy of voice command decoding in conditions of high background noise.

A novel three-dimensional surface reconstruction method for fabric appearance evaluation

Traditional fabric appearance evaluation method mainly relies on subjective and manual operation, which is not uniform and time-consuming. In this paper, one set of self-developed 3D reconstruction system and method based on photometric stereo is proposed for objective fabric appearance evaluation. A photometric stereo image acquisition system equipped with multi-directional lighting sources is developed for the 3D surface reconstruction. Lighting and camera calibration is conducted to obtain the position of light sources and the intrisic parameters of digital camera. Our experimental results show that 3D fabric surface could be reconstructed effectively based on photometric stereo vision method, it has the potential to be used for fabric appearance evaluation.

Method for three-dimensional reconstruction of dynamic stereo vision based on line structured light using global optimization

To overcome the limitation of the measurement range of static stereo vision, a three-dimensional (3-D) reconstruction method of dynamic stereo vision based on line structured light for metal surface is presented in this paper. The dynamic stereo vision is mainly composed of binocular cameras and a laser line generator, and moves along a horizontal guide rail. The intrinsic parameters of the cameras and the rotation and translation of the right camera relative to the left camera are previously obtained by Zhang’s method. To obtain the 3-D coordinates of the line structured light fringe in each position, the sub-pixel coordinates of the fringe center are extracted by Steger algorithm, and a joint calibration method between single camera and a laser line generator is proposed. The joint calibration method calculates the plane equation of the line structured light by using multiple checkerboard images with projected structured light line. To unify the 3-D coordinates of structured light fringe in each position to the reference coordinate system, the rotation and translation of the camera in each position relative to the reference position (called absolute pose) should be solved. The direct linear transformation method is firstly used to calculate the rough relative pose between adjacent frames, and principle of re-projection error minimization in object space is used to obtain the refined relative pose. Instead of using step-by-step transfer method, the absolute pose of the camera in each position is eventually calculated by using global optimization algorithm after the relative pose between all adjacent frames are obtained. Computer simulation and abundant experimental data validate the effectiveness of proposed method. The proposed 3-D reconstruction method using global optimization is respectively compared with step-by-step transfer method based on pixel re-projection error minimization and re-projection error minimization in object space. The experimental results reflect that our method is superior to step-by-step transfer method, and the 3-D reconstruction accuracy is improved by 28.05 % after global optimization.

A target spatial location method for fuze detonation point based on deep learning and sensor fusion

The spatial location of fuze detonation point is crucial for evaluating the working condition and improving the performance of fuze. Considering the observation safety, the non-contact long-distance accurate measurement technology is essential. In this paper, we propose a method that takes the sensor data of optics, spatial attitude and GPS as input and outputs the spatial position of the fuze detonation point. The proposed method consists of two steps. First, an object detection algorithm with post-processing algorithm is proposed to obtain rich information of the target. The algorithm achieves high-accuracy detection by introducing powerful backbone, attention mechanism, group convolution, and improved multi-scale feature fusion. Second, a Variational Auto-Encoder (VAE) algorithm model improved by dense connection structure and multiple source heterogeneous sensor information fusion structure is proposed as the position regression algorithm. It receives the status information of the observer camera and the output of the object detection algorithm, and then outputs the three-dimensional coordinates of the explosion point. Finally, method validation and performance analysis are realized through virtual scene simulation. Experiment results show the superiority of the proposed object detection algorithm over other typical algorithms on explosion flare detection, with its Average Precision (AP) of 0.889. The positioning error of the spatial location method is 0.896 m, while that of the binocular stereo vision method is 2.863 m. Therefore, the proposed target spatial location method is proved to be effective and accurate.

A Variable Photo-Model Method for Object Pose and Size Estimation with Stereo Vision in a Complex Home Scene.

Model-based stereo vision methods can estimate the 6D poses of rigid objects. They can help robots to achieve a target grip in complex home environments. This study presents a novel approach, called the variable photo-model method, to estimate the pose and size of an unknown object using a single photo of the same category. By employing a pre-trained You Only Look Once (YOLO) v4 weight for object detection and 2D model generation in the photo, the method converts the segmented 2D photo-model into 3D flat photo-models assuming different sizes and poses. Through perspective projection and model matching, the method finds the best match between the model and the actual object in the captured stereo images. The matching fitness function is optimized using a genetic algorithm (GA). Unlike data-driven approaches, this approach does not require multiple photos or pre-training time for single object pose recognition, making it more versatile. Indoor experiments demonstrate the effectiveness of the variable photo-model method in estimating the pose and size of the target objects within the same class. The findings of this study have practical implications for object detection prior to robotic grasping, particularly due to its ease of application and the limited data required.

Determination of the Live Weight of Farm Animals with Deep Learning and Semantic Segmentation Techniques

In cattle breeding, regularly taking the animals to the scale and recording their weight is important for both the performance of the enterprise and the health of the animals. This process, which must be carried out in businesses, is a difficult task. For this reason, it is often not performed regularly or not performed at all. In this study, we attempted to estimate the weights of cattle by using stereo vision and semantic segmentation methods used in the field of computer vision together. Images of 85 animals were taken from different angles with a stereo setup consisting of two identical cameras. The distances of the animals to the camera plane were calculated by stereo distance calculation, and the areas covered by the animals in the images were determined by semantic segmentation methods. Then, using all these data, different artificial neural network models were trained. As a result of the study, it was revealed that when stereo vision and semantic segmentation methods are used together, live animal weights can be predicted successfully.

Research on Real-time and High-precision Positioning Method of Ground Target through UAV Stereo Vision and Spatial Information Fusion

Background: Positioning accuracy is the most important index of the reconnaissance positioning system. Positioning accuracy involves many factors, such as the position, attitude and motion state of the flight platform, the pointing accuracy of the stable platform, and various coordinate transformations. A reasonable fusion strategy can guarantee stable positioning accuracy, so it is very important to study the precise fusion of machine vision, 3D geographic data and UAV airborne positioning information to realize the optimal combination of positioning data and complete the accurate and rapid positioning of the ground target. Methods: In this paper, a location model based on stereo vision and spatial information fusion method is proposed. It fully integrates visual information, satellite positioning information and spatial geographic information, greatly improves positioning accuracy, and through a real-time processing algorithm, significantly improves real-time positioning. Results: Through the related experiments, positioning accuracy and real-time ability of positioning could reach about three meters. Conclusion: The proposed real-time and high-precision positioning method of the ground target through UAV stereo vision and spatial information fusion is proposed showing significant improvement compared to other traditional methods.

Incorporation of motion prediction and state estimation to track dots for vision-based in-crash toeboard deformation measurement

This paper presents a marker-based stereovision method to measure toeboard deformation during a crash test. The method tracks a set of colored dots marked on the surface for stereovision using recursive state estimation that consists of a prediction step and a correction step. Prediction is based on a motion model derived by feature matching and mapping in the stereo image. In the correction step, the dots are first tracked using a dot centroid tracking (DCT) algorithm that weights the color, saturation, and brightness of pixels in a search kernel predicted by the motion model. An extended Kalman filter (EKF) subsequently fuses the prediction and the DCT measurement for the most likely dot centroid. The accuracy and robustness of the proposed method for in-crash toeboard deformation measurement are presented in a vehicle crash test. The experiment shows the proposed technique tracks the deformed surface accurately and is robust in varied light conditions and conditions when the field of view (FOV) is temporarily blocked.

BSV System with MSC and Its Application in the Measurement of Surface Cracks

In this paper, we have proposed a flexible noncontact crack‐size measurement method that can realize binocular stereo vision measurement with only a single camera. On the premise that the camera’s intrinsic parameters have been accurately calibrated, we use a camera to collect the image of the crack from two directions. Then, we calculate the motion parameters using the collected images from the camera in different positions. In addition, Canny algorithm is used to extract the edge pixels of crack images. Finally, we establish the binocular stereo vision model for crack measurement according to the camera parameters, the motion parameters, and the edge information of crack images. Thus, we can measure the crack size through this model. Experimental results show that the measurement error is less than 5% under a distance of 2 meters, which can effectively prove the precision of the proposed method. In addition, our method only uses a single camera. Compared with the traditional binocular stereo vision method, this method is not only flexible but also more economical.

Three-Dimensional Reconstruction of Secondary Arc Based on Improved Binocular Stereo Vision Method

The secondary arc’s 3D spatial morphology is significant for the study of its suppression technology. However, the brightness of secondary arc is high and the spatial morphology is complex. In addition, the haze-like plasma causes shielding and interruption on arc reconstruction. The reconstruction quality is seriously affected. Consequently, to obtain accurate physical parameters of secondary arc, a 3D reconstruction experimental platform based on binocular stereo vision was built. In view of the unique haze-like plasma occlusion and arc interruption phenomenon of secondary arc image, this paper used the dehazing algorithm based on the atmospheric scattering model to dehaze and enhance the arc image. A double-threshold recovery algorithm based on the grayscale of secondary arc image was used to repair intermittent arcs. Then, based on the optimized disparity map, the 3D image of secondary arc was reconstructed. Results show that there are spatial differences in the shape changes of secondary arc. Moreover, the variation in arc length and diameter is divided into two stages. Compared with the previous work, this paper realizes the 3D reconstruction of secondary arc, and the shape of the arc is more realistic, which can provide a reference for exploring the physical characteristics of secondary arc.

Vehicle Safe Distance Detection System Based On Image Processing As Accident Prevention With Faster R-CNN Method

Numerous victims and huge economic and social losses have resulted from the escalating number of traffic accidents. From these issues, a technique to create a camera capable of detecting vehicles going around the driver using the Faster R-CNN method and calculating the vehicle's distance using the Stereo Vision and Mono Vision methods was discovered. The determination of safe distance between these cars is determined by the speed of the driver's vehicle, with the LED and buzzer warning system activating when the parameters are met. Based on the results of object detection experiments utilizing the Faster R-CNN, the model's success rate in identifying and classifying objects had an average success rate of 83.33 percent across 35 object situations examined from different perspectives. The success rates for distance estimates utilizing the Stereo Vision and Mono Vision methods with the Linear Regression equation were 98.84% and 98.10%, respectively.

Identification of a Mathematical Model for the Transformation of Images for Stereo Correspondence Measurements of Mining Equipment

The stereometry of the working units of mining machines is optimized at the design stage, in terms of selected criteria based on computer simulations of the mining process. The recovered bodies of cutting heads or drums used in manufacturing are regenerated in the overhaul process. Ensuring that their dimensions comply with the nominal ones is labor-intensive and raises production costs. However, deviations of these components from the nominal shape can make it difficult to position the pick holders (which can cause collisions) or make welding them impossible (which results from too large a distance between the pick holders’ base and the side surface of the working unit). This applies especially to robotic technologies. By utilizing automatic (online) measurements of the distribution of the actual distances of the pick holders’ bases from the side surface of the working unit (taken during their positioning using a robot), it is possible to correct their positions without changing the setting of the pick itself. This measurement can be carried out using the non-contact stereovision method. This paper presents a method of matching raster images obtained from a stereovision system installed on an experimental robotic station. The results are presented of the numerical research carried out to select a mathematical model of the geometric transformation of images, for ensuring the effectiveness of the developed method.

A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

The human face is an important channel for human interaction and is the most expressive part of the human body with personalized and diverse characteristics. To improve the modeling speed as well as recognition speed and accuracy of 3D faces, this paper proposes a laser scanning binocular stereo vision imaging method based on binocular stereo vision and depth-sensing detection method. Existing matching methods have a weak anti-interference capability, cannot identify the spacing of objects quickly and efficiently, and have too large errors. Using the technique mentioned in this paper can remedy these shortcomings by mainly using the laser lines scanned onto the object as strong feature cues for left and right views for binocular vision matching and thus for depth measurement perception of the object. The experiments in this paper first explore the effect of parameter variation of the laser scanning binocular vision imaging system on the accuracy of object measurement depth values in an indoor environment; the experimental data are selected from 68 face feature points for modeling, cameras that photograph faces, stereo calibration of the cameras using calibration methods to obtain the parameters of the binocular vision imaging system, and after stereo correction of the left and right camera images, the laser line scan light is used as a pixel matching strong features for binocular camera matching, which can achieve fast recognition and high accuracy and then select the system parameters with the highest accuracy to perform 3D reconstruction experiments on actual target objects in the indoor environment to achieve faster recognition.

Hybrid calibration procedure for structured light field system

A typical structured light field system consists of a light field camera and a pattern projector. By combining Fringe Projection Profilometry and light field imaging technology, the phase distribution characteristics of each position and each angle of the scene under test could be obtained, thus providing more information dimensions, and therefore improving the robustness and dynamic range of three-dimensional (3D) measurement. System calibration is a decisive factor for high-efficiency and high precision measurement. However, the complicated structure and working method make the calibration of structured light field system usually costly and cumbersome. In this paper, we propose a hybrid calibration approach combining the flexibility of the stereo vision method and high precision characteristics of the phase-coordinate mapping method. The feasibility of the proposed method has been verified in actual experiments with typical 3D targets.

Stereovision-based method for free vibration measurement of a mine hoisting rope

Currently, there are no suitable means to measure the composite vibration response of mine hoisting rope. However, machine vision-based measurement technology is a potential way to solve this problem. Therefore, a non-contact and non-intrusive stereovision method is proposed to obtain the 2D vibration displacement of a moving hoisting rope. In this methodology, a novel 3D digital image correlation (DIC) method that combines digital image processing (DIP) algorithm with 2D-DIC algorithm is developed, and the method can simplify the procedure of rope target stereo matching in two synchronous image sequences. In order to confirm the correctness of the stereovision method, some free vibration responses of a rope are measured in a hoist experimental system. Test results demonstrate that the stereovision measurement technique can keep satisfactory consistency with the laser displacement sensor for vibration data identification. The method is verified to have high adaptability to different lighting conditions and be reasonable to measure the vibration parameters of dynamic mine hoisting rope.