Stereo Vision Research Articles

Content Analysis of Three-Dimensional Model Technologies and Applications for Construction: Current Trends and Future Directions.

The proliferation of digital technologies is substantially transforming inspection methodologies for construction activities. Although the implementation of a three-dimensional (3D) model has emerged as an advantageous, feasible inspection application, the selection of the most suitable 3D models is challenging due to multiple technology options. The primary objectives of this study were to investigate current trends and identify future technologies for 3D models in the construction industry. This study utilized systematic reviews by identifying and selecting quality journals, analyzing selected articles, and conducting content analysis and meta-analysis to identify dominant themes in 3D models. Results showed that the top technologies used to model construction projects are building information models, remote sensing, stereo vision system/photo processing programs, and augmented reality/virtual reality. The main benefits and challenges of these technologies for modeling were also determined. This study identified three areas with significant knowledge gaps for future research: (1) the amalgamation of two or more technologies to overcome project obstacles; (2) solution optimization for inspections in remote areas; and (3) the development of algorithm-based technologies. This research contributes to the body of knowledge by exploring current trends and future directions of 3D model technologies in the construction industry.

A Binocular Color Line-Scanning Stereo Vision System for Heavy Rail Surface Detection and Correction Method of Motion Distortion.

Thanks to the line-scanning camera, the measurement method based on line-scanning stereo vision has high optical accuracy, data transmission efficiency, and a wide field of vision. It is more suitable for continuous operation and high-speed transmission of industrial product detection sites. However, the one-dimensional imaging characteristics of the line-scanning camera cause motion distortion during image data acquisition, which directly affects the accuracy of detection. Effectively reducing the influence of motion distortion is the primary problem to ensure detection accuracy. To obtain the two-dimensional color image and three-dimensional contour data of the heavy rail surface at the same time, a binocular color line-scanning stereo vision system is designed to collect the heavy rail surface data combined with the bright field illumination of the symmetrical linear light source. Aiming at the image motion distortion caused by system installation error and collaborative acquisition frame rate mismatch, this paper uses the checkerboard target and two-step cubature Kalman filter algorithm to solve the nonlinear parameters in the motion distortion model, estimate the real motion, and correct the image information. The experiments show that the accuracy of the data contained in the image is improved by 57.3% after correction.

A stereoscopic video computer vision system for weed discrimination in rice field under both natural and controlled light conditions by machine learning

A site-specific weed detection and classification system was implemented with a stereoscopic video camera to reduce the adverse effects of chemical herbicides in rice field. A computer vision and meta-heuristic hybrid NN-ICA classifier were used to accurately discriminate between two weed varieties and rice plants, under either natural light (NLC) or controlled light conditions (CLC). Preprocessing, segmentation, and matching procedures were performed on images coming from either right or left camera channels. Most discriminant features were selected from average, either arithmetic or geometric, images using a NN-PSO algorithm. Accuracy classification results with the stereo computer vision system under NLC were 85.71 % for the arithmetic mean (AM) and 85.63 % for the geometric mean (GM), test set. At the same time, accuracy classification results of the computer vision system under CLC reached 96.95 % for the AM case and 94.74 % for the GM case, being consistently higher than those under NLC.

A lightweight detection method of pavement potholes based on binocular stereo vision and deep learning

The identification of pavement defects is crucial and must be done efficiently, accurately, and cost-effectively. A commercial-grade sports camera and mobile vehicle work together to detect pavement potholes in a lightweight manner. A dataset of 6,186 images with a resolution of 1500 by 1500 pixels has been constructed based on vehicle-mounted tilted images. Object detection and image segmentation are performed using a single-model MASK R-CNN, and the principle of binocular stereo vision is employed to reconstruct the three-dimensional (3D) structures of potholes. A novel methodology is introduced to calculate the 3D feature parameters of potholes, enabling the precise determination of the pavement pothole damage ratio. An average accuracy of 98% for detection and 94% for segmentation is achieved with the Mask R-CNN model. The proposed algorithms achieve an average DR calculation accuracy of 82%, facilitating precise identification of surface irregularities. The paper presents an intelligent approach for detecting pavement potholes.

Playing three-dimensional video games boosts stereo vision

Playing two-dimensional video games has been shown to result in improvements in a range of visual and cognitive tasks, and these improvements appear to generalize widely1,2,3,4,5,6. Here we report that young adults with healthy vision, surprisingly, showed a dramatic improvement in stereo vision after playing three-dimensional, but not two-dimensional, video games for a relatively short period of time. Intriguingly, neither group showed any significant improvement in binocular contrast sensitivity. This dissociation suggests that the visual enhancement was specific to genuine stereoscopic processing, not indirectly resulting from enhanced contrast processing, and required engaging in a disparity cue-rich three-dimensional environment.

P‐38: The Influence of Parallax and Shape Type Factors on the Perception of AR Equipment in Dark Environment

In this paper, the influence of parallax and image shape type on stereo vision perception is quantitatively explored by using RDS stimuli based on AR equipment through subjective scoring. Results show that an angular parallax amplitude between 0.6 ° and 1.2 ° can bring the best stereo sense. When amplitude ranged between 0 ° and 1.2°, the best visual comfort and the fastest image fusion speed could be achieved.

Modulation of prey capture kinematics in relation to prey distance helps predict success.

Predators are not perfect, as some of their prey capture attempts result in failure. Successful attempts may be partly due to predators modulating their capture kinematics in relation to variation in the visual cues of the prey to increase the probability of success. In praying mantises, which have been suggested to possess stereoscopic vision, variation in prey distance has been shown to elicit variation in the probability of an attempt. However, it remains to be examined whether variation in prey distance results in mantises modulating their attempt to successfully capture prey. The goals of this study were to examine these relationships using the praying mantis system. Using 11 adult female Sphodromantis lineola, we recorded 192 prey capture attempts at 1000 Hz with two cameras to examine the 3D kinematics of successful and unsuccessful prey capture attempts. Using a combination of principal component analysis (PCA) and logistic regression, our results show that as prey distance increases, mantises adjust through greater and faster expansion of the forelegs and body (PC1), which significantly predicts capture success. However, PC1 only explains 22% of the variation in all prey capture attempts, suggesting that the other components may be related to additional aspects of the prey. Our results suggest that the distances at which mantises prefer to attempt to capture prey may be the result of their greater probability of successfully capturing the prey. These results highlight the range of motions mantises use when attempting to capture prey, suggesting flexibility in their prey capture attempts in relation to prey position.

Visual training for the treatment of retinal photoreceptor inner and outer segment connection (IS/OS) fracture: A case report.

The inner segments and outer segments (IS/OS) of the retinal photoreceptors are the areas that receive light signals and are the most initial sites for generating visual impulses, and the integrity of the IS/OS has a direct impact on visual sensitivity. We performed OCT on a 6-year-old child with vision loss and found that the cause of his vision loss was a retinal IS/OS fracture, and the child underwent some treatments to improve microcirculation and nourish the retina at a higher-level hospital, but his vision never improved. Our examination of this child revealed that this child not only had decreased visual acuity, but also hypermetropia, but his near stereopsis was normal. The symptoms were similar to those of amblyopia, so we tried to use visual training as a treatment. First, 6 sessions of fine visual stimulation were given, followed by 3 sessions of accommodation training, and we followed the 4 stages of accommodation training: perception of accommodation, amplitude of accommodation, sensitivity of accommodation, and autonomic accommodation. After 9 consecutive visual training sessions, the child's visual acuity was stabilized at 0.6, and then we added eye movement training, and after the child's visual acuity was improved to 0.7, we suppressed the visual acuity of the left eye to 0.6, so as to make the visual acuity of both eyes similar, which would promote the establishment of binocular stereo vision, and then we carried out 9 more visual training sessions, and the patient's visual acuity was stabilized at 0.8 gradually. OCT review showed that the child's retinal IS/OS fracture was basically closed. In conclusion, our study found that visual training can restore visual acuity in children with monocular IS/OS fracture and also promote repair of IS/OS fracture, which increases our understanding and knowledge of the treatment of retinal IS/OS fracture, and this case may provide some lessons for the treatment of retinal IS/OS fracture in children. We hope to have more samples of retinal IS/OS fracture in the future to evaluate the efficacy of visual training for retinal IS/OS fracture.

A low-cost, open-source-based optical surgical navigation system using stereoscopic vision

AbstractComputer-assisted surgical navigation systems have gained popularity in surgical procedures that demand high amounts of precision. These systems aim to track the real-time positioning of surgical instruments in relation to anatomical structures. Typically, state-of-the-art methods involve tracking reflective 3D marker spheres affixed to both surgical instruments and patient anatomies with infrared cameras. However, these setups are expensive and financially impractical for small healthcare facilities. This study suggests that a fully optical navigation approach utilizing low-cost, off-the-shelf parts may become a viable alternative. We develop a stereoscopic camera setup, costing around $120, to track and monitor the translational movement of open-source based fiducial markers on a positioning platform. We evaluate the camera setup based on its reliability and accuracy. Using the optimal set of parameters, we were able to produce a root mean square error of 2 mm. These results demonstrate the feasibility of real-time, cost-effective surgical navigation using off-the-shelf optical cameras.

Design and development of an occlusion-free surface profile measurement device

The prevalent surface profile measurement techniques in the sub-millimetre measurement scale, based on active triangulation (such as laser scanners), stereo vision and photometric stereo, may miss certain parts of the surface of the object due to occlusion. Occlusion occurs in these techniques as they rely on a top-view configuration, where the illumination rays and the surface profile lie on the same plane. In this study, a surface profile measurement device is designed and developed, which can generate occlusion-free measurements by adopting a side-view configuration. This proposed measurement technique is tested on a few surfaces, and the results are presented.

Robust distance measurement using illumination map estimation and MAHNet in underground coal mines

An effective binocular stereo distance measurement method is proposed to address challenges posed by low brightness and weak texture of images captured in underground coal mines for the machine vision method. This approach is based on illumination map estimation and the MobileNetV3 attention hourglass stereo matching network (MAHNet) model. First, a binocular stereo vision system is established in which infrared LEDs are uniformly distributed on both sides of the belt conveyor bracket as visual feature points. Second, images are preprocessed using illumination map estimation, and the optimization of inhomogeneous brightness image enhancement is achieved by adopting adaptive Gamma correction. Third, the YOLOv5 target detection network and Gaussian fitting fusion algorithm are utilized to detect infrared LED feature points. Fourth, the MAHNet model is employed to generate the cost volume and perform disparity regression, resulting in the acquisition of accurate disparity images. Finally, triangulation is applied to determine the depth of feature points. The experimental results of distance measurement demonstrate that an average relative ranging accuracy of 1.52% within the range of 50.0 cm to 250.0 cm can be achieved by the optimized method, thereby validating the effectiveness of this binocular distance measurement method in underground coal mines.

A novel data-driven algorithm for object detection, tracking, distance estimation, and size measurement in stereo vision systems

A novel data-driven algorithm for object detection, tracking, distance estimation, and size measurement in stereo vision systems

COMPUTER VISION TECHNOLOGIES FOR DETECTING DEFECTS IN CYLINDRICAL OBJECTS

Currently, the development of computer vision makes it possible to solve many problems of detecting defects at various industrial facilities. One of the promising areas of application of these technologies is to identify inconsistencies in geometric parameters on cylindrical products. The purpose of this work is to review and systematize modern computer vision methods used to solve the problem of detecting defects on vertical cylindrical surfaces. The study analyzed existing approaches to the extraction of spatial characteristics of objects, including methods of stereo vision, spatial filtering and 3D reconstruction. Algorithms for identifying the main landmarks on a cylindrical surface were considered, which makes it possible to bind the coordinate system and localize areas of possible defects. Methods for estimating geometric deviations on the surface, which can act as criteria for detecting defects, have also been studied. As a result of the analysis, a classification of computer vision methods applicable to the problem of detecting defects on cylindrical objects was proposed. Promising directions for further research in the field of improving the accuracy of defect detection through a combination of various image processing algorithms have been identified.

Highly accurate three-dimensional measurement of large structures using multiple stereo vision with improved two-step calibration algorithm

A multiple stereo-based three-dimensional (3D) measurement method of large-scale structures is presented, which replaces time-consuming and laborious 3D measurement procedure of laser-based conventional equipment. To have 3D reconstruction accuracy with millimeter-precision, two-step calibration algorithms are improved. Specifically, we use 2D epipolar constraint, using fundamental matrix determined by matched points, to new combination of objective function for intrinsic parameter calibration. We additionally propose an onsite structure parameter calibration algorithm using different epipolar constraint of measurement points in images of large ship block as well as reprojection error of checkerboard, to bypass the limitation of the coverage of checkerboard in large-scale measurement. The actual 3D measurement of the three ship blocks is conducted, and the 3D reconstruction accuracy is validated by comparing the 3D coordinates measured by total station. The experimental results showed that the mean 3D reconstruction error ranged from 6 to 14 mm.

AI-Driven Dim-Light Adaptive Camera (DimCam) for Lunar Robots

Abstract. The past decade has been a boom in lunar exploration. China, India, Japan and other countries have successfully landed landers or rovers on the lunar surface (Wu et al., 2014, 2018, 2020; Prasad et al., 2023). Future missions to explore the Moon are focusing on the lunar south pole (Peña-Asensio et al., 2024). The solar altitude angle at the lunar south pole is extremely low, resulting in low solar irradiance and large areas often in dim light or shadows. The permanently shadowed regions (PSRs) at the lunar south pole are also likely containing substantial amounts of water ice (Li et al., 2018). Future lunar robots exploring the lunar south pole will need to operate in low light or shadowed regions, making camera sensors sensitive to the dim-light environments necessary for these robots. Common night vision sensors usually use near-infrared cameras. However, sensors based on passive infrared technology have image resolution limited by several factors, including the intensity of infrared radiation emitted by the object, the sensitivity of the camera, and the performance of the optical system. For instance, thermal imagers typically have a resolution of 388*284 pixels only. We have developed a dim-light adaptive camera (DimCam) that is ultra-sensitive to the varying illumination conditions driven by AI to achieve high-definition imaging of 1080P or above, for future lunar robots operating in shadows or dim-light regions. The DimCam integrates two starlight-level ultra-sensitive imaging sensors connected by a rigid base to provide stereo vision in low illumination environment. An AI edge computing unit is embedded inside the DimCam to adaptively denoise and enhance image quality. The AI module uses an end-to-end image denoising network to identify and remove noises in the images more accurately by utilizing depth information from the stereo sensors. Compared with traditional monocular denoising algorithms, the denoising network based on stereo vision can significantly improve denoising effects and efficiency by enhancing the signal-to-noise ratio of the data input in the front end. The superposition of overlapping scenes can be regarded as a delayed exposure. Concurrently, the residual analysis of the aligned images aids in noise identification. In addition, for pixels obscured by noise, more accurate pixel values can be restored through interpolation or replacement using depth information obtained from the stereo sensors. Subsequently, a pre-trained lightweight deep network modified from Zero-DCE (Guo et al., 2020) is used for image quality enhancement in terms of brightness and contrast, providing high-quality images even in low-light environments for subsequent applications, such as positioning and navigation of robots, 3D mapping of the surrounding environment, and autonomous driven. We have tested the DimCam in a simulated environment in the laboratory, and the results show that the DimCam has promising performances and great potential for various applications.

Multifunctional nanorobot system in precise evaluation and manipulation of virus capsid

It is imperative to have high adaptive techniques for sensing and manipulating biological targets at the nanoscale. This necessity becomes particularly crucial when dealing with fragile living bio-organisms like viruses, where the expression of capsids is closely linked to viral functions and genome constitution. Therefore, the development of a comprehensive system for dissecting and measuring viruses holds significant implications for the pharmaceutical industry and drug manufacturing. Leveraging the sub-nanometer spatial resolution and controllable tip-cantilever architecture of atomic force microscopy (AFM), a probe-laser system has been integrated as a self-sensing robotic end effector. To address intrinsic challenges in AFM-based robotic systems such as the lack of real-time monitoring, low scanning rates, and nonlinear motion caused by piezoelectric actuators, an augmented reality robotic system has been implemented. This system incorporates stereoscopic vision, a haptic feedback controller, a position recovery scheme, and a real-time force control algorithm. The integration of these components enhances the system’s capability to accurately dissect virus capsids. Operators can now perform highly efficient nanoscale tasks with multidimensional perception, utilizing the combination of stereoscopic vision and haptic force control. The position correction during manipulation can achieve a frame rate of over 30 frames per second, imperceptible to the operator, enabling closed-loop operation control. By adopting the proposed nanorobotic system in virology studies, it becomes possible to achieve accurate manipulation and dissection of SARS-CoV-2 pseudovirus capsids, and derive multi-parametric properties such as structural integrity, protein fragment thickness, and adhesive forces. The established nanobot system and experimental results serve as a guiding platform for high-accuracy evaluation in drug manufacturing development.

Estimation of rock joint roughness using binocular stereo vision

To quickly and cost-effectively evaluate the joint roughness coefficient (JRC) of the rock joints, a technique based on binocular stereo vision was developed to reconstruct the three-dimensional (3D) model. After camera calibration, only two images of rock joints were needed to be captured for 3D reconstruction. Then, the generated point clouds were processed to extract the geometric information, mainly focusing on the JRC calculation. A comparison was made with the point clouds obtained from laser scanning to verify the proposed method, and the possibility of the method in large-scale modeling was also explored. The results show that the proposed technique for 3D reconstruction of rock joints can meet the basic accuracy requirements in laboratory work, and it has a good consistency in roughness estimation with laser scanning. Compared to traditional photogrammetry, this method significantly simplifies the working procedure to improve measurement efficiency and reduce the project cost.

Non-Contact Tilapia Mass Estimation Method Based on Underwater Binocular Vision

The non-destructive measurement of fish is an important link in intelligent aquaculture, and realizing the accurate estimation of fish mass is the key to the stable operation of this link. Taking tilapia as the object, this study proposes an underwater tilapia mass estimation method, which can accurately estimate the mass of free-swimming tilapia under non-contact conditions. First, image enhancement is performed on the original image, and the depth image is obtained by correcting and stereo matching the enhanced image using binocular stereo vision technology. And the fish body is segmented by an SAM model. Then, the segmented fish body is labeled with key points, thus realizing the 3D reconstruction of tilapia. Five mass estimation models are established based on the relationship between the body length and the mass of tilapia, so as to realize the mass estimation of tilapia. The results showed that the average relative errors of the method models were 5.34%~7.25%. The coefficient of determination of the final tilapia mass estimation with manual measurement was 0.99, and the average relative error was 5.90%. The improvement over existing deep learning methods is about 1.54%. This study will provide key technical support for the non-destructive measurement of tilapia, which is of great significance to the information management of aquaculture, the assessment of fish growth condition, and baiting control.

An empirical study of virtual museum based on dual-mode mixed visualization: the Sanxingdui bronzes

The design and implementation of virtual museums have evolved over decades, broadening the scope of traditional museum exhibitions and serving as an effective complement to physical museums. However, virtual visual effects have consistently fallen short of replicating the authentic exhibition experience and cannot substitute for physical museums. In this paper, we report a dual-mode mixed visualization system combining computational photography and binocular stereo vision techniques, and introduce an innovative virtual museum for Sanxingdui bronzes. This approach enables visitors to engage with the artifacts in a multifaceted manner-allowing for navigation, interaction, auditory guidance, and the examination of textural details as if handling the artifacts directly. Further analysis of participant responses in terms of visual cognition and aesthetic appreciation across different settings demonstrates that this dual-mode mixed visualization delivers not only a spatiotemporal experience consistent with the real world but also surreal visual effects. This immersive experience allows audiences to experience Sanxingdui art intimately, offering both high-resolution visuals and cost-effectiveness.

Measurement Of Spatial-Temporal Waves In The Laboratory Using Computer Vision Technolog

Given the complexity of ocean wave phenomena, physical modeling continues to be essential in various engineering and research applications. Wave height, a characteristic measured in nearly all projects, is commonly assessed using resistance/capacitance wave gauges. However, a significant limitation of these traditional wave gauges is their point-based nature, which restricts their ability to describe detailed spatial wave characteristics, especially during wave nonlinearity or when waveforms change. Furthermore, to reconstruct the 3D wave field, an array of wave gauges is often employed alongside an appropriate interpolation algorithm. Additionally, wave gauges may alter flow/wave conditions in specific scenarios. To address these limitations, various non-intrusive measurement techniques have been proposed. Gomit et al. (2022) classified these into stereoscopic, projection, and light-based methods, including LiDAR (Blenkinsopp et al., 2012), RGB-D cameras (Martínez-Aranda et al., 2018), and binocular stereo vision (Li et al., 2022). However, their applicability is constrained by factors such as optical setup complexity and device cost. With the rapid advancement of computer vision technology, it is possible to improve these issues and better describe temporal and spatial wave field variations using such techniques. This conference contribution details the reconstruction of the temporal and spatial 3D free water surface using photogrammetry techniques applied to images from consumer webcams. It also compares these results with measurements obtained from a wave gauge.