Stereovision Images Research Articles

Adaptable 2D to 3D Stereo Vision Image Conversion Based on a Deep Convolutional Neural Network and Fast Inpaint Algorithm.

Algorithms for converting 2D to 3D are gaining importance following the hiatus brought about by the discontinuation of 3D TV production; this is due to the high availability and popularity of virtual reality systems that use stereo vision. In this paper, several depth image-based rendering (DIBR) approaches using state-of-the-art single-frame depth generation neural networks and inpaint algorithms are proposed and validated, including a novel very fast inpaint (FAST). FAST significantly exceeds the speed of currently used inpaint algorithms by reducing computational complexity, without degrading the quality of the resulting image. The role of the inpaint algorithm is to fill in missing pixels in the stereo pair estimated by DIBR. Missing estimated pixels appear at the boundaries of areas that differ significantly in their estimated distance from the observer. In addition, we propose parameterizing DIBR using a singular, easy-to-interpret adaptable parameter that can be adjusted online according to the preferences of the user who views the visualization. This single parameter governs both the camera parameters and the maximum binocular disparity. The proposed solutions are also compared with a fully automatic 2D to 3D mapping solution. The algorithm proposed in this work, which features intuitive disparity steering, the foundational deep neural network MiDaS, and the FAST inpaint algorithm, received considerable acclaim from evaluators. The mean absolute error of the proposed solution does not contain statistically significant differences from state-of-the-art approaches like Deep3D and other DIBR-based approaches using different inpaint functions. Since both the source codes and the generated videos are available for download, all experiments can be reproduced, and one can apply our algorithm to any selected video or single image to convert it.

Research on calibration of a binocular stereo-vision imaging system based on the artificial neural network.

Camera calibration is a key problem for 3D reconstruction in computer vision. Existing calibration methods, such as traditional, active, and self-calibration, all need to solve the internal and external parameters of the imaging system to clarify the image-object mapping relationship. The artificial neural network, which is based on connectionist architecture, provides a novel idea for the calibration of nonlinear mapping vision systems. It can learn the image-object mapping relationship from some sample points without considering too many uncertain factors in the middle. This paper discusses the learning ability. A binocular stereo-vision mapping model is used as the learning model to explore the ability of image-object mapping for artificial neural networks. This paper constructs sample libraries by pixel and world coordinates of checkerboard corners, builds the artificial neural network, and, through the training samples and test samples prediction, verifies the learning performance of the network. Furthermore, by the laser scanning binocular vision device constructed in the authors' laboratory and trained-well network, the 3D point cloud reconstruction of a physical target is performed. The experimental results show that the artificial neural network can learn the image-object mapping relationship well and more effectively avoid the impact of lens distortion and achieve more accurate nonlinear mapping at the edge of the image. When the X and Y coordinates are in the range of 100mm and the Z coordinates are in the range of a 1000mm, the absolute error rarely exceeds 2.5mm, and the relative error is in the level of 10-3; for 1000mm distance measurement, the standard deviation does not exceed 1.5mm. Network parameter selection experiments show that, for image-object mapping, a three-layer network and increasing the number of hidden layer's nodes can improve the training time more significantly.

Spatial-temporal measurement of waves in laboratory based on binocular stereo vision and image processing

Acquisition of wave field data is essential for hydrodynamics studies. Current studies show deficiencies in the spatial measurement of wave fields in the laboratory, and a more efficient method is urgently needed. A stereo imaging method was proposed for the spatial-temporal measurement of waves in the laboratory based on binocular stereo vision and digital image processing. Then, a series of regular and focused waves were simulated in a wave flume to examine the feasibility and reliability of the method by comparison with the measurements by wave probes. Floating particles were evenly sprinkled on the water surface as the representation of the water-air interface, providing texture features for wave surface images and allowing the realization of dense reconstruction. Synchronized cameras provided the synchronized stereo pairs sequence for each wave, and temporal reconstruction was achieved by processing frame by frame. The 4D spatial-temporal volume for every wave could be constructed. Subsequently, spatial slices of the volume obtained the instantaneous profiles of the wave field in different directions. Temporal slices obtained the wave time series at a fixed point, which were compared with the measurements of the standard resistive probe to verify the accuracy. The results showed that the proposed method was capable of measuring the spatial-temporal field of waves efficiently and precisely and offered the potential to measure more complex wave fields in nonlinear wave-structure interactions.

Design of laser scanning binocular stereo vision imaging system and target measurement

Binocular stereo vision is an important scheme of road environment perception for intelligent driving, because it can provide image information and spatial three-dimensional position information. However, binocular stereo matching is often disturbed by the environment, and noise, shadow, light transformation, and so on, will all cause mismatching, therefore it is impossible to carry out accurate 3D reconstruction. In this paper, an active imaging perception and target measurement method combining laser scanning and binocular vision is proposed. The binocular vision structure is used to simulate the human eye's observation of the environment in front of the vehicle. At the same time, through the laser lines scanned to the target, the strong characteristics of binocular matching are provided to enhance the anti-interference of binocular matching, and calculate the three-dimensional point cloud data of the front object according to the binocular stereo vision theory. Firstly, this paper studies the influence of the parameters changing of the system on the accuracy of object depth value in indoor environment, and then selects the most suitable system parameters to carry out three-dimensional reconstruction experiment on the actual target object in indoor environment. Finally, the scene experiment in outdoor environment is carried out. In the outdoor environment, the red spectrum laser equipped in the system can still effectively reconstruct the target surface radian, but there are some limitations due to the limitation of outdoor light intensity. This paper preliminarily summarizes the current structural design and experiment, and further discusses the follow-up research and improvement ideas. Research and experiments show that the laser scanning binocular stereo vision imaging system proposed in this paper has the function of forward target ranging, especially under some bad imaging environment, this method can also effectively match the binocular images, and present the three-dimensional information of the internal details of the target. Under the indoor experimental environment, the minimum relative error is 1.42 % for the target with a distance of about 1.5 m, and in the evening outdoor environment, the minimum the minimum relative error is 0.87 % for the target with a distance of about 3 m. Due to the advantages of laser line intensity feature matching, it can accelerate the speed of binocular matching, and has potential application prospects in autonomous system.

A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

The human face is an important channel for human interaction and is the most expressive part of the human body with personalized and diverse characteristics. To improve the modeling speed as well as recognition speed and accuracy of 3D faces, this paper proposes a laser scanning binocular stereo vision imaging method based on binocular stereo vision and depth-sensing detection method. Existing matching methods have a weak anti-interference capability, cannot identify the spacing of objects quickly and efficiently, and have too large errors. Using the technique mentioned in this paper can remedy these shortcomings by mainly using the laser lines scanned onto the object as strong feature cues for left and right views for binocular vision matching and thus for depth measurement perception of the object. The experiments in this paper first explore the effect of parameter variation of the laser scanning binocular vision imaging system on the accuracy of object measurement depth values in an indoor environment; the experimental data are selected from 68 face feature points for modeling, cameras that photograph faces, stereo calibration of the cameras using calibration methods to obtain the parameters of the binocular vision imaging system, and after stereo correction of the left and right camera images, the laser line scan light is used as a pixel matching strong features for binocular camera matching, which can achieve fast recognition and high accuracy and then select the system parameters with the highest accuracy to perform 3D reconstruction experiments on actual target objects in the indoor environment to achieve faster recognition.

Enhancement of Digital Stereo Vision Images based on Histogram and Gamma Correction Strategy

Image contrast enhancement methods have been a topic of interest in digital image processing for various applications like satellite imaging, recognition, medical imaging, and stereo vision. This paper studies the technique for image enhancement utilizing Adaptive Histogram Equalization and Weighted Gamma Correction to cater radiometric condition and illumination variations of stereo image pairs. In the proposed method, the stereo pair images are segmented together with weighted distribution into sub-histograms supported with Histogram Equalization (HE) mapping or gamma correction and guided filtering. The experimental result shows the experimented techniques outperform compare with the original image in evaluation parameters of visual quality and image enhancement parameters via Entropy, MSE, and PSNR, etc. taken for performance analysis and validation.

Underwater Modeling and Calibration Method of Binocular Stereo Vision Imaging System

Underwater Modeling and Calibration Method of Binocular Stereo Vision Imaging System

Analysis of conical slump shape reconstructed from stereovision images for yield stress prediction

Slump cones are widely utilised to evaluate the workability of cement materials owing to their simplicity. Analytical models have been established to relate this empirical value to the yield stress. However, they tend to be less effective for the cone and more valid for cylinder slumps. The cause of this deviation is yet to be fully understood. In this work, more details were sought from slump profiles using stereovision technology. A direct comparison of the predicted shapes based on conical model as well as cylindrical model to actual shapes was undertaken. The visual illustration clearly indicated that conical slumping shows a cylinder-like behaviour, and the slumped shapes in cone tests agree well with those predicted by an equivalent cylinder. By converting to cylindrical dimensionless units, the conical slump height can provide an accurate yield stress. The results suggest that the theory is valid if the slumping behaviour is effectively assumed.

Coupled deformation behavior analysis for the glass panel in unitized hidden-frame supported glass curtain wall system

As a kind of building envelope, glass panel has been emerging and widely employed in unitized hidden-frame supported (UHFS) glass curtain wall system in modern construction engineering. In practical service conditions, the large area of glass panels involved in the all-glass facades of UHFS glass curtain wall system usually expose to external wind loads, thus it is quite essential to focus on the coupled deformation of the glass panel caused by complex boundary constraints stemming from connected components and their interactions in UHFS glass curtain wall system. In this paper, the optical measurement system associated with the standard air-tight test chamber of UHFS glass curtain wall samples for DIC analysis of the glass panel was first established based on the theory of stereo vision and digital image correlation. Then, the conventional digital indicators and finite element analysis (FEA) models were employed to provide the actual and theoretical deformation information of the panel, which were used to demonstrate that the adopted DIC method had the unique advantage of characterizing full-field coupled deformation behavior of the glass panel affected by complicated boundary conditions. Consequently, the spatiotemporal deformation behavior of the glass panel in practical service conditions was investigated and thus the principal strain, maximum relative deflection (MRD) as well as strain energy density on the panel surface were accordingly evaluated. The above metrics could be used for failure risk evaluation and providing safety design suggestions for the glass panel in UHFS curtain wall system.

Camera Imaging Calibration Optimization for Stereo Vision System of Marine Unmanned Ship

Zheng, X.; Wang, X.; Luo, Z., and Guo, B., 2020. Camera imaging calibration optimization for stereo vision system of marine unmanned ship. In: Liu, X. and Zhao, L. (eds.), Today's Modern Coastal Society: Technical and Sociological Aspects of Coastal Research. Journal of Coastal Research, Special Issue No. 111, pp. 288–292. Coconut Creek (Florida), ISSN 0749-0208.As a depth perception technology, stereo vision is widely used in the intelligent operation of marine unmanned ships. Focusing on the camera imaging problem of the stereo vision system of marine unmanned ship, the causes of camera imaging distortion were analyzed and an improved adaptive genetic simulated annealing algorithm based on Sigmoid function is proposed to improve the accuracy of camera calibration. Experiments have shown that this method has higher accuracy and stronger robustness and can be used to optimize the stereo vision imaging system of marine unmanned ships.

Estimation of Body Weight for Korean Cattle Using Three-Dimensional Image

This study was conducted to estimate the body weight of Korean cattle (Hanwoo) using three-dimensional image. A time-of-flight camera and a stereo vision camera were used to acquire top-view images for Hanwoo. Multiple linear regression model incorporating body dimensions and age was developed to estimate the body weight. The prediction models with or without age variable were determined. Incorporating age variable that correlates with body weight improved the coefficient of determination for both TOF images and stereo vision images. The weight estimation error was greatly reduced under the condition that calves less than 6 months old were excluded. It was concluded that 3-D image is a promising technique for estimating the body weight of Hanwoo cattle.

Hand-Held Stereovision System for Image Updating in Open Spine Surgery.

Image guidance in open spinal surgery is compromised by changes in spinal alignment between preoperative images and surgical positioning. We evaluated registration of stereo-views of the surgical field to compensate for vertebral alignment changes. To assess accuracy and efficiency of an optically tracked hand-held stereovision (HHS) system to acquire images of the exposed spine during surgery. Standard midline posterior approach exposed L1 to L6 in 6 cadaver porcine spines. Fiducial markers were placed on each vertebra as "ground truth" locations. Spines were positioned supine with accentuated lordosis, and preoperative computed tomography (pCT) was acquired. Spines were re-positioned in a neutral prone posture, and locations of fiducials were acquired with a tracked stylus. Intraoperative stereovision (iSV) images were acquired and 3-dimensional (3D) surfaces of the exposed spine were reconstructed. HHS accuracy was assessed in terms of distances between reconstructed fiducial marker locations and their tracked counterparts. Level-wise registrations aligned pCT with iSV to account for changes in spine posture. Accuracy of updated computed tomography (uCT) was assessed using fiducial markers and other landmarks. Acquisition time for each image pair was<1s. Mean reconstruction time was <1 s for each image pair using batch processing, and mean accuracy was 1.2±0.6 mm across 6 cases. Mean errors of uCT were 3.1±0.7 and 2.0±0.5 mm on the dorsal and ventral sides, respectively. Results suggest that a portable HHS system offers potential to acquire accurate image data from the surgical field to facilitate surgical navigation during open spine surgery.

Automated Asian Fruit Grading System Using Stereo Vision Technique

In Malaysia, most of fruit planters are still using the conventional methods of inspection and grading their products by manual weightage measurement and visual inspection. Conventional methods rely on human eyes to perform the grading which leads to unproductive system and lack of data management. Therefore, in this study, an automated weight grading system is proposed to perform the grading system via size measurement using stereo vision system and image processing. The fruit selected to verify the system proposed performance were fruit samples provided by MARDI Bukit Tangga located in Kedah, Malaysia. In the vision system, triangulation method is used where the samples were graded more accurately using two cameras to form a stereo vision system. The image capture through the vision system processed using multiple image processing steps such as image segmentation, sobel edge sensor and multiple morphological operation. The relationship between weight and area of fruits done via regression methods with adjusted R-square of 0.9823. Using this goodness of fit, weight estimation can be executed with Root Mean Squared Error (RMSE) of 6.811. Overall, fruit grading system with the stereo vision of a low-cost web camera implementation has successfully performed with the accuracy of 85.65% and a Mean Absolute Percentage Error (MAPE) of 6.551%.

Advantages of using digital image correlation techniques in uniaxial compression tests

This paper discusses the advantages of using stereo vision imaging along with Digital Image Correlation techniques as an alternate data acquisition procedure in uniaxial compression testing of specimens that may include undetected internal defects before testing. The discussed techniques were implemented during an extensive experimental investigation at the University of South Carolina related to concrete material characterization in order to investigate inconsistencies in test measurements acquired through conventional instrumentation. The use of the proposed system in compression tests of concrete and rock specimens facilitated the accurate and efficient measurement of the modulus of elasticity of the materials based on full field strain measurements. Most importantly, however, the proposed system facilitated the assessment of the quality of the specimen and measurements, as these may be affected by internal material defects that do not show any evidence on the surface of the specimen before any testing takes place, or the deviation of the shape and size of the specimen from the specifications.

Single-shot multi-view imaging enabled by scattering lens.

Imaging three-dimensional (3D) objects has been realized by methods such as binocular stereo vision and multi-view imaging. These methods, however, needs multiple cameras or multiple shots to get elemental images. In this paper, we develop a single-shot multi-view imaging technique by utilizing the natural randomness of scattering media. By exploiting the memory effect and uncorrelated point spread functions (PSF) among scattering media, we demonstrate that both stereo imaging with large disparity and up to seven-view imaging of a 3D object can be reconstructed from only one speckle pattern by deconvolution. The elemental images are consistent with 3D object projection and images taken by multi-shot imaging. Our technique provides a feasible method to capture multi-view imaging with short acquisition time and easy calibration.

Binocular Light-Field: Imaging Theory and Occlusion-Robust Depth Perception Application.

Binocular stereo vision (SV) has been widely used to reconstruct the depth information, but it is quite vulnerable to scenes with strong occlusions. As an emerging computational photography technology, light-field (LF) imaging brings about a novel solution to passive depth perception by recording multiple angular views in a single exposure. In this paper, we explore binocular SV and LF imaging to form the binocular-LF imaging system. An imaging theory is derived by modeling the imaging process and analyzing disparity properties based on the geometrical optics theory. Then an accurate occlusion-robust depth estimation algorithm is proposed by exploiting multibaseline stereo matching cues and defocus cues. The occlusions caused by binocular SV and LF imaging are detected and handled to eliminate the matching ambiguities and outliers. Finally, we develop a binocular-LF database and capture realworld scenes by our binocular-LF system to test the accuracy and robustness. The experimental results demonstrate that the proposed algorithm definitely recovers high quality depth maps with smooth surfaces and precise geometric shapes, which tackles the drawbacks of binocular SV and LF imaging simultaneously.

Image Updating for Brain Shift Compensation During Resection.

In open-cranial neurosurgery, preoperative magnetic resonance (pMR) images are typically coregistered for intraoperative guidance. Their accuracy can be significantly degraded by intraoperative brain deformation, especially when resection is involved. To produce model updated MR (uMR) images to compensate for brain shift that occurred during resection, and evaluate the performance of the image-updating process in terms of accuracy and computational efficiency. In 14 resection cases, intraoperative stereovision image pairs were acquired after dural opening and during resection to generate displacement maps of the surgical field. These data were assimilated by a biomechanical model to create uMR volumes of the evolving surgical field. A tracked stylus provided independent measurements of feature locations to quantify target registration errors (TREs) in the original coregistered pMR and uMR as surgery progressed. Updated MR TREs were 1.66 ± 0.27 and 1.92 ± 0.49 mm in the 14 cases after dural opening and after partial resection, respectively, compared to 8.48 ± 3.74 and 8.77 ± 4.61 mm for pMR, respectively. The overall computational time for generating uMRs after partial resection was less than 10 min. We have developed an image-updating system to compensate for brain deformation during resection using a computational model with data assimilation of displacements measured with intraoperative stereovision imaging that maintains TREs less than 2 mm on average.

Study of drag and orientation of regular particles using stereo vision, Schlieren photography and digital image processing

A new experimental, image-based methodology suitable to track the changes in orientation of non-spherical particles and their influence on the drag coefficient as they settle in fluids is presented. Given the fact that non-spherical solids naturally develop variations in their angular orientation during the fall, none-intrusiveness of the technique of analysis is of paramount importance in order to preserve the particle/fluid interaction undisturbed. Three-dimensional quantitative data about the motion parameters is obtained through single-camera stereo vision whilst qualitative visualizations of the adjacent fluid patterns are achieved with Schlieren photography. The methodology was validated by comparing the magnitudes of the drag coefficient of a set of spherical particles at terminal velocity conditions against those estimated from drag correlations published in the literature. A noteworthy similarity was attained. During the fall of non-spherical solids, once the particle Reynolds number approximated 163 for disks, and 240 for cylinders, or exceeded those values, secondary motions composed by regular oscillations and tumbling were present. They altered the angular orientation of the particles with respect to the main motion direction and caused complete turbulent patterns in the surrounding flow, therefore affecting the instantaneous projected area, drag force, and coefficient of resistance. The impact of the changes in angular orientation onto the drag coefficient was shown graphically as a means for reinforcing existing numerical approaches, however, an explicit relation between both variables could not be observed.

Non-destructive Phenotypic Analysis of Early Stage Tree Seedling Growth Using an Automated Stereovision Imaging Method.

A plant phenotyping approach was applied to evaluate growth rate of containerized tree seedlings during the precultivation phase following seed germination. A simple and affordable stereo optical system was used to collect stereoscopic red–green–blue (RGB) images of seedlings at regular intervals of time. Comparative analysis of these images by means of a newly developed software enabled us to calculate (a) the increments of seedlings height and (b) the percentage greenness of seedling leaves. Comparison of these parameters with destructive biomass measurements showed that the height traits can be used to estimate seedling growth for needle-leaved plant species whereas the greenness trait can be used for broad-leaved plant species. Despite the need to adjust for plant type, growth stage and light conditions this new, cheap, rapid, and sustainable phenotyping approach can be used to study large-scale phenome variations due to genome variability and interaction with environmental factors.

Intraoperative image updating for brain shift following dural opening.

OBJECTIVE Preoperative magnetic resonance images (pMR) are typically coregistered to provide intraoperative navigation, the accuracy of which can be significantly compromised by brain deformation. In this study, the authors generated updated MR images (uMR) in the operating room (OR) to compensate for brain shift due to dural opening, and evaluated the accuracy and computational efficiency of the process. METHODS In 20 open cranial neurosurgical cases, a pair of intraoperative stereovision (iSV) images was acquired after dural opening to reconstruct a 3D profile of the exposed cortical surface. The iSV surface was registered with pMR to detect cortical displacements that were assimilated by a biomechanical model to estimate whole-brain nonrigid deformation and produce uMR in the OR. The uMR views were displayed on a commercial navigation system and compared side by side with the corresponding coregistered pMR. A tracked stylus was used to acquire coordinate locations of features on the cortical surface that served as independent positions for calculating target registration errors (TREs) for the coregistered uMR and pMR image volumes. RESULTS The uMR views were visually more accurate and well aligned with the iSV surface in terms of both geometry and texture compared with pMR where misalignment was evident. The average misfit between model estimates and measured displacements was 1.80 ± 0.35 mm, compared with the average initial misfit of 7.10 ± 2.78 mm between iSV and pMR, and the average TRE was 1.60 ± 0.43 mm across the 20 patients in the uMR image volume, compared with 7.31 ± 2.82 mm on average in the pMR cases. The iSV also proved to be accurate with an average error of 1.20 ± 0.37 mm. The overall computational time required to generate the uMR views was 7-8 minutes. CONCLUSIONS This study compensated for brain deformation caused by intraoperative dural opening using computational model-based assimilation of iSV cortical surface displacements. The uMR proved to be more accurate in terms of model-data misfit and TRE in the 20 patient cases evaluated relative to pMR. The computational time was acceptable (7-8 minutes) and the process caused minimal interruption of surgical workflow.