Stereo Vision Research Articles

Analysis of the Teaching Path of Animation Film Majors Integrating Digital Visual Space in the Context of Digital Media Art

Abstract In this study, we first adopt the dynamic stereo vision spatial coordinate measurement method of non-zoom cameras, establish a coordinate measurement model based on differential GPS, and study the application of the spatial coordinates of dynamic stereo digital vision systems in animation design. Secondly, the internal and external parameters of the camera are defined as the static error of the dynamic stereo vision system so as to provide an easy and quick-to-use design tool for the teaching of animation film majors. Finally, a test sequence animation sample set is used to verify the teaching effect of digital visual space based on animation design. The results show that in the multi-comparison test of animation quality against pixel bit rate, the animation design samples were found to have frame rates in the range of 5-30 Fps. And under the condition of having extra bandwidth, the animation feature learning evaluation dimension is 2589, and increasing the animation visual frame rate can improve the output picture quality of animation more effectively. It shows that the integration of digital visual space in the context of digital media art can allow students to have enough project sources for learning and practicing so as to improve their professional ability.

A novel three-dimensional surface reconstruction method for fabric appearance evaluation

Traditional fabric appearance evaluation method mainly relies on subjective and manual operation, which is not uniform and time-consuming. In this paper, one set of self-developed 3D reconstruction system and method based on photometric stereo is proposed for objective fabric appearance evaluation. A photometric stereo image acquisition system equipped with multi-directional lighting sources is developed for the 3D surface reconstruction. Lighting and camera calibration is conducted to obtain the position of light sources and the intrisic parameters of digital camera. Our experimental results show that 3D fabric surface could be reconstructed effectively based on photometric stereo vision method, it has the potential to be used for fabric appearance evaluation.

Light Field Display Based on Monocular Stereo Vision

Light Field Display Based on Monocular Stereo Vision

Energy-Efficient ReS2-Based Optoelectronic Synapse for 3D Object Reconstruction and Recognition.

The neuromorphic vision system (NVS) equipped with optoelectronic synapses integrates perception, storage, and processing and is expected to address the issues of traditional machine vision. However, owing to their lack of stereo vision, existing NVSs focus on 2D image processing, which makes it difficult to solve problems such as spatial cognition errors and low-precision interpretation. Consequently, inspired by the human visual system, an NVS with stereo vision is developed to achieve 3D object recognition, depending on the prepared ReS2 optoelectronic synapse with 12.12 fJ ultralow power consumption. This device exhibits excellent optical synaptic plasticity derived from the persistent photoconductivity effect. As the cornerstone for 3D vision, color planar information is successfully discriminated and stored in situ at the sensor end, benefiting from its wavelength-dependent plasticity in the visible region. Importantly, the dependence of the channel conductance on the target distance is experimentally revealed, implying that the structure information on the object can be directly captured and stored by the synapse. The 3D image of the object is successfully reconstructed via fusion of its planar and depth images. Therefore, the proposed 3D-NVS based on ReS2 synapses for 3D objects achieves a recognition accuracy of 97.0%, which is much higher than that for 2D objects (32.6%), demonstrating its strong ability to prevent 2D-photo spoofing in applications such as face payment, entrance guard systems, and others.

Full field-of-view virtual reality goggles for mice

Visual virtual reality (VR) systems for head-fixed mice offer advantages over real-world studies for investigating the neural circuitry underlying behavior. However, current VR approaches do not fully cover the visual field of view of mice, do not stereoscopically illuminate the binocular zone, and leave the lab frame visible. To overcome these limitations, we developed iMRSIV (Miniature Rodent Stereo Illumination VR)-VR goggles for mice. Our system is compact, separately illuminates each eye for stereo vision, and provides each eye with an ∼180° field of view, thus excluding the lab frame while accommodating saccades. Mice using iMRSIV while navigating engaged in virtual behaviors more quickly than in a current monitor-based system and displayed freezing and fleeing reactions to overhead looming stimulation. Using iMRSIV with two-photon functional imaging, we found large populations of hippocampal place cells during virtual navigation, global remapping during environment changes, and unique responses of place cell ensembles to overhead looming stimulation.

Comparison-embedded evidence-CNN model for fuzzy assessment of wear severity using multi-dimensional surface images

AbstractWear topography is a significant indicator of tribological behavior for the inspection of machine health conditions. An intelligent in-suit wear assessment method for random topography is here proposed. Three-dimension (3D) topography is employed to address the uncertainties in wear evaluation. Initially, 3D topography reconstruction from a worn surface is accomplished with photometric stereo vision (PSV). Then, the wear features are identified by a contrastive learning-based extraction network (WSFE-Net) including the relative and temporal prior knowledge of wear mechanisms. Furthermore, the typical wear degrees including mild, moderate, and severe are evaluated by a wear severity assessment network (WSA-Net) for the probability and its associated uncertainty based on subjective logic. By integrating the evidence information from 2D and 3D-damage surfaces with Dempster–Shafer (D–S) evidence, the uncertainty of severity assessment results is further reduced. The proposed model could constrain the uncertainty below 0.066 in the wear degree evaluation of a continuous wear experiment, which reflects the high credibility of the evaluation result.

Optical Flow-Based Stereo Visual Odometry With Dynamic Object Detection

Autonomous vehicles (AVs) play an important role in the next-generation intelligent transportation system (ITS), which requires AVs to have the ability of rapid decision-making and control. As a part of ITS, simultaneous localization and mapping (SLAM) technology is the basis for AVs to operate in a complex and unknown environment. In this article, a stereo vision odometry method in a dynamic environment is proposed, which can not only effectively overcome the influence of dynamic objects but also detect the position of dynamic objects. An optical flow filtering algorithm based on the quantitative histogram (QH) and the optical flow angle histogram (OFAH) of the feature points is proposed to obtain dynamic points. Furthermore, a multifeature fusion mechanism is used to perform binary segmentation and get the bounding box of dynamic object. Experiments show that the proposed method can improve the accuracy of pose estimation and detect moving objects in a dynamic environment.

Fine-grained damage detection of cement concrete pavement based on UAV remote sensing image segmentation and stitching

A hybrid model for pavement damage segmentation, stitching, and detection based on unmanned aerial vehicle (UAV) remote-sensing pavement images was proposed as high-resolution panoramic images are required to analyze pavement damage feature parameters during the inspection process. Based on target recognition, active stereo vision was used to determine the autonomy awareness of the UAV system in the 3D structure. Then, incremental search strategy was employed to triangulate scale-invariant feature transform (SIFT) point features based on geometric constraints. Perspective-n-point and random sample consensus (PNP-RANSAC) algorithm were combined to find the best matching surface for mis-matching rejection, and the trigonometric function theory was used to fuse images. Subsequently, the semantic segmentation framework based on deep learning context encoder network (CE-Net) was employed to train suitable models to simultaneously detect and segment pavement damages. Finally, the stitching technique and hue, saturation, value (HSV) segmentation technique were combined to obtain the feature parameters to be detected. This method achieved fast and accurate stitching, feature parameter segmentation, and pavement health condition assessment of long-threaded pavement disease image datasets. The splicing accuracy rate and the global feature segmentation overlap rate were greater than 84% and 80%, respectively. The experimental data show that the method is able to detect full-width pavement breakage parameters better than existing optical sensor-based automotive or drone inspection methods.

Fully automatic system for fish biomass estimation based on deep neural network

The approach for estimating biomass in non-contact, free-swimming fish has encountered difficulties such as fish body occlusion, bending, non-orthogonal angles, and low efficiency. To address these issues, this study had combined fish posture recognition (using deep learning technology) with biomass estimation (utilizing stereo vision technology) for the first time, and developed a fast, precise, and fully automatic fish biomass estimation system. The improved single-stage target detection algorithm significantly improved the direct detection and extraction of high-quality images of moving fish in real-time, eliminating the need for manual processing of images that may have imperfect posture. Fish body length and height were measured in the real world using binocular stereo vision technology. Finally, the fish body weight can be estimated by considering the relationship among their body length, height, and weight. The experiment confirmed that the system had successfully avoided influencing factors that could affect fully automatic estimation. The results demonstrated a strong linear relationship between the estimated and measured fish body weights, with a mean relative error (MRE) of 2.87%. There were no significant differences between the estimated and measured weights (p = 0.94). The MRE of the multi-factor model was much lower than that of the single-factor model (length-weight of 8.86% and height-weight of 7.41%). The results indicate that the system developed is a highly effective approach to fully automated biomass estimation. This can be used to guide actual production and further study of the mechanism of fish growth.

A stereo vision SLAM with moving vehicles tracking in outdoor environment

A stereo vision SLAM with moving vehicles tracking in outdoor environment

Design and Development of Traditional Elements in Visual Communication Design under the Integration of Virtual Reality Technology

Abstract This paper proposes a method based on visual communication to generate two-dimensional motion graphics that can enhance the perception of depth and three-dimensional structure in stereoscopic vision. Through eye tracking and calibration, gaze points in 3D space are calculated and applied to Chinese traditional elements in visual communication design. Feature vectors and improved plain Bayesian classifier methods are used to achieve user gesture recognition in virtual design and construct interaction design scenarios based on virtual reality technology. Lastly, the performance impact of augmented reality technology on traditional element design is investigated through a questionnaire survey. Based on the eight samples of conventional elements, the evaluation factors with the highest scores were found to be 10. In the subjective assessment of visual communication design, traditional components fall between mediocre and good, according to the ratings of all 18 assessment factors, which range from 0.2 to 1. The study presented in this paper effectively integrates virtual reality technology into the graphical design of traditional elements, significantly enhancing their visual expression.

A markless 3D human motion data acquisition method based on the binocular stereo vision and lightweight open pose algorithm

This study aimed to achieve precise 3D human motion data acquisition without the need for markers. Specifically, the proposed system utilized two affordable cameras to capture human motion from diverse angles. It developed a lightweight Open Pose algorithm to extract 2D human body joints from recorded videos. Subsequently, a stereo-matching method was applied to derive accurate 3D human body joint data. Comprehensive experiments with 10 healthy participants showcased the system's exceptional performance, demonstrating superior measurement accuracy compared to established standards like the NOKOV system (RMSE values consistently below 9°) and commonly used Microsoft Kinect devices (V2 and Azure). This innovative binocular vision system presents a new approach to motion capture, potentially offering a cost-effective and precise tool for practical applications across various scenarios (e.g., motor functional assessment, virtual avatar streaming), contributing significantly to both scientific and academic communities.

How Do Most Young Moderate Hyperopes Avoid Strabismus?

Uncorrected hyperopic children must overcome an apparent conflict between accommodation and vergence demands to focus and align their retinal images. This study tested hypotheses about simultaneous accommodation and vergence performance of young hyperopes to gain insight into ocular motor strategies used to maintain eye alignment. Simultaneous eccentric photorefraction and Purkinje image tracking were used to assess accommodative and vergence responses of 26 adult emmetropes (AE) and 94 children (0-13 years) viewing cartoons. Children were habitually uncorrected (CU) (spherical equivalent refractive error [SE] -0.5 to +4 D), corrected and aligned (CCA), or corrected with a history of refractive esotropia (CCS). Accommodative and vergence accuracy, dissociated heterophoria, and vergence/accommodation ratios in the absence of retinal disparity cues were measured for 33- and 80-cm viewing distances. In binocular viewing, median accommodative lags for 33 cm were 1.0 D (AE), 1.33 D (CU), 1.25 D (CCA), and 1.0 D (CCS). Median exophorias at 80 and 33 cm were 1.2 and 4.5 pd (AE), 0.8 and 2.5 pd (CU), and 0 and 1.2 pd (CCA), respectively. Without disparity cues, most response vergence/accommodation ratios were between 1 and 2 meter angle/D (∼5-10 pd/D) (69% of AE, 44% of CU, 60% of CCA, and 50% of CCS). Despite apparent conflict in motor coupling, uncorrected hyperopes were typically exophoric and achieved adultlike accuracy of both vergence and accommodation simultaneously, indicating ability to compensate for conflicting demands rather than bias to accurate vergence while tolerating inaccurate accommodation. Large lags and esophoria are therefore atypical. This analysis provides normative guidelines for clinicians and a deeper mechanistic understanding of how hyperopes avoid strabismus.

Method for three-dimensional reconstruction of dynamic stereo vision based on line structured light using global optimization

To overcome the limitation of the measurement range of static stereo vision, a three-dimensional (3-D) reconstruction method of dynamic stereo vision based on line structured light for metal surface is presented in this paper. The dynamic stereo vision is mainly composed of binocular cameras and a laser line generator, and moves along a horizontal guide rail. The intrinsic parameters of the cameras and the rotation and translation of the right camera relative to the left camera are previously obtained by Zhang’s method. To obtain the 3-D coordinates of the line structured light fringe in each position, the sub-pixel coordinates of the fringe center are extracted by Steger algorithm, and a joint calibration method between single camera and a laser line generator is proposed. The joint calibration method calculates the plane equation of the line structured light by using multiple checkerboard images with projected structured light line. To unify the 3-D coordinates of structured light fringe in each position to the reference coordinate system, the rotation and translation of the camera in each position relative to the reference position (called absolute pose) should be solved. The direct linear transformation method is firstly used to calculate the rough relative pose between adjacent frames, and principle of re-projection error minimization in object space is used to obtain the refined relative pose. Instead of using step-by-step transfer method, the absolute pose of the camera in each position is eventually calculated by using global optimization algorithm after the relative pose between all adjacent frames are obtained. Computer simulation and abundant experimental data validate the effectiveness of proposed method. The proposed 3-D reconstruction method using global optimization is respectively compared with step-by-step transfer method based on pixel re-projection error minimization and re-projection error minimization in object space. The experimental results reflect that our method is superior to step-by-step transfer method, and the 3-D reconstruction accuracy is improved by 28.05 % after global optimization.

Analysis of independent risk factors for acute acquired comitant esotropia.

To explore the risk factors for acute acquired comitant esotropia (AACE). A retrospective cohort study was conducted by analyzing 83 patients (case group) with AACE who underwent strabismus correction surgery from January 1, 2021 to June 30, 2022. Totally 73 outpatient volunteers were recruited during the same period as the normal control group. The case group's binocular vision time, near and distance esotropia angle, and near stereo vision function were recorded, and the age, gender, refractive status, and best-corrected visual acuity (BCVA) of both groups were analyzed. Additionally, multiple logistic regression analysis was conducted using an eye usage condition questionnaire to determine the independent risk factors for AACE. In the case group, 61 patients (73.49%) had myopia, with a mean equivalent spherical power (SE) of -3.35±3.31 D (range: +2.75 to -10.62 D) of the right eye and -2.87±3.35 D (range: +2.75 to -11.12 D) of the left eye. The average duration of diplopia in the case group was 29.83±35.72mo, of which 80 patients (96.39%) were primarily with distance diplopia. The near and distance esotropia angle after wearing glasses were 52.36±20.95 prism degree (PD) and 56.71±19.54 PD, respectively, and there was no statistically significant difference between the two (t=1.38, P=0.169). The incidence of improper glasses wearing and unhealthy eye habits in the case group was significantly higher than those in the control group (P<0.05). Close-up work without glasses [β=2.30, odds ratio (OR)=10, 95% confidence interval (CI) 2.35-42.51, P=0.002] and near work in supine position (β=1.80, OR=6.02, 95%CI 3.29-11.02, P<0.001) were independent risk factors for AACE. Patients with AACE mainly present with distance diplopia, and there is a high degree of variation in myopia. Near work without wearing glasses and in supine position are independent risk factors for AACE.

Neural correlates of virtual reality-based attention training: An fMRI study

Theoretical backgroundVirtual Reality technology is increasingly used in attention rehabilitation for functional training purposes. However, the neural mechanisms by which Virtual Reality can affect attentional functioning are still unclear. The current study's objective is to examine the effects of stereoscopic vs. monoscopic presentation on neural processing during a visual attention task. MethodThirty-two healthy participants performed a visual attention task in an immersive virtual environment that was displayed via MR-compatible video goggles in an MRI scanner. The paradigm altered between trials that required active engagement with the task and mere observation trials. Furthermore, the form of binocular presentation switched between monoscopic and stereoscopic presentation. ResultsAnalyses yielded evidence for increased activation in stereoscopic compared to monoscopic trials in the tertiary visual cortex area V3A as well as elevated activation in the dorsal attention network when engaging in the attention task. An additional ROI analysis of area V3A revealed significantly lower attentional engagement costs in stereoscopic conditions. DiscussionResults support previous findings suggesting that V3A is involved in binocular depth perception. Furthermore, heightened activation in V3A following stereoscopic presentation seemed to facilitate attentional engagement with the task. Considering that V3A is the origin of the dorso-dorsal, ventro-dorsal, and ventral visual processing pathways, we regard it as a gating area that decides which kind of visual perception is processed.

Camera calibration method through multivariate quadratic regression for depth estimation on a stereo vision system

Machine vision systems have demonstrated practical applications in various fields where the perception of the environment is essential. Stereo vision systems are one of the most used methods to perform three-dimensional mapping. These systems have several considerations in order to fulfill this task; One of them is camera calibration. This is why this paper proposes a novel camera calibration method for improving depth estimation accuracy in stereo vision systems. The method addresses the lens distortion issue by adjusting the surface point's pixel coordinates before the triangulation process. The new pixel coordinates are computed using calibration coefficients, which are calculated through multivariate quadratic regression. Also, the proposed method corrects the relative orientation of the cameras and computes a compensation angle. The proposed method is remarkably robust to varying illumination levels because it adjusts pixel positions independently of image content. In comparison to traditional methods, this method requires fewer steps to implement, as it only requires one image per camera. The proposed method could have significant implications for fields such as autonomous navigation and robotics, where depth estimation is a crucial component. Overall, this paper presents a valuable contribution to the field of computer vision by offering a new, efficient and effective approach to camera calibration for stereo vision systems. The performed experiments demonstrated that the proposed method leads to improved depth estimation in the stereo vision system with an improvement of 34.15% on the MAE and 48.38% on the STD when compared to one of the most commonly used methods.

Seaweed Growth Monitoring with a Low-Cost Vision-Based System.

In this paper, we introduce a method for automated seaweed growth monitoring by combining a low-cost RGB and stereo vision camera. While current vision-based seaweed growth monitoring techniques focus on laboratory measurements or above-ground seaweed, we investigate the feasibility of the underwater imaging of a vertical seaweed farm. We use deep learning-based image segmentation (DeeplabV3+) to determine the size of the seaweed in pixels from recorded RGB images. We convert this pixel size to meters squared by using the distance information from the stereo camera. We demonstrate the performance of our monitoring system using measurements in a seaweed farm in the River Scheldt estuary (in The Netherlands). Notwithstanding the poor visibility of the seaweed in the images, we are able to segment the seaweed with an intersection of the union (IoU) of 0.9, and we reach a repeatability of 6% and a precision of the seaweed size of 18%.

Single-center experience of transitioning from video-assisted laparoscopic to robotic Heller myotomy with Dor fundoplication for esophageal motility disorders

BackgroundVideo-assisted laparoscopic Heller myotomy (LHM) has become the standard treatment option for achalasia. While robotic surgery offering some specific advantages such as better three-dimensional (3D) stereoscopic vision, hand-eye consistency, and flexibility and stability with the endowrist is expected to be shorter in learning curve than that of LHM for surgeons who are proficient in LHM. The aim of this study was to describe a single surgeon’s experience related to the transition from video-assisted laparoscopic to robotic Heller myotomy with Dor fundoplication.MethodsWe conducted a retrospective observational study based on the recorded data of the first 66 Heller myotomy performed with laparoscopic Heller myotomy with Dor fundoplication (LHMD, 26 cases) and with the robotic Heller myotomy with Dor fundoplication (RHMD, 40 cases) by the same surgeon in Department of Thoracic Surgery of The First Affiliated Hospital of Nanchang University in China. The operation time and intraoperative blood loss were analyzed using the cumulative sum (CUSUM) method. Corresponding statistical tests were used to compare outcomes of both serials of cases.ResultsThe median operation time was shorter in the RHMD group compared to the LHMD group (130 [IQR 123–141] minutes vs. 163 [IQR 153–169]) minutes, p < 0.001). In the RHMD group, one patient (2.5%) experienced mucosal perforation, whereas, in the LHMD group, the incidence of this complication was significantly higher at 19.2% (5 patients) (p = 0.031). Based on cumulative sum analyses, operation time decreased starting with case 20 in the LHMD group and with case 18 in the RHMD group. Intraoperative blood loss tended to decline starting with case 19 in the LHMD group and with case 16 in the RHMD group.ConclusionsBoth RHMD and LHMD are effective surgical procedures for symptom relief of achalasia patients. RHMD demonstrates superior outcomes in terms of operation time and mucosal perforation during surgery compared to LHMD. Proficiency with RHMD can be achieved after approximately 16–18 cases, while that of LHMD can be obtained after around 19–20 cases.

What's special about horizontal disparity.

Horizontal disparity has been recognized as the primary signal driving stereoscopic depth since the invention of the stereoscope in the 1830s. It has a unique status in our understanding of binocular vision. The direction of offset of the eyes gives the disparities of corresponding image point locations across the two retinas a strong horizontal bias. Beyond the retina, other factors give shape to the effective disparity direction used by visual mechanisms. The influence of orientation is examined here. I argue that horizontal disparity is an inflection point along a continuum of effective directions, and its role in stereo vision can be reinterpreted. The pointwise geometric justification for its special status neglects the oriented structural elements of spatial vision, its physiological support is equivocal, and psychophysical support of its special status may partially reflect biased stimulus sampling. The literature shows that horizontal disparity plays no particular role in the processing of one-dimensional stimuli, a reflection of the stereo aperture problem. The resulting depth is non-veridical, even non-transitive. Although one-dimensional components contribute to the stereo depth of visual objects generally, two-dimensional stimuli appear not to inherit the aperture problem. However, a look at the two-dimensional stimuli that predominate in experimental studies shows regularities in orientation that give a new perspective on horizontal disparity.