Conventional Stereo Research Articles

Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technology.

Stereoscopic imagery often aims to evoke three-dimensional (3-D) percepts that are accurate and realistic-looking. The "gap" between 3-D imagery and real scenes is small, but focus cues typically remain incorrect because images are displayed on a single focal plane. Research has concentrated on the resulting vergence-accommodation conflicts. Yet, incorrect focus cues may also affect the appearance of 3-D imagery. We investigated whether incorrect focus cues reduce perceived realism of 3-D structure ("depth realism"). Experiment1 used a multiple-focal-planes display to compare depth realism with correct focus cues vs. conventional stereo presentation. The stimuli were random-dot stereograms, which isolated the role of focus cues. Depth realism was consistently lower with incorrect focus cues, providing proof-of-principle evidence that they contribute to perceptual realism. Experiments2 and3 examined whether focus cues play a similar role with realistic objects, presented with an almost complete set of visual cues using a high-resolution, high-dynamic-range multiple-focal-planes display. We also examined the efficacy of approximating correct focus cues via gaze-contingent depth-of-field rendering. Improvements in depth realism with correct focus cues were less clear in more realistic scenes, indicating that the role of focus cues in depth realism depends on scene content. Rendering-based approaches, if anything, reduced depth realism, which we attribute to their inability to present higher-order aspects of blur correctly. Our findings suggest future general 3-D display solutions may need to present focus cues correctly to maximise perceptual realism.

Measurement of Three‐Dimensional Bubble distributionVia Single‐Directional Capture Using a Light Field Camera

In water electrolysis, which converts electricity into hydrogen, the distribution of bubbles affects ion transport in the electrolyte. Therefore, measuring the three‐dimensional distribution of bubbles in the electrolysis cell can be effective in improving efficiency. However, the electrolysis cell contains electrodes and membranes, making it difficult to use conventional stereo photography to measure the three‐dimensional bubble distribution. Therefore, in this study, we investigated the measurement of the three‐dimensional distribution of bubbles in liquids using a light field camera that enables three‐dimensional imaging from one direction by using a microlens array. The study was conducted by performing imaging experiments of bubbles ejected from an axisymmetric nozzle using a light field camera and determining whether the measurement of the three‐dimensional distribution of bubbles was possible. As a result, we found that the axisymmetric bubble distribution can be measured in three dimensions and that a light field camera can be used for this measurement. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

A multimodal virtual vision platform as a next-generation vision system for a surgical robot

Robot-assisted surgery platforms are utilized globally thanks to their stereoscopic vision systems and enhanced functional assistance. However, the necessity of ergonomic improvement for their use by surgeons has been increased. In surgical robots, issues with chronic fatigue exist owing to the fixed posture of the conventional stereo viewer (SV) vision system. A head-mounted display was adopted to alleviate the inconvenience, and a virtual vision platform (VVP) is proposed in this study. The VVP can provide various critical data, including medical images, vital signs, and patient records, in three-dimensional virtual reality space so that users can access medical information simultaneously. An availability of the VVP was investigated based on various user evaluations by surgeons and novices, who executed the given tasks and answered questionnaires. The performances of the SV and VVP were not significantly different; however, the craniovertebral angle of the VVP was 16.35° higher on average than that of the SV. Survey results regarding the VVP were positive; participants indicated that the optimal number of displays was six, preferring the 2 × 3 array. Reflecting the tendencies, the VVP can be a neoconceptual candidate to be customized for medical use, which opens a new prospect in a next-generation surgical robot.Graphical

Precision Terrain Modeling Approach in Complex Mountainous Areas Based on Compact UAV Ka-InSAR Data

A High-Precision digital elevation model (DEM) is useful for disaster investigation and evaluation in cloudy, rainy, and complex mountainous areas. However, clouds and rain make the optical images and laser point cloud data acquisition difficult, while noise prohibits obtaining accurate surface information. Additionally, the complex elevation difference in mountainous areas increases the data processing difficulty, such as phase unwrapping and filtering. To overcome these problems, first, we introduce a new airborne multi-baseline Ka-Interferometric Synthetic Aperture Radar (InSAR) system developed by the Beijing Institute of Radio Measurement. The system affords high resolution, small volume, is lightweight, has a good top view angle, and is flexible. Thus, it reduces the flight platform’s dependence and improves the aircrafts adaptability and universality. Moreover, a multi-baseline phase unwrapping method of a two-stage programming approach (TSPA) is selected to overcome the influence of severe noise and the phase continuity assumption limitation. Additionally, an adaptive filtering method for InSAR point clouds considering coherence and optimal bending energy is proposed. This methods validity is verified using stereo satellite images, ground observation point precision checks, and geomorphic texture analysis against existing DEM results. The experimental results demonstrate that the proposed scheme has a good filtering effect on noise, vegetation, residential building areas, and bridges, significantly reducing manual intervention. Moreover, the results highlight that our method is well integrated with stereo images and has more texture details than conventional stereo mapping results, with a mean square error of elevation of 1.938m.

Research on Omnidirectional Stereo Measurement Using Convex Mirrors and Vertical Disparity

We propose an omnidirectional measurement method without blind spots by using a convex mirror, which in principle does not cause chromatic aberration, and by using vertical disparity by installing cameras at the top and bottom of the image. In recent years, there has been significant research in the fields of autonomous cars and robots. In these fields, three-dimensional measurements of the surrounding environment have become indispensable. Depth sensing with cameras is one of the most important sensors for recognizing the surrounding environment. Previous studies have attempted to measure a wide range of areas using fisheye and full spherical panoramic cameras. However, these approaches have limitations such as blind spots and the need for multiple cameras to measure all directions. Therefore, this paper describes a stereo camera system that uses a device capable of taking an omnidirectional image with a single shot, enabling omnidirectional measurement with only two cameras. This achievement was challenging to attain with conventional stereo cameras. The results of experiments confirmed an improvement in accuracy of up to 37.4% compared to previous studies. In addition, the system succeeded in generating depth image that can recognize distances in all directions in a single frame, demonstrating the possibility of omnidirectional measurement with two cameras.

Variable Baseline and Flexible Configuration Stereo Vision Using Two Aerial Robots.

In this work, a new method for aerial robot remote sensing using stereo vision is proposed. A variable baseline and flexible configuration stereo setup is achieved by separating the left camera and right camera on two separate quadrotor aerial robots. Monocular cameras, one on each aerial robot, are used as a stereo pair, allowing independent adjustment of the pose of the stereo pair. In contrast to conventional stereo vision where two cameras are fixed, having a flexible configuration system allows a large degree of independence in changing the configuration in accordance with various kinds of applications. Larger baselines can be used for stereo vision of farther away targets while using a vertical stereo configuration in tasks where there would be a loss of horizontal overlap caused by a lack of suitable horizontal configuration. Additionally, a method for the practical use of variable baseline stereo vision is introduced, combining multiple point clouds from multiple stereo baselines. Issues from using an inappropriate baseline, such as estimation error induced by insufficient baseline, and occlusions from using too large a baseline can be avoided with this solution.

Attitude path design and adaptive robust tracking control of a remote sensing satellite in various imaging modes

This paper addresses the attitude path design of a remote sensing satellite in various imaging modes. The novel path design algorithm is based on local polynomial regression, which produces a smooth attitude path by receiving the size and timing of maneuvers in each axis according to the desired imaging mode. This algorithm is described for stereo and snapshot modes in an imaging operation. The adaptive robust tracking control (ARTC) law is designed using quaternion algebra to perform the required maneuvers in an attitude path. The ARTC structure includes a sliding mode strategy, a projection-based adaptive model compensation, and a linear feedback term. Suitable conditions for imaging in each mode and the time of taking the image are determined by defining and evaluating the attitude control indices. These indices are defined by the half-cone error along the payload line of sight and relative performance error as a jitter with a 3-sigma confidence level. Despite the challenges in the path design, such as smoothness, system agility, and finite time realization of indices, in a conventional stereo imaging mode, taking an image in the nadir-pointing attitude is neglected. As a result, our work provides suitable conditions for taking the image in this attitude with an interval of 1.2 s. Finally, numerical simulations and verification are performed using multidisciplinary simulation, indicating the effectiveness of the proposed algorithm and model-based ARTC law.

Progress towards a Miniaturised PIV System.

Particle image velocimetry is an important optical flow diagnostic tool due to its capacity for investigating a whole flow field without introducing disturbances. However, a significant drawback of PIV methods is their requirement for optical access, making capturing data in closed cavities and confined spaces extremely challenging. A potential approach to overcome this difficulty is miniaturising the system and placing the optical components inside the model. Conventional cross-correlation PIV methods do not allow this due to the size of current PIV cameras. In this study, a miniaturised autocorrelation-based stereo PIV system, which is volumetrically 1.2% of the conventional PIV cameras, was developed and tested. The miniature system is compared with a conventional stereo PIV in wind tunnel experiments up to 16 m/s free stream velocity and a 1.6% velocity difference is observed in the boundary layer flow. Despite a comparatively slow measurement rate of 4.5 Hz, the miniature PIV system demonstrates the ability to measure inside confined spaces and cavities and the ability to be mounted on board models and vehicles. However, limitations remain around conducting measurements with large velocity ranges and with regions of reversed flow due to the challenge of resolving a velocity of 0 m/s.

Motion stereo at sea: Dense 3D reconstruction from image sequences monitoring conveyor systems on board fishing vessels

A system that reconstructs 3D models from a single camera monitoring fish transported on a conveyor system is investigated. Models are subsequently used for training a species classifier and for improving estimates of discarded biomass. It is demonstrated that a monocular camera, combined with a conveyor's linear motion produces a constrained form of multiview structure from motion, that allows the 3D scene to be reconstructed using a conventional stereo pipeline analogous to that of a binocular camera. Although motion stereo was proposed several decades ago, the present work is the first to compare the accuracy and precision of monocular and binocular stereo cameras monitoring conveyors and operationally deploy a system. The system exploits Convolutional Neural Networks (CNNs) for foreground segmentation and stereo matching. Results from a laboratory model show that when the camera is mounted 750 mm above the conveyor, a median accuracy of <5 mm can be achieved with an equivalent baseline of 62 mm. The precision is largely limited by error in determining the equivalent baseline (i.e. distance travelled by the conveyor belt). When ArUco markers are placed on the belt, the inter quartile range (IQR) of error in z (depth) near the optical centre was found to be ±4 mm.

Neural-Network-Based Model-Free Calibration Method for Stereo Fisheye Camera.

The fisheye camera has a field of view (FOV) of over 180°, which has advantages in the fields of medicine and precision measurement. Ordinary pinhole models have difficulty in fitting the severe barrel distortion of the fisheye camera. Therefore, it is necessary to apply a nonlinear geometric model to model this distortion in measurement applications, while the process is computationally complex. To solve the problem, this paper proposes a model-free stereo calibration method for binocular fisheye camera based on neural-network. The neural-network can implicitly describe the nonlinear mapping relationship between image and spatial coordinates in the scene. We use a feature extraction method based on three-step phase-shift method. Compared with the conventional stereo calibration of fisheye cameras, our method does not require image correction and matching. The spatial coordinates of the points in the common field of view of binocular fisheye camera can all be calculated by the generalized fitting capability of the neural-network. Our method preserves the advantage of the broad field of view of the fisheye camera. The experimental results show that our method is more suitable for fisheye cameras with significant distortion.

Description of ticks species from Rattus tiomanicus in Mangrove Forests through scanning electron microscopy

Ticks are ill-famed vectors of many pathogenic organisms which can cause various diseases and life-threatening illnesses to animals and humans. Each tick’s species and its life stages have distinct morphological features that can permit them to be accurately identified. However, the use of conventional stereo microscopes limits the accuracy of species identification. The taxonomy of ticks, in general, is not much understood and existing information is based on sparse morphology data. Thus, this study aims to examine and describe the morphological characteristics of different species of ticks collected on rodents in mangrove forests using scanning electron microscopy (SEM). This method renders high-quality images of body parts of ticks. Five different morphospecies of ticks from one host species (Rattus tiomanicus) were examined under SEM, followed by the PCR technique using mitochondrial 16S rDNA gene for species validation. This study revealed that the ticks belong to five species: Dermacentor auratus, Ixodes granulatus, Haemaphysalis hystricis, D. atrosignatus and Amblyomma cordiferum. The combination of stereomicroscopic and SEM methods has improved our understanding of the morphological characteristics of different tick species, hence establishing up-to-date taxonomic keys for these species. Moreover, due to the lack of taxonomic keys on the immature stage of ticks, the SEM method is essential in characterising the morphological features of these stages in detail, subsequently helpful in revising the taxonomic keys for certain ticks species.

Example-Based Multispectral Photometric Stereo for Multi-Colored Surfaces.

A photometric stereo needs three images taken under three different light directions lit one by one, while a color photometric stereo needs only one image taken under three different lights lit at the same time with different light directions and different colors. As a result, a color photometric stereo can obtain the surface normal of a dynamically moving object from a single image. However, the conventional color photometric stereo cannot estimate a multicolored object due to the colored illumination. This paper uses an example-based photometric stereo to solve the problem of the color photometric stereo. The example-based photometric stereo searches the surface normal from the database of the images of known shapes. Color photometric stereos suffer from mathematical difficulty, and they add many assumptions and constraints; however, the example-based photometric stereo is free from such mathematical problems. The process of our method is pixelwise; thus, the estimated surface normal is not oversmoothed, unlike existing methods that use smoothness constraints. To demonstrate the effectiveness of this study, a measurement device that can realize the multispectral photometric stereo method with sixteen colors is employed instead of the classic color photometric stereo method with three colors.

Fast Direct Stereo Visual SLAM

We propose a novel approach for fast and accurate stereo visual Simultaneous Localization and Mapping (SLAM) independent of feature detection and matching. We extend monocular Direct Sparse Odometry (DSO) to a stereo system by optimizing the scale of the 3D points to minimize photometric error for the stereo configuration, which yields a computationally efficient and robust method compared to conventional stereo matching. We further extend it to a full SLAM system with loop closure to reduce accumulated errors. With the assumption of forward camera motion, we imitate a LiDAR scan using the 3D points obtained from the visual odometry and adapt a LiDAR descriptor for place recognition to facilitate more efficient detection of loop closures. Afterward, we estimate the relative pose using direct alignment by minimizing the photometric error for potential loop closures. Optionally, further improvement over direct alignment is achieved by using the Iterative Closest Point (ICP) algorithm. Lastly, we optimize a pose graph to improve SLAM accuracy globally. By avoiding feature detection or matching in our SLAM system, we ensure high computational efficiency and robustness. Thorough experimental validations on public datasets demonstrate its effectiveness compared to the state-of-the-art approaches.

Mosaic, block-face microscopy for measuring cell dimensions, cell collapse, and spatial relationships in wood

Summary A conventional stereo light microscope was used to image polished wood surfaces at cellular resolution over size scales of the growth ring or larger. Bandpass filtering and local area contrast enhancement were used to aid automatic image thresholding and binarisation. An estimate for the location and proportion of cell collapse was introduced based on the distance between uncollapsed cell lumens. Additionally, spatial associations between vessels were determined using a Euclidean distance transform. The analysis of pith to bark cores provided sufficient detail to show significant intra and inter-annual trends in Pinus radiata tracheid dimensions (wall thickness, wall area, and radial widths). These trends were consistent with expectations and in agreement with the literature. Measured cell dimensions may be influenced by cell collapse and deformation as a result of drying. The analysis of air, kiln and oven-dried Eucalyptus nitens showed that cell collapse was highly variable but generally more prominent in the outer third of growth rings. There were significant changes in vessel shape across the growth rings and vessel area was significantly reduced by drying. The technique provides an intermediate step between detailed microscopy and macroscopic imaging that allows spatial analysis at the wood cell level.

Self-Localization of Mobile Robots Using a Single Catadioptric Camera with Line Feature Extraction.

This paper presents a novel self-localization technique for mobile robots using a central catadioptric camera. A unified sphere model for the image projection is derived by the catadioptric camera calibration. The geometric property of the camera projection model is utilized to obtain the intersections of the vertical lines and ground plane in the scene. Different from the conventional stereo vision techniques, the feature points are projected onto a known planar surface, and the plane equation is used for depth computation. The 3D coordinates of the base points on the ground are calculated using the consecutive image frames. The derivation of motion trajectory is then carried out based on the computation of rotation and translation between the robot positions. We develop an algorithm for feature correspondence matching based on the invariability of the structure in the 3D space. The experimental results obtained using the real scene images have demonstrated the feasibility of the proposed method for mobile robot localization applications.

Method for large-scale structured-light system calibration.

We propose a multi-stage calibration method for increasing the overall accuracy of a large-scale structured light system by leveraging the conventional stereo calibration approach using a pinhole model. We first calibrate the intrinsic parameters at a near distance and then the extrinsic parameters with a low-cost large-calibration target at the designed measurement distance. Finally, we estimate pixel-wise errors from standard stereo 3D reconstructions and determine the pixel-wise phase-to-coordinate relationships using low-order polynomials. The calibrated pixel-wise polynomial functions can be used for 3D reconstruction for a given pixel phase value. We experimentally demonstrated that our proposed method achieves high accuracy for a large volume: sub-millimeter within 1200(H) × 800 (V) × 1000(D) mm3.

A Benchmark Environment for Neuromorphic Stereo Vision.

Without neuromorphic hardware, artificial stereo vision suffers from high resource demands and processing times impeding real-time capability. This is mainly caused by high frame rates, a quality feature for conventional cameras, generating large amounts of redundant data. Neuromorphic visual sensors generate less redundant and more relevant data solving the issue of over- and undersampling at the same time. However, they require a rethinking of processing as established techniques in conventional stereo vision do not exploit the potential of their event-based operation principle. Many alternatives have been recently proposed which have yet to be evaluated on a common data basis. We propose a benchmark environment offering the methods and tools to compare different algorithms for depth reconstruction from two event-based sensors. To this end, an experimental setup consisting of two event-based and one depth sensor as well as a framework enabling synchronized, calibrated data recording is presented. Furthermore, we define metrics enabling a meaningful comparison of the examined algorithms, covering aspects such as performance, precision and applicability. To evaluate the benchmark, a stereo matching algorithm was implemented as a testing candidate and multiple experiments with different settings and camera parameters have been carried out. This work is a foundation for a robust and flexible evaluation of the multitude of new techniques for event-based stereo vision, allowing a meaningful comparison.

A Low-Complexity End-to-End Stereo Matching Pipeline From Raw Bayer Pattern Images to Disparity Maps

Conventional computer vision algorithms, including stereo matching algorithms, take finely rendered color images as input. However, existing image signal processing (ISP) pipelines for color image generation are designed for photography with a goal of generating pleasing images for human eyes. This paper describes a new end-to-end pipeline for stereo matching from raw Bayer pattern images to disparity maps with customized ISP. Unlike conventional stereo matching systems which need a complete ISP module to render full-size standard RGB (sRGB) images, a subsampling-based demosaicing-downsampling (SDD) operation is introduced in the proposed pipeline to demosaic and downsample the Bayer pattern images. The resultant half-size color image pairs are processed with simple denoising and tone mapping algorithms to generate the final input images of stereo matching algorithms. It is found that the simple nearest neighbor upsampling method is good enough to generate the final full-size disparity maps. Experimental results show that the proposed pipeline is capable of generating comparable or even better stereo matching results than the conventional pipeline. By skipping most of the unnecessary ISP steps and reducing the size of input images, the computational complexity of the end-to-end stereo matching pipeline is significantly reduced.

Depth Measurement Based on Stereo Vision With Integrated Camera Rotation

Measuring the distance is one of the main objectives for sensor development. In this article, we present a new sensor system design for depth recovery. It is based on stereopsis with the integrated rotation of an image sensor. Different from conventional stereo vision systems with multiple cameras, the proposed rotational stereo configuration is built upon a single imaging device. By directing its lateral motion with a circular constraint, the visual parallax is generated from multiple viewpoints. The image pairs captured with the integrated sensor rotation are used for the correspondence matching and disparity computation. Moreover, a variation of multiple-baseline stereo with multiple base-angle implementation is proposed. It provides a flexible baseline setting to improve the depth measurement accuracy. The geometric formulation and computational algorithm are derived and developed. In the experiments, a prototype range sensing device is constructed. The results obtained from the real scene environment have demonstrated the feasibility of the proposed depth measurement approach.

Scale-Aware Multi-View Reconstruction Using an Active Triple-Camera System.

This paper presents an active wide-baseline triple-camera measurement system designed especially for 3D modeling in general outdoor environments, as well as a novel parallel surface refinement algorithm within the multi-view stereo (MVS) framework. Firstly, the pre-processing module converts the synchronized raw triple images from one single-shot acquisition of our setup to aligned RGB-Depth frames, which are then used for camera pose estimation using iterative closest point (ICP) and RANSAC perspective-n-point (PnP) approaches. Afterwards, an efficient dense reconstruction method, mostly implemented on the GPU in a grid manner, takes the raw depth data as input and optimizes the per-pixel depth values based on the multi-view photographic evidence, surface curvature and depth priors. Through a basic fusion scheme, an accurate and complete 3D model can be obtained from these enhanced depth maps. For a comprehensive test, the proposed MVS implementation is evaluated on benchmark and synthetic datasets, and a real-world reconstruction experiment is also conducted using our measurement system in an outdoor scenario. The results demonstrate that (1) our MVS method achieves very competitive performance in terms of modeling accuracy, surface completeness and noise reduction, given an input coarse geometry; and (2) despite some limitations, our triple-camera setup in combination with the proposed reconstruction routine, can be applied to some practical 3D modeling tasks operated in outdoor environments where conventional stereo or depth senors would normally suffer.