Acquisition Of Depth Information Research Articles

Dual-Frequency Lidar for Compressed Sensing 3D Imaging Based on All-Phase Fast Fourier Transform

Lidar, with its advantages of high measurement accuracy, fine angular resolution, and strong anti-interference capability, plays a pivotal role in the field of scene depth information acquisition. Traditional approaches to achieving lateral spatial resolution in imaging include raster scanning and array detectors. The former necessitates frequent scanning to acquire depth maps, resulting in time consumption and instability. The latter encounters challenges such as high dark count rates, pixel crosstalk, and excessive costs for obtaining high-resolution images using array detectors. The introduction of compressed sensing (CS) offers a novel perspective on realizing non-scanning three-dimensional imaging. In this context, we propose a novel three-dimensional imaging system that combines CS with coherent dual-frequency continuous-wave lidar and utilizes the all-phase fast Fourier transform to extract both amplitude and phase information. This system requires only M measurements, and through a reconstruction algorithm, it achieves the inversion of depth information for N-pixel scenes (M << N). Integrating cost-effective components such as digital micromirrors and single-point detectors, this affordable system accomplishes three-dimensional imaging of a single target. Notably, it significantly reduces the required number of measurements while concurrently ensuring enhanced eye safety and signal-to-noise ratio.

Deep learning based image quality improvement of a light-field microscope integrated with an epi-fluorescence microscope

Light-field three-dimensional (3D) fluorescence microscopes can acquire 3D fluorescence images in a single shot, and followed numerical reconstruction can realize cross-sectional imaging at an arbitrary depth. The typical configuration that uses a lens array and a single image sensor has the trade-off between depth information acquisition and spatial resolution of each cross-sectional image. The spatial resolution of the reconstructed image degrades when depth information increases. In this paper, we use U-net as a deep learning model to improve the quality of reconstructed images. We constructed an optical system that integrates a light-field microscope and an epifluorescence microscope, which acquire the light-field data and high-resolution two-dimensional images, respectively. The high-resolution images from the epifluorescence microscope are used as ground-truth images for the training dataset for deep learning. The experimental results using fluorescent beads with a size of 10 µm and cultured tobacco cells showed significant improvement in the reconstructed images. Furthermore, time-lapse measurements were demonstrated in tobacco cells to observe the cell division process.

Monocular Depth Estimation of Noncooperative Spacecraft Based on Deep Learning

Estimating position and stereographic geometry information of noncooperative spacecraft through three-dimensional (3-D) reconstruction techniques is of great significance. Depth information acquisition is an important component of 3-D reconstruction. Monocular cameras are cheaper and more widely used than depth sensors. The relative positions of noncooperative spacecraft and our spacecraft can be calculated from depth maps of monocular depth estimation and the camera parameters, providing data support for subsequent tracking and capture missions. This paper proposes a monocular depth estimation network combining the convolutional neural network (CNN) and a vision transformer (VIT) to improve the prediction accuracy of few-shot samples. We extract detail features and global features from the CNN and VIT encoders, respectively, and then fuse deep features and shallow features by a skip-connected upsampling decoder. Compared with the representative depth estimation algorithms in recent years on the NYU-Depth V2 dataset, the proposed network structure combines the advantages of the CNN and VIT as well as estimates the global depth of the scene more accurately while maintaining details. To solve the lack of spacecraft data collection, a new dataset is made from 3-D simulation models. Experiments on the self-made dataset demonstrate the feasibility of this method in aerospace engineering.

Depth information acquisition and image measurement algorithm using microarray camera

Depth information acquisition and image measurement algorithm using microarray camera

A Low-Cost High-Resolution LiDAR System With Nonrepetitive Scanning

Obstacle visualization and detection are essential parts of autonomous driving. The capability of accurately and rapidly collecting 3D information of the surrounding environment plays a key role in driving decisions of autonomous vehicles. As an active detection device, LiDAR enables high-speed and real-time depth information acquisition by emitting laser pulses and then receiving echoes. However, the frame rate and density of the point cloud generated by conventional LiDARs are limited by the fixing trajectory of the beam-steering unit, which cannot satisfy the requirements of various scenes. This paper presented a novel non-repetitive scanning algorithm and a LiDAR prototype using a galvanometer scanner as the beam-steering unit. The algorithm achieves dynamic scanning between frame rate and lateral resolution by the superposition of multiple raw frames with sparser non-repetitive trajectory. The LiDAR was made of off-the-shelf components to limit the cost less than 450 US dollars and extend its application scenarios. Experimental results demonstrated that the proposed LiDAR achieved dynamic frame rate and lateral resolution. Each raw frame of point cloud has a lateral resolution of 190×16 and four raw frames with different trajectories synthesize a superposition frame with quadruple resolution of 190×64. Complex profiles of targets become clear with the superposition of multiple raw frames. Thus, the novel non-repetitive scanning improves the LiDAR performance of detecting obstacles with small sizes and maintains the detection of large obstacles at higher speed.

Learning Optimal Wavefront Shaping for Multi-Channel Imaging.

Fast acquisition of depth information is crucial for accurate 3D tracking of moving objects. Snapshot depth sensing can be achieved by wavefront coding, in which the point-spread function (PSF) is engineered to vary distinctively with scene depth by altering the detection optics. In low-light applications, such as 3D localization microscopy, the prevailing approach is to condense signal photons into a single imaging channel with phase-only wavefront modulation to achieve a high pixel-wise signal to noise ratio. Here we show that this paradigm is generally suboptimal and can be significantly improved upon by employing multi-channel wavefront coding, even in low-light applications. We demonstrate our multi-channel optimization scheme on 3D localization microscopy in densely labelled live cells where detectability is limited by overlap of modulated PSFs. At extreme densities, we show that a split-signal system, with end-to-end learned phase masks, doubles the detection rate and reaches improved precision compared to the current state-of-the-art, single-channel design. We implement our method using a bifurcated optical system, experimentally validating our approach by snapshot volumetric imaging and 3D tracking of fluorescently labelled subcellular elements in dense environments.

Sa1057 THREE-DIMENSIONAL UPPER GASTROINTESTINAL ENDOSCOPY: A CLINICAL STUDY OF SAFETY AND AN EX VIVO STUDY OF UTILITY IN ENDOSCOPIC SUBMUCOSAL DISSECTION

The utility of three-dimensional (3D) endoscopy in the field of therapeutic endoscopy was investigated ex vivo using a newly developed 3D endoscope. Also, we conducted a clinical study to investigate the safety of 3D endoscopy as an upper gastrointestinal (GI) screening tool in actual clinical practice using the 3D endoscope, which had recently been approved as a medical device. To investigate the utility of this 3D endoscope in therapeutic endoscopy, an excised pig stomach with a 10-mm hypothetical lesion was attached to an endoscopic training simulator. The outcomes of endoscopic submucosal dissection (ESD) performed under two-dimensional (2D) and 3D conditions were compared based on the rate of en bloc resection, incidence of perforation, time required for circumferential incision and resection, and severity of eyestrain based on the symptoms of eyestrain, eye pain, blurred vision, heavy-headedness, and headache. Severity of eyestrain was evaluated before and after ESD using a visual analog scale (VAS; 0–100 mm). Eight endoscopists each performed ESD on 4 hypothetical lesions (2 lesions each for 2D and 3D endoscopy). A clinical study was conducted to investigate the safety of 3D endoscopy in 60 patients (36 men, 24 women; mean age 59.3 years) who had given written informed consent to participate in the study. We evaluated the severity of eyestrain in endoscopists before and after each endoscopic procedure, by using the VAS. All 32 lesions were resected en bloc. Perforation was observed in one 2D case. Circumferential incision time was significantly shorter in the 3D group than in the 2D group (102.8 ± 42.1 vs. 135.8 ± 65.7, respectively; p < 0.05), as was resection time (366.3 ± 187.6 s vs. 517.8 ± 282.3 s, respectively; p < 0.05). Intergroup comparisons revealed that the severity of eyestrain-related symptoms, except for blurred vision, was significantly higher in the 3D group than in the 2D group. In the clinical study, none of the 60 patients developed adverse events such as bleeding and perforation, demonstrating the safety of 3D endoscopy. Severity of eyestrain in the endoscopists was significantly higher for all items, except for headache, after endoscopic procedure. Resection time was significantly shorter in 3D ESD than in 2D ESD. Also, 3D ESD was performed safely, with no incidence of perforation. These findings suggest that 3D endoscopy enables rapid and stable ESD because of easy recognition of structures and acquisition of depth information. Furthermore, in the clinical study, 3D endoscopy was performed safely, with no occurrence of severe adverse events.

基于偏振的水下目标深度信息获取方法

基于偏振的水下目标深度信息获取方法

Akwizycja obrazów RGB-D: metody

The two-part article is devoted to sensors enabling the acquisition of depth information from the environment. The following, first part concentrates on three main methods of depth measurement: stereovision, structured light and time of flight (ToF). Along with the principle of operation of each of the method we also deliberate on their properties, analyse the complexity of required computations and present potential applications.

Understanding the contribution of binocular vision to the control of adaptive locomotion.

Abstract. Although the contribution of binocular vision to reach-to-grasp movements has been extensively studied, it has been largely ignored in locomotion. The aim of these studies was to explore the role of binocular vision during the approach phase and step over the obstacle and the contribution of head movements to acquisition of depth information under monocular vision. Binocular and monocular vision was manipulated in different phases using either an eye patch or liquid crystal glasses. Head movement relative to the trunk was restricted in the first experiment by a modified Ferno Universal Head Immobilizer attached to a rigid board strapped to the participant's back. Whole body kinematics were collected by placing infrared diodes on anatomical landmarks and using an Optotrak imaging system. Several measures related to head and limb movement were analyzed. Three major findings emerged from these studies. First, binocular vision is important for the acquisition of accurate information about the surrounding environment: accuracy but not precision of limb elevation over the obstacle was adversely affected when binocular vision was unavailable. Second, motion parallax due to self-motion provides the most critical depth information and it can be used to partially compensate for the loss of binocular vision. Although head movement is not essential to augment depth information, it is important for reorientation of the visual field to obtain the necessary information about the moving limbs when visual field is suddenly limited under monocular vision. Third, step over the obstacle is pre-planned based on visual information acquired during the approach phase: changes in visual condition during the adaptive step do not influence the limb trajectory. Collectively these three studies provide unique insights into the contribution of binocular vision during adaptive locomotion.

Three-dimensional laser microvision

A three-dimensional (3-D) optical imaging system offering high resolution in all three dimensions, requiring minimum manipulation and capable of real-time operation, is presented. The system derives its capabilities from use of the superstructure grating laser source in the implementation of a laser step frequency radar for depth information acquisition. A synthetic aperture radar technique was also used to further enhance its lateral resolution as well as extend the depth of focus. High-speed operation was made possible by a dual computer system consisting of a host and a remote microcomputer supported by a dual-channel Small Computer System Interface parallel data transfer system. The system is capable of operating near real time. The 3-D display of a tunneling diode, a microwave integrated circuit, and a see-through image taken by the system operating near real time are included. The depth resolution is 40 mum; lateral resolution with a synthetic aperture approach is a fraction of a micrometer and that without it is approximately 10 mum.

Small-scale navigation in the honeybee: active acquisition of visual information about the goal

In a series of behavioural studies we found that bees use depth information extracted from self-induced image motion in several visual tasks involving pin-pointing the goal. Some of the results are reviewed here in an attempt to emphasise the active nature of this performance. They show that bees acquire depth information during free flight by employing two different strategies. One is to adapt flight behaviour, upon arrival at the food source, to the requirements of the task, a performance that is based on a learning process. The other is based on a stereotyped, innate flight pattern performed upon departure from the food source. The latter has probably evolved specifically for the acquisition of depth information.