Depth Of Field Imaging Research Articles

A multi‐focus image fusion network deployed in smart city target detection

AbstractIn the global monitoring of smart cities, the demands of global object detection systems based on cloud and fog computing in intelligent systems can be satisfied by photographs with globally recognized properties. Nevertheless, conventional techniques are constrained by the imaging depth of field and can produce artefacts or indistinct borders, which can be disastrous for accurately detecting the object. In light of this, this paper proposes an artificial intelligence‐based gradient learning network that gathers and enhances domain information at different sizes in order to produce globally focused fusion results. Gradient features, which provide a lot of boundary information, can eliminate the problem of border artefacts and blur in multi‐focus fusion. The multiple‐receptive module (MRM) facilitates effective information sharing and enables the capture of object properties at different scales. In addition, with the assistance of the global enhancement module (GEM), the network can effectively combine the scale features and gradient data from various receptive fields and reinforce the features to provide precise decision maps. Numerous experiments have demonstrated that our approach outperforms the seven most sophisticated algorithms currently in use.

Fusion of full-field optical angiography images via gradient feature detection

Full-field optical angiography (FFOA)—a real-time non-invasive imaging technique for extracting biological blood microcirculation information—contributes to an in-depth understanding of the functional and pathological changes of biological tissues. However, owing to the limitation of the depth-of-field (DOF) of optical lenses, existing FFOA imaging methods cannot capture an image containing every blood-flow information. To address this problem, this study develops a long-DOF full-field optical angiography imaging system and proposes a novel multi-focus image fusion scheme to expand the DOF. First, FFOA images with different focal lengths are acquired by the absorption intensity fluctuation modulation effect. Second, an image fusion scheme based on gradient feature detection in a nonsubsampled contourlet transform domain is developed to capture focus features from FFOA images and synthesize an all-focused image. Specifically, FFOA images are decomposed by NSCT into coefficients and low-frequency difference images; thereafter, two gradient feature detection-based fusion rules are used to select the pre-fused coefficients. The experimental results of both phantom and animal cases show that the proposed fusion method can effectively extend the DOF and address practical FFOA image defocusing problems. The fused FFOA image can provide a more comprehensive description of blood information than a single FFOA image.

Aberration correction for deformable-mirror-based remote focusing enables high-accuracy whole-cell super-resolution imaging

Single-molecule localization microscopy (SMLM) enables three-dimensional (3D) investigation of nanoscale structures in biological samples, offering unique insights into their organization. However, traditional 3D super-resolution microscopy using high numerical aperture (NA) objectives is limited by imaging depth of field (DOF), restricting their practical application to relatively thin biological samples. Here, we developed a unified solution for thick sample super-resolution imaging using a deformable mirror (DM) which served for fast remote focusing, optimized point spread function (PSF) engineering, and accurate aberration correction. By effectively correcting the system aberrations introduced during remote focusing and sample aberrations at different imaging depths, we achieved high-accuracy, large DOF imaging ( ∼8 μm ) of the whole-cell organelles [i.e., nuclear pore complex (NPC), microtubules, and mitochondria] with a nearly uniform resolution of approximately 35 nm across the entire cellular volume.

Joint focusing method based on optimal image-distance interval for multi-image-distance imaging system

The multi-image-distance imaging system captures multiple images simultaneously with different focused depths, and uses image fusion to obtain an all-in-focus image for extending the depth of field (DoF). However, an image-distance adjustment strategy that takes into account the optimal DoF extension capability and imaging quality remains to be explored. In this paper, we propose a joint focusing method based on the optimal image-distance interval between the three sensors. First, we propose multi-sensor image-distance adjustment control strategies for different scenes. Then, theoretical analyses and calibration experiments of the optimal image-distance interval are conducted for continuous depth scenes. In order to quantitatively evaluate the DoF extension performance of the imaging system, a DoF estimation method based on point spread function measurement is proposed. Additionally, an evaluation metrics for extended DoF imaging based on the range and continuity of DoF is presented. We have developed a multi-image-distance electric focusing imaging system. Imaging experiments are conducted in various scenes to verify the effectiveness of the joint focusing method. The joint focusing method, based on the optimal image-distance interval, provides an image-distance adjustment strategy to achieve high-quality extended DoF imaging for multi-image-distance imaging systems.

Determining depth of field for slanted lenticular 3D displays

The availability of higher resolution display panels has increasingly made glasses-free 3D displays become a viable mainstream commercial product; it is therefore important to define and measure their parameters. We will discuss the measurement of multiview 3D displays that use a slanted lenticular screen in front of the display panel to control the light directions. Multiple perspective views are formed across the viewing field giving viewers the sensation of depth and motion parallax. In addition to the usual parameters of resolution, luminance, contrast etc., it is important that we know the image depth of field (DOF). In this paper, we will first define the DOF and then describe means of measuring it. The aim of the paper is to describe general theory and procedure, and not the measurement of specific displays. However, a comparison of the results on a sample test display of the three methods described, is reported in order to give an indication of accuracy.

High-resolution and high-speed 3D tracking of microrobots using a fluorescent light field microscope.

Imaging and tracking are crucial for microrobots which navigate through complex 3D environments. Fluorescent imaging (FI) by microscope offers a high-resolution and high-sensitive imaging method to study the property of microrobots. However, conventional microscope suffers from shallow depth of field (DOF) and lacks 3D imaging capability. We proposed a high-resolution and high-speed 3D tracking method for microrobots based on a fluorescent light field microscope (FLFM). We designed the FLFM system according to the size of a representative helical microrobot (150 μm body length, 50 μm screw diameter), and studied the system's performance. We also proposed a 3D tracking algorithm for microrobots using digital refocusing. We validated the method by simulations and built an FLFM system to perform the tracking experiments of microrobots with representative size. Our 3D tracking method achieves a 30 fps data acquisition rate, 10 μm lateral resolution and approximately 40 μm axial resolution over a volume of 1,200×1,200×326 μm3. Results indicate that the accuracy of the method can reach about 9 μm. Compared with the FI by a conventional microscope, the FLFM-based method gains wider DOF and 3D imaging capability with a single-shot image. The tracking method succeeds in providing the trajectory of the microrobot with a good lateral resolution.

Integrated MPCAM: Multi-PSF learning for large depth-of-field computational imaging

Large DOF (depth-of-field) imaging with high SNR (signal-noise-ratio) is useful for applications such as machine vision and medical imaging. In traditional optical systems, DOF extension is always implemented at the cost of SNR. In this paper, we present a MPCAM (Multi-PSF Camera) system highly integrated with AF (auto-focus) function to realize both large DOF and high SNR imaging. MPCAM based on MPGAN (Multi-PSF Generative Adversarial Network) is first proposed to automatically extract multiple PSFs (point spread functions) and realize high fidelity image reconstruction by features fusion. The proposed end-to-end generative image fusion network is flexible and can be designed with different input dimensions for a given AF application, which is vital to circumvent the trade-off between DOF and SNR. We build a dataset containing 5000 raw images tailored to the proposed network by an off-the-shelf camera. Results show that our MPCAM system can produce images with average higher values than raw images over 4.625, and 0.061 in PNSR (peak signal to noise ratio), and SSIM (structure similarity) metrics, respectively. Moreover, compared to the classic and latest image fusion methods, the results also verify that our method has achieved comparable or even better performance. Due to its advance in high SNR and large DOF imaging, this novel, portable and inexpensive system is suitable for computational applications such as microscopic pathological diagnosis, domain-specific computational imaging and smartphone photography. The implementation code of MPGAN and dataset are available from https://www.kaggle.com/datasets/ktd970903/multi-psf-camera.

Freeform optics for variable extended depth of field imaging.

Imaging depth of field is shallow in applications with high magnification and high numerical aperture, such as microscopy, resulting in images with in- and out-of-focus regions. Therefore, methods to extend depth of field are of particular interest. Researchers have previously shown the advantages of using freeform components to extend depth of field, with each optical system requiring a specially designed phase plate. In this paper we present a method to enable extended depth-of-field imaging for a range of numerical apertures using freeform phase plates to create variable cubic wavefronts. The concept is similar to an Alvarez lens which creates variable spherical wavefronts through the relative translation of two transmissive elements with XY polynomial surfaces. We discuss design and optimization methods to enable extended depth of field for lenses with different numerical aperture values by considering through-focus variation of the point spread function and compare on- and off-axis performance through multiple metrics.

Spatial light path analysis and calibration of four-mirror-based monocular stereo vision.

The feasibility and accuracy of four-mirror-based monocular stereo vision (FMSV) are related to the system layout and calibration accuracy, respectively. In this study, a spatial light path analysis method and a calibration method are proposed for an FMSV system. As two-dimensional light path analysis cannot fully characterize the imaging parameters, a spatial light path model is proposed, which allows refinement of the system design. Then, considering the relationship between the lens distortion and the imaging depth of field (DoF), a DoF-distortion equal-partition-based model is established. In the traditional calibration method, the optical axis must be perpendicular to the chessboard. Here, an accurate and practical FMSV calibration method without this constraint is proposed based on the above model. Using the proposed spatial light path analysis technique, a high-accuracy, high-portability FMSV system is constructed and calibrated, for which the average error of the vision-reconstructed distance is 0.0298 mm. In addition, robot path accuracy is evaluated by the system and compared to laser-tracker measurement results. Hence, high accuracy of 0.031 mm is determined for the proposed vision system.

Virtual Source Synthetic Aperture for Accurate Lateral Displacement Estimation in Ultrasound Elastography.

Ultrasound elastography (USE) is an emerging noninvasive imaging technique in which pathological alterations can be visualized by revealing the mechanical properties of the tissue. Estimating tissue displacement in all directions is required to accurately estimate the mechanical properties. Despite capabilities of elastography techniques in estimating displacement in both axial and lateral directions, estimation of axial displacement is more accurate than lateral direction due to higher sampling frequency, higher resolution, and having a carrier signal propagating in the axial direction. Among different ultrasound imaging techniques, synthetic aperture (SA) has better lateral resolution than others, but it is not commonly used for USE due to its limitation in imaging depth of field. Virtual source synthetic aperture (VSSA) imaging is a technique to implement SA beamforming on the focused transmitted data to overcome the limitation of SA in depth of field while maintaining the same lateral resolution as SA. Besides lateral resolution, VSSA has the capability of increasing sampling frequency in the lateral direction without interpolation. In this article, we utilize VSSA to perform beamforming to enable higher resolution and sampling frequency in the lateral direction. The beamformed data are then processed using our recently published elastography technique, OVERWIND. Simulation and experimental results show substantial improvement in the estimation of lateral displacements.

Ranking-Based Salient Object Detection and Depth Prediction for Shallow Depth-of-Field.

Shallow depth-of-field (DoF), focusing on the region of interest by blurring out the rest of the image, is challenging in computer vision and computational photography. It can be achieved either by adjusting the parameters (e.g., aperture and focal length) of a single-lens reflex camera or computational techniques. In this paper, we investigate the latter one, i.e., explore a computational method to render shallow DoF. The previous methods either rely on portrait segmentation or stereo sensing, which can only be applied to portrait photos and require stereo inputs. To address these issues, we study the problem of rendering shallow DoF from an arbitrary image. In particular, we propose a method that consists of a salient object detection (SOD) module, a monocular depth prediction (MDP) module, and a DoF rendering module. The SOD module determines the focal plane, while the MDP module controls the blur degree. Specifically, we introduce a label-guided ranking loss for both salient object detection and depth prediction. For salient object detection, the label-guided ranking loss comprises two terms: (i) heterogeneous ranking loss that encourages the sampled salient pixels to be different from background pixels; (ii) homogeneous ranking loss penalizes the inconsistency of salient pixels or background pixels. For depth prediction, the label-guided ranking loss mainly relies on multilevel structural information, i.e., from low-level edge maps to high-level object instance masks. In addition, we introduce a SOD and depth-aware blur rendering method to generate shallow DoF images. Comprehensive experiments demonstrate the effectiveness of our proposed method.

Extended DoF and achromatic inverse imaging for lens and lensless MPM camera based on Wiener filtering of defocused OTFs

Optimal optical transfer function (OTF) is proposed for RGB inverse imaging with extended depth of field (DoF) and diminished color aberrations. This optimal OTF is derived as the Wiener filter of broadband defocused OTFs. The performance of this new inverse imaging is demonstrated for optical setups with lens and lensless with a multilevel phase mask instead of the lens. The later lensless system designed for the wavelength range (400 to 700) nm and DoF range (0.5 to 1000) m demonstrates the best performance.

Extended depth-of-field imaging employing integrated binary phase pupil mask and principal component analysis image fusion.

The imaging depth of field (DOF) of white-light illuminated objects is extended by carefully integrating two image-processing techniques, one optical and one digital. The optical technique makes use of a tailored phase mask positioned at the pupil of the imaging system to cause different color channels to have different focal lengths; accordingly, the phase-mask equipped imaging system acquires a high resolution and reasonably focused image in at least one of the three, red, green, blue (RGB), color channels at any location within the specified DOF. The digital processing comprises fusing the separate RGB images with an original technique that implements principal component analysis to deliver the overall sharpest grayscale composite image throughout the DOF region. The obtained experimental results agree well with the theoretical predictions and demonstrate the capability of the integrated technique to extend the DOF.

Volumetric PIV measurement for capturing the port flow characteristics within annular gas turbine combustors

The three-dimensional flows within a full featured, unmodified annular gas turbine combustor have been investigated using a scanned stereoscopic PIV measurement technique. Volumetric measurements have been achieved by rigidly translating a stereoscopic PIV system to scan measurements around the combustor, permitting reconstruction of volumetric single-point statistics. Delivering the measurements in this way allows the measurement of larger volumes than are accessible using techniques relying upon high depth of field imaging. The shallow depth of field achieved in the stereoscopic configuration furthermore permits measurements in close proximity to highly detailed geometry. The measurements performed have then been used to assess the performance of the combustor port flows, which are central to the emissions performance and temperature/velocity profile at turbine inlet. Substantially differing performance was observed in the primary ports with circumferential position, which was found to influence the behaviour of the second secondary port jets. The measurements indicated that the interaction between the primary and secondary jets occurred due to variations in the external boundary conditions imposed by the annular passages in which the combustor is located.Graphic abstract

Tunable Aperture With Liquid Crystal for Real-Time Distance Sensor

For distance detection, a tunable aperture using liquid crystal is proposed, achieving high resolution images in real-time. The tunable aperture inserted into a camera has been designed to two sizes, which can create different Depth of Field images. By analyzing these two images, distance to object can be extracted. Initially, the aperture has large size with zero voltage. When the driving voltage is applied, the aperture transfers itself to smaller size by realignment of liquid crystal molecules in the device. The diameter of aperture is 4.4 mm with f/1.8 and 2.0 mm with f/4.0. The proposed device has low driving voltage of 2.8 V and fast response time of 9.84 ms. Compact size aperture of $10\times {10}\times 1.8$ mm3 is assembled in a camera with focal length of 8 mm which contains 5 glass lenses and 1/2.7- inch full high definition (FHD) image sensor. The tunable aperture has high stability due to no mechanically moving parts. This distance sensor can be applied to the various field of depth map application which is the self-driving car, drones and manufacturing machine.

Rationing multi-shot techniques to maximizing depth of field.

The depth of the field is an important element for the formulation of the visual message; it defines the relationship between the center of attention and the surrounding elements, affecting how the image is perceived, and thus its ability to tell and express. Despite this vital rule of depth of field in image designing, the photographer loses full control of it, especially in those cases that require maximizing it, especially when shooting close up. The lack of depth of the field is overcome by photographing graduates focus series of images, then stacking them to extend the depth of, The study examined the production of the series of shots through the lens of selection and the methods of focusing and capture, and then the processing stage to produce the final image.Objective: The research aims to standardize the Focus stacking technique to maximize the depth of field in the photograph, especially when shooting close up.

The impact of Image design and quality at distinctive industries in the Arab Republic of Egypt

The depth of the field is an important element for the formulation of the visual message; it defines the relationship between the center of attention and the surrounding elements, affecting how the image is perceived, and thus its ability to tell and express. Despite this vital rule of depth of field in image designing, the photographer loses full control of it, especially in those cases that require maximizing it, especially when shooting close up.

Extending the depth-of-field of imaging systems with a scattering diffuser

Large depth of field (DOF) is a longstanding goal in optical imaging field. In this paper we presented a simple but efficient method to extend the DOF of a diffraction-limited imaging system using a thin scattering diffuser. The DOF characteristic of the imaging system with random phase modulation was analyzed based on the analytical model of ambiguity function as a polar display of the optical transfer function (OTF). The results of numerical simulation showed that more high-frequency components existed in the defocused OTF curve when the exit pupil of the imaging system exhibited a random phase modulation. It proved the important role of the scattering diffuser in extending the DOF of imaging systems. For the reconstruction, a stack of point spread functions (PSFs) corresponding to different axial locations within a measurement range were superimposed to construct the stacked PSF. Then the large DOF image was recovered from a speckle pattern by deconvolution. In this proof-of-concept, we experimentally demonstrated the single-shot imaging with larger DOF using a thin glass scattering diffuser in both a single-lens imaging system and a microscopic imaging system.

Bimodal Incoherent Digital Holography for Both Three-Dimensional Imaging and Quasi-Infinite\u2013Depth-of-Field Imaging

Although three-dimensional (3D) imaging and extended depth-of-field (DOF) imaging are completely opposite techniques, both provide much more information about 3D scenes and objects than does traditional two-dimensional imaging. Therefore, these imaging techniques strongly influence a wide variety of applications, such as broadcasting, entertainment, metrology, security and biology. In the present work, we derive a generalised theory involving incoherent digital holography to describe both 3D imaging and quasi-infinite–DOF (QIDOF) imaging, which allows us to comprehensively discuss the functions of each imaging technique. On the basis of this theory, we propose and develop a bimodal incoherent digital holography system that allows both 3D imaging and QIDOF imaging. The proposed system allows imaging objects using spatially incoherent light and reconstructing 3D images or QIDOF images solely by changing the phase pattern of a spatial light modulator and without requiring mechanical adjustments or any other modifications to the setup. As a proof-of-principle experiment, we evaluate the DOF and record holograms of a reflective object with the proposed system. The experimental results show that the generalised theory is effective; our demonstration platform provides the function of 3D and QIDOF imaging.

DeepLens

We aim to generate high resolution shallow depth-of-field (DoF) images from a single all-in-focus image with controllable focal distance and aperture size. To achieve this, we propose a novel neural network model comprised of a depth prediction module, a lens blur module, and a guided upsampling module. All modules are differentiable and are learned from data. To train our depth prediction module, we collect a dataset of 2462 RGB-D images captured by mobile phones with a dual-lens camera, and use existing segmentation datasets to improve border prediction. We further leverage a synthetic dataset with known depth to supervise the lens blur and guided upsampling modules. The effectiveness of our system and training strategies are verified in the experiments. Our method can generate high-quality shallow DoF images at high resolution, and produces significantly fewer artifacts than the baselines and existing solutions for single image shallow DoF synthesis. Compared with the iPhone portrait mode, which is a state-of-the-art shallow DoF solution based on a dual-lens depth camera, our method generates comparable results, while allowing for greater flexibility to choose focal points and aperture size, and is not limited to one capture setup.