Optical Image Processing Research Articles

Engineering Large-Area Antidust Surfaces by Harnessing Interparticle Forces.

Dust accumulation is detrimental to optical elements, electronic devices, and mechanical systems and is a significant problem in space missions and renewable energy deployment. In this paper, we report the demonstration of antidust nanostructured surfaces that can remove close to 98% of lunar particles solely via gravity. The dust mitigation is driven by a novel mechanism, whereby particle removal is facilitated by the formation of particle aggregates due to interparticle forces, allowing the particles to be removed in the presence of other particles. The structures are fabricated using a highly scalable nanocoining and nanoimprint process, where nanostructures with precise geometry and surface properties are patterned on polycarbonate substrates. The dust mitigation properties of the nanostructures have been characterized using optical metrology, electron microscopy, and image processing algorithms to demonstrate that the surfaces can be engineered to remove nearly all of the particles above 2 μm in size in the presence of Earth's gravity. Compared to the 35.0% area coverage on a smooth polycarbonate surface, the particle coverage on nanostructures with 500 nm period is significantly reduced to 2.4%, an improvement of 93%. This work enhances the understanding of the particulate adhesion on textured surfaces and demonstrates a scalable, effective solution to antidust surfaces that can be broadly applied to windows, solar panels, and electronics.

Dynamically switchable multifunctional terahertz absorber based on graphene and vanadium dioxide hybrid metamaterials

A multifunctional terahertz absorber based on a hybrid configuration of vanadium dioxide ( V O 2 ) and graphene is proposed, which is enabled to dynamically switch between ultrabroadband and double-narrowband absorption characteristics using the phase-shifting property of V O 2 . When V O 2 is in the metallic phase and the Fermi energy level of graphene is 0.01 eV, ultrabroadband absorption from 3.04 THz to 8.78 THz can be achieved. The absorption can be continuously adjusted from 0.7% to 99.9% by changing the conductivity of the V O 2 through temperature. When V O 2 is in the insulating phase and Fermi energy level of graphene is 0.4 eV, the proposed absorber exhibits a double-narrowband absorption characteristic, achieving 99.8% and 97.3% absorption at 1.79 THz and 4.62 THz, respectively. Since graphene is an electronically controlled material, it is able to dynamically modulate the resonant frequency and absorption intensity by changing the Fermi energy level of graphene. In addition, the physical mechanism of the multifunctional absorber is revealed using electric field distribution and the impedance-matching theory. The proposed absorber has potential applications in optical switching, image processing, and stealth technology.

All-optical image edge detection based on the two-dimensional photonic spin Hall effect in anisotropic metamaterial.

Optical image processing based on the photonic spin Hall effect (SHE) has been gaining significant attention as a convenient and an accurate way for image edge detection. However, the recent edge imaging techniques depending on optical differentiation is mainly achieved by modulation of one-dimensional photonic SHE. Here, we theoretically predict the two-dimensional photonic SHE in the anisotropic metamaterial, and find that its longitudinal and transverse displacements exhibit spin-dependent property at filling factors with increasing incidence angle. As the transverse and in-plane displacements induced by the photonic SHE can be controlled by the filling factor of the crystal structure, the optical axis angle, and the incident angle, this intrinsical effect can be used to realize a tunable edge imaging. Interestingly, by changing the optical axis of the anisotropic metamaterial, the in-plane displacements are equal to the transverse displacements for a certain filling factor and the corresponding incident angle. Therefore, we propose a two-dimensional image edge detection method based on the photonic SHE in anisotropic metamaterial. Further numerical results validate the theoretical proposal.

Spatiotemporal Optical Vortices: Toward Tailoring Orbital Angular Momentum of Light in Full Space-Time

Longitudinal orbital angular momentum (OAM) is a fundamental property of light that has been the subject of extensive research and applied to diverse important applications, such as optical tweezers, super-resolution imaging, and optical information processing. Recently, photons have been observed to possess transverse OAM, where the OAM vector is orthogonal to the direction of light propagation. As the carrier for this unique OAM, the spatiotemporal (ST) optical vortex has attracted considerable interest. Here, we provide a general overview of recent experimental and theoretical advances on such vortices, presenting their synthesis and measurement strategies, highlighting their nontrivial properties in linear propagation and interaction with nonlinear matter, and discussing the possible future trends and challenges.

Optical image addition and subtraction based on electromagnetically induced transparency effect

The electromagnetically induced transparency (EIT) effect is a quantum coherence effect between different atomic transition channels. The absorption of the atomic ensemble is significantly reduced or even close to zero, and at the same time, this effect is accompanied by strong spectral dispersion near the resonant frequency of atoms, which is widely used in group velocity control, light pulse storage, and coherent manipulation on the light field. In the light pulse storage based on the EIT effect, the retrieval field is determined by the signal, coupling, and readout fields, enabling the retrieval field to be dynamically controlled by manipulating the spatial frequency components of the interacting light fields. In this paper, according to the EIT effect, we achieve experimentally the optical image addition and subtraction in the imaging plane of a 4<inline-formula><tex-math id="M2">\begin{document}$f$\end{document}</tex-math><alternatives><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="9-20221560_M2.jpg"/><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="9-20221560_M2.png"/></alternatives></inline-formula> system through the coherent manipulation of the retrieval field via the interacting signal, coupling, and readout fields. Specifically, we first store the spatial frequency spectrum of a double-slit mask in the praseodymium-doped yttrium silicate crystal located in the confocal plane of the 4<inline-formula><tex-math id="M3">\begin{document}$f$\end{document}</tex-math><alternatives><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="9-20221560_M3.jpg"/><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="9-20221560_M3.png"/></alternatives></inline-formula> system based on the EIT effect. Then, we utilize a specially designed mask containing the target objects and perform spatial filtering by using the spatial frequency spectrum of the double-slit mask through the retrieval of stored light pulse. By moving the double-slit mask, the addition and subtraction between the images of target objects can be achieved in the imaging plane of the 4<inline-formula><tex-math id="M4">\begin{document}$f$\end{document}</tex-math><alternatives><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="9-20221560_M4.jpg"/><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="9-20221560_M4.png"/></alternatives></inline-formula> system. We present a theoretical model to describe the addition and subtraction between the target images through the EIT-based light pulse storage technique. The experimental results accord well with the theoretical prediction. Compared with the traditional scheme with a cosine grating filtering the spatial frequency spectrum, our method does not require the preparation of a spatial frequency filter, and only produces the first-order positive and negative diffraction images without the influence of the zeroth-order diffraction image. The optical image addition and subtraction based on the EIT effect provide a new approach to optical image processing. This approach is not limited to image addition or subtraction, and it can be extended to achieve more complex operations such as differentiation, enhancement, encryption, and decryption with rational design. Therefore, it can be widely used in areas such as coherent manipulation of light fields and dynamic optical image processing.

Microscopic strain mapping in polymers equipped with non-covalent mechanochromic motifs.

The mechanical failure of polymers remains challenging to understand and predict, as it often involves highly localised phenomena that cannot be probed with bulk characterisation techniques. Here, we present a generalisable protocol based on optical microscopy, tensile testing, and image processing that permits the spatially resolved interrogation of mechanical deformation at the molecular level around defects in mechanophore-containing polymers. The approach can be applied to a broad range of polymeric materials, mechanophores, and deformation scenarios.

Self-healing of holographically generated moiré lattice wave fields

Self-healing in optics generally refers to the ability to reconstruct itself and restore the original state after encountering obstacles in the propagation of the light field. In this research, we observe the processes of the wave fields from perfect to defect in front of the focal plane of the 4f system, finally returning to an intact situation after the plane. According to simulations and experimental results, there is a minimum self-healing distance for the moiré lattice field that positively associates with the radius of the defect (obstacle) in the nondiffracting transmission range. Furthermore, it is observed that the defect self-healing is a process of “repairing the center and then repairing the edges.” These findings can be applied in areas such as optical imaging, capture, and information processing.

Optimization for Anchor-Free Object Detection via Scale-Independent GIoU Loss

Arbitrary-oriented target detection is widely used in optical remote-sensing image processing, and there have been lots of anchor-based detectors using horizontal bounding boxes. However, the image targets of various scales and shapes make it difficult to tune optimal anchor parameters, whereas the complex background and nonmaximum suppression (NMS) require well-aligned bounding box to predict dense targets. In this letter, a scale-independent IoU (SIoU) loss is proposed for bounding box regression, which can adaptively adjust the shape of predicted boxes and speed up the convergence. Besides, the regression branch of the fully convolutional one-stage object detector (FCOS) is refined to implement the novel intersection over union (IoU) loss for rotated bounding box regression. Extensive experiments on HRSC2016 and a large-scale dataset for object detection in aerial images (DOTA) show that our method obtains 88.1% mean average precision (mAP) under an IoU threshold of 0.5 on HRSC2016, which is 1.1% higher than generalized IoU (GIoU) loss and 0.7% than complete IoU (CIoU) loss.

Desorption electrospray ionization mass spectrometry imaging (DESI-MSI) in disease diagnosis: an overview.

Tissue analysis, which is essential to histology and is considered the benchmark for the diagnosis and prognosis of many illnesses, including cancer, is significant. During surgery, the surgical margin of the tumor is assessed using the labor-intensive, challenging, and commonly subjective technique known as frozen section histopathology. In the biopsy section, large numbers of molecules can now be visualized at once (ion images) following recent developments in [MSI] mass spectrometry imaging under atmospheric conditions. This is vastly superior to and different from the single optical tissue image processing used in traditional histopathology. This review article will focus on the advancement of desorption electrospray ionization mass spectrometry imaging [DESI-MSI] technique, which is label-free and requires little to no sample preparation. Since the proportion of molecular species in normal and abnormal tissues is different, DESI-MSI can capture ion images of the distributions of lipids and metabolites on biopsy sections, which can provide rich diagnostic information. This is not a systematic review but a summary of well-known, cutting-edge and recent DESI-MSI applications in cancer research between 2018 and 2023.

Research advances in simple and compact optical imaging techniques

Computational imaging enables optical imaging systems to acquire more information with miniaturized setups. Computational imaging can avoid the object-image conjugate limitation of the imaging system, and introduce encoding and decoding processes based on physical optics to achieve more efficient information transmission. It can simultaneously increase the amount of information and reduce the complexity of the system, thereby paving the way for miniaturizing imaging systems. Based on computational imaging, the simple and compact optical imaging techniques are developed, which is also called simple optics. To develop miniaturized optical imaging elements and integrated systems, simple optics utilizes the joint design of optical system and image processing algorithms, thereby realizing high-quality imaging that is comparable to complex optical systems. The imaging systems are of small-size, low-weight, and low-power consumption. With the development of micro-nano manufacturing, the optical elements have evolved from a single lens or a few lenses, to flat/planar optical elements, such as diffractive optical elements and metasurface optical elements. As a result, various lensless and metalens imaging systems have emerged. Owing to the introduction of encoding process and decoding process, an optical imaging model is developed to represent the relationship between the target object and the acquired signal, from which the computational reconstruction is used to restore the image. In the image restoration part, the algorithms are discussed in three categories, i.e. the classic algorithm, the model-based optimization iterative algorithm, and the deep learning (neural network) algorithm. Besides, the end-to-end optimization is highlighted because it introduces a new frame to minimize the complexity of optical system. In this review, the imaging techniques realized by simple optics are also discussed, such as depth imaging, high-resolution and super-resolution imaging, large field of view imaging, and extended depth of field imaging, as well as their important roles in developing consumer electronics, unmanned driving, machine vision, security monitoring, biomedical devices and metaverse. Last but not least, the challenges and future developments are prospected.

A reference methodology for microplastic particle size distribution analysis: Sampling, filtration, and detection by optical microscopy and image processing

AbstractMicroplastic (MP) contamination in natural water circulation is a concern for environmental issues and human health. Various types of polymer materials have been identified and were detected in MP analytic test procedures. Beyond MP polymer type, particle size and form play a major role in water analysis due to possible negative toxicologic effects on flora and fauna. However, the correct quantitative measurement of MP size distribution over several orders of magnitude is strongly influenced by sample preparation, filtration materials and processes, and microanalytical techniques, as well as data acquisition and analysis. In this paper, a reference methodology is presented aiming at an improved quantitative analysis of MP particles. An MP analysis workflow is demonstrated including all steps from reference materials to sample preparation, filtration handling, and MP particle size distribution analysis. Background‐corrected particle size distributions (1–1000 µm) have been determined for defined polyethylene (PE) and polyethylene terephthalate (PET) reference samples. Microscopically measured particle numbers and errors have been cross‐checked with the total initial mass. In particular, defined reference MP samples (PE, PET) are initially characterized and applied to filtration experiments. Optical microscopy imaging on full‐area Si filters with subsequent image analysis algorithms is used for statistical particle size distribution analysis. To quantify the effects of handling and filtration, several blind tests with distilled water are carried out to determine the particle background for data evaluation. Particle size distributions of PE and PET reference samples are qualitatively and quantitatively reproduced with respect to symmetry, and maximum and cut‐off diameter of the distribution. It is shown that especially MP particles with a radius of >50 µm can be detected and retrieved with high reliability. For particle sizes <50 µm, a significant interference with background contamination is observed. Data from blank samples allows a correction of background contaminations. Furthermore, for enhanced sampling statistics, the recovery of the initial amount of MP will be qualitatively shown. The results are intended as an initial benchmark for MP analytics quality. This quality is based on statistical MP particle distributions and covers the complete analytic workflow starting from sample preparation to filtration and detection. Microscopic particle analysis provides an important supplement for the evaluation of established spectroscopic methods such as Fourier‐transform infrared spectroscopy or Raman spectroscopy.

A Cloud Detection Method Based on Spectral and Gradient Features for SDGSAT-1 Multispectral Images

Due to the limited penetration of visible bands, optical remote sensing images are inevitably contaminated by clouds. Therefore, cloud detection or cloud mask products for optical image processing is a very important step. Compared with conventional optical remote sensing satellites (such as Landsat series and Sentinel-2), sustainable development science Satellite-1 (SDGSAT-1) multi-spectral imager (MII) lacks a short-wave infrared (SWIR) band that can be used to effectively distinguish cloud and snow. To solve the above problems, a cloud detection method based on spectral and gradient features (SGF) for SDGSAT-1 multispectral images is proposed in this paper. According to the differences in spectral features between cloud and other ground objects, the method combines four features, namely, brightness, normalized difference water index (NDWI), normalized difference vegetation index (NDVI), and haze-optimized transformation (HOT) to distinguish cloud and most ground objects. Meanwhile, in order to adapt to different environments, the dynamic threshold using Otsu’s method is adopted. In addition, it is worth mentioning that gradient features are used to distinguish cloud and snow in this paper. With the test of SDGSAT-1 multispectral images and comparison experiments, the results show that SGF has excellent performance. The overall accuracy of images with snow surface can reach 90.80%, and the overall accuracy of images with other surfaces is above 94%.

Morphological characterization of chip segmentation in Ti-6Al-7Nb machining: A novel method based on digital image processing

α + β titanium alloys, namely Ti-6Al-7Nb, have found growing importance in the biomedical industry, in which CNC machining plays a decisive role in the production of high-quality components. Metal chip analysis is a consolidated method for machining optimization, particularly in difficult-to-cut materials that tend to form saw-tooth/segmented chips. A novel automatic method for quantification of classical geometric aspects of Ti-6Al-7Nb alloy saw-tooth chips was developed. Geometric features, including the minimum and maximum chip thickness, as well as the saw-tooth width, were evaluated. These aspects were obtained by association of optical microscopy and digital image processing. The advantage of this methodology is the fully automated analysis of the geometric aspects of the chip, which are generally obtained manually. The robustness of the measurements was ensured by the selection of shape descriptors tested on a range of cutting speeds. This work also delivers a set of valuable experimental results about Ti-6Al-7Nb machining.

Novel Approach for Generalizing the Process Chain of Optical Satellite Images Based on Knowledge Capitalisation

Optical remote sensing from satellites such as SPOT, Landsat, Sentinel2, Terra Modis, offers exceptional solutions for monitoring the earth’s surface. Indeed, these satellites deliver multi-source, multi-date images. These are Bigdata, voluminous, heterogeneous and their treatment are not only complex but also require a lot of time. Studies have been carried out on the formalization of experimental protocols in satellite image processing. Those propose automated processing chains. But, these solutions are specific and their reuse are not possible for other cases. Our reflection allows us to propose a new approach for generalizing the optical satellite image processing chains. This approach respects standards in knowledge capitalization. The objective is to ensure the interoperability to the use of optical satellite images. Metamodels for the generalization of processing chains, a method of execution, generalization rules and implementation prototypes have been developed. These results are obtained through theoretical analyses and also experiments on the processing of multidate, multisource optical satellite images applied in multisectoral domains. Thus, this research solves today’s interrogability problem in the processing of satellite images optical.

An intelligent method for measuring high refractive index based on optical coherence tomography and image processing

This study introduces a new method to measure high refractive index (RI) materials with irregular shapes based on optical coherent tomography (OCT) images. Since the optical path length is the critical parameter to calculate RI, image fusion, thresholding, boundary extraction, and Hough transform were adopted in a well-organized sequence to extract this parameter when analyzing OCT images. Hough transform plays a key role in target recognition and parameter extraction. Glass and diamond were used as samples to test the validity of this method. The measurement result has reasonable consistency with the standard value. The method introduced in this paper has the potential to raise the automation and intelligence level of the existing research. A bright application prospect can be expected in industrial fields, such as optical glass, jewelry, and mineral substance.

Low-temperature ozone treatment for p-xylene perm-selective MFI type zeolite membranes: Unprecedented revelation of performance-negating cracks larger than 10 nm in polycrystalline membrane structures

We report that low-temperature ozone calcination allowed MFI type polycrystalline zeolite membranes to maximize their p-/o-xylene separation factor (as high as ca. 2000) by suppressing defect formation. Conventional high-temperature calcination and rapid thermal processing, which provide poor and marked p-/o-xylene separation abilities, respectively, were used for comparison. The corresponding defect structures were quantitatively analyzed by image processing of fluorescence confocal optical microscopy images combined with membrane permeation modeling, revealing the main defects (grain boundary defects and cracks) and their tortuosity, porosity, and size. To the best of our knowledge, we, for the first time, demonstrated that in contrast to common belief, the minor portion of wider cracks instead of the major portion of narrower grain boundary defects determined the final permeation rates. Specifically, the MFI membrane prepared by high-temperature calcination contained many grain boundary defects (narrow; ca. 1 nm) and few cracks (wide; ca. 20 nm) that accounted for ca. 0.1% and 99.8%, respectively, of the slowly permeating o-xylene molar flux. In contrast, the ozone-treated MFI membranes, which only possessed grain boundary defects, achieved the high p-/o-xylene separation performance, underlining the need for the selective reduction in the number of wider cracks rather than ubiquitous, narrow grain boundary defects.

Extended depth-of-field infrared imaging with deeply learned wavefront coding

Wavefront coding (WFC) techniques, including optical coding and digital image processing stages, enable significant capabilities for extending the depth of field of imaging systems. In this study, we demonstrated a deeply learned far-infrared WFC camera with an extended depth of field. We designed and optimized a high-order polynomial phase mask by a genetic algorithm, exhibiting a higher defocus consistency of the modulated transfer functions than works published previously. Additionally, we trained a generative adversarial network based on a synthesized WFC dataset for the digital processing part, which is more effective and robust than conventional decoding methods. Furthermore, we captured real-world infrared images using the WFC camera with far, middle, and near object distances. Their results after wavefront coding/decoding showed that the model of deeply learned networks improves the image quality and signal-to-noise ratio significantly and quickly. Therefore, we construct a novel artificial intelligent method of deeply learned WFC optical imaging by applying infrared wavelengths, but not limited to, and provide good potential for its practical application in "smart" imaging and large range target detection.

Defect corrections for coherent optical information processing of grayscale images in a DMD-based 4f-system using a collimated light source.

Digital micromirror device (DMD)-based 4f-systems, a type of coherent optical information processing system, have become a powerful tool for optical convolutional neural networks taking advantage of their fast modulation speed and high-resolution capability. However, proper high bit-depth image information processing remains challenging due to the optical diffractions that arise from the binary nature of DMD operation. In this paper, we first characterize the diffraction phenomena that cause irradiance defects, namely the nonlinear grayscale and unintended dark lines. Then to resolve the issues, we propose a DMD operation method and a modified structure of the 4f-system based on blazed diffraction grating theory and numerical calculation of the Rayleigh-Sommerfeld propagation model. As a demonstration, we implement high bit-depth image information processing with an optimized optical 4f-system using DMDs and a collimated coherent light source.

Advanced Fluorescence Microscopy Methods to Study Dynamics of Fluorescent Proteins In Vivo.

Fluorescent proteins are standard tools for addressing biological questions in a cell biology laboratory. The genetic tagging of protein of interest with fluorescent proteins opens the opportunity to follow them in vivo and to understand their interactions and dynamics. In addition, the latest advances in optical microscopy image acquisition and processing allow us to study many cellular processes in vivo. Techniques such as fluorescence lifetime microscopy and hyperspectral imaging provide valuable tools for understanding fluorescent protein interactions and their photophysics. Finally, fluorescence fluctuation analysis opens the possibility to address questions of molecular diffusion, protein-protein interactions, and oligomerization, among others, yielding quantitative information on the subject of study. This chapter will cover some of the more important advances in cutting-edge technologies and methods that, combined with fluorescent proteins, open new frontiers for biological studies.

A hybrid generative adversarial network for weakly-supervised cloud detection in multispectral images

Cloud detection is a crucial step in the optical satellite image processing pipeline for Earth observation. Clouds in optical remote sensing images seriously affect the visibility of the background and greatly reduce the usability of images for land applications. Traditional methods based on thresholding, multi-temporal or multi-spectral information are often specific to a particular satellite sensor. Convolutional Neural Networks for cloud detection often require labeled cloud masks for training that are very time-consuming and expensive to obtain. To overcome these challenges, this paper presents a hybrid cloud detection method based on the synergistic combination of generative adversarial networks (GAN) and a physics-based cloud distortion model (CDM). The proposed weakly-supervised GAN-CDM method (available online https://github.com/Neooolee/GANCDM) only requires patch-level labels for training, and can produce cloud masks at pixel-level in both training and testing stages. GAN-CDM is trained on a new globally distributed Landsat 8 dataset (WHUL8-CDb, available online doi:https://doi.org/10.5281/zenodo.6420027) including image blocks and corresponding block-level labels. Experimental results show that the proposed GAN-CDM method trained on Landsat 8 image blocks achieves much higher cloud detection accuracy than baseline deep learning-based methods, not only in Landsat 8 images (L8 Biome dataset, 90.20% versus 72.09%) but also in Sentinel-2 images (“S2 Cloud Mask Catalogue” dataset, 92.54% versus 77.00%). This suggests that the proposed method provides accurate cloud detection in Landsat images, has good transferability to Sentinel-2 images, and can quickly be adapted for different optical satellite sensors.