Optical Image Processing Research Articles

Dispersion engineered metasurfaces for broadband, high-NA, high-efficiency, dual-polarization analog image processing

Optical metasurfaces performing analog image processing – such as spatial differentiation and edge detection – hold the potential to reduce processing times and power consumption, while avoiding bulky 4 F lens systems. However, current designs have been suffering from trade-offs between spatial resolution, throughput, polarization asymmetry, operational bandwidth, and isotropy. Here, we show that dispersion engineering provides an elegant way to design metasurfaces where all these critical metrics are simultaneously optimized. We experimentally demonstrate silicon metasurfaces performing isotropic and dual-polarization edge detection, with numerical apertures above 0.35 and spectral bandwidths of 35 nm around 1500 nm. Moreover, we introduce quantitative metrics to assess the efficiency of these devices. Thanks to the low loss nature and dual-polarization response, our metasurfaces feature large throughput efficiencies, approaching the theoretical maximum for a given NA. Our results pave the way for low-loss, high-efficiency and broadband optical computing and image processing with free-space metasurfaces.

Optimizing wavefront coding for extended depth of field: a synchronous algorithm for optical element and decoding optimization.

Wavefront coding (WFC) combines phase mask design and image restoration algorithm to extend the depth of field (DOF) for various applications. However, discrete design limits finding globally optimal solutions, increasing the complexity of system design, and affecting the accuracy and robustness of image restoration. An end-to-end imaging system design has emerged to break through these limitations by integrating optical design and image processing algorithms. In this study, we propose an algorithm that synchronously optimizes the optical elements and decoding algorithm in WFC using ray-tracing simulation. We also derive formulas for the optical layer's forward and backward propagation for joint optimization of the optical layer and decoding algorithm. Experimental verification demonstrates the algorithm's effectiveness in optimizing the WFC system and offers improved performance under a unified design framework.

Generalized binary spiral zone plates with a single focus obtained by feedforward neural network.

Traditional spiral zone plates (SZPs) have been widely used to generate optical vortices, but this structure suffers from multiple focuses. To eliminate high-order foci, the current method is to design a binary structure that has a sinusoidal transmittance function along the radial direction. With the rapid development of artificial neural networks, they can provide alternative methods to design novel SZPs with a single focus. In this paper, we first propose the concept of generalized binary spiral zone plates (GBSZPs), and train a feedforward neural network (FNN) to obtain the mapping relationship between the relative intensity of each focus and the structural parameters of GBSZPs. Then the structural parameters of GBSZPs with a single focus were predicted by the trained FNN. It is found by simulations and experiments that the intensities of high-order foci can be as low as 0.2% of the required first order. By analyzing the radial transmittance function, it is found that this structure has a different distribution function from the previous radial sinusoidal function, which reveals that the imperfect radial sinusoidal form also can guide the design of binary zone plates to eliminate high-order foci diffraction. These findings are expected to direct new avenue towards improving the performance of optical image processing and quantum computation.

Generating tunable multi-orbital angular momentum mode vortex beams by intracavity edge diffraction modulation

In the past three decades, vortex beams (VBs) with orbital angular momentum (OAM) have attracted extensive attention in the fields of optical manipulation, image processing, optical communication, and optical measurement. Here, we demonstrate the generation of multi-OAM mode VBs using an intracavity diffraction edge modulation method by selectively “clipping” the high-order modes of the optical field through variable aperture diaphragms and variable slits. Multi-OAM mode VBs with 2, 3, 4, 6, 8, and 10 vortices are generated, and the phase singularities in vortices are verified through interference experiments. We propose the principle of intracavity mode superposition and obtain the corresponding intensity and phase profiles by simulation. The generated VBs can maintain stable OAM propagation within the range of 83 mm. This work breaks the traditional understanding of generating multi-OAM mode VBs and provides guidance for the design of special vortex laser devices.

Computer Vision and Image Processing in Structural Health Monitoring: Overview of Recent Applications

Structural deterioration is a primary long-term concern resulting from material wear and tear, events, solicitations, and disasters that can progressively compromise the integrity of a cement-based structure until it suddenly collapses, becoming a potential and latent danger to the public. For many years, manual visual inspection has been the only viable structural health monitoring (SHM) solution. Technological advances have led to the development of sensors and devices suitable for the early detection of changes in structures and materials using automated or semi-automated approaches. Recently, solutions based on computer vision, imaging, and video signal analysis have gained momentum in SHM due to increased processing and storage performance, the ability to easily monitor inaccessible areas (e.g., through drones and robots), and recent progress in artificial intelligence fueling automated recognition and classification processes. This paper summarizes the most recent studies (2018–2022) that have proposed solutions for the SHM of infrastructures based on optical devices, computer vision, and image processing approaches. The preliminary analysis revealed an initial subdivision into two macro-categories: studies that implemented vision systems and studies that accessed image datasets. Each study was then analyzed in more detail to present a qualitative description related to the target structures, type of monitoring, instrumentation and data source, methodological approach, and main results, thus providing a more comprehensive overview of the recent applications in SHM and facilitating comparisons between the studies.

Three-dimensional modelling of blur property for conventional optical microscopes

Intensity diffusion caused by optical diffraction limits the imaging resolution of conventional optical microscopes, therefore modelling and measuring the intensity transmission and distribution property of the light sources is a significant research topic in system development and pattern recognition. However, the complicated wave propagation process in optical imaging makes it difficult to provide a direct, analytical and simple mathematical model to measure the relationship between the blur degree and various camera parameters. In this study, an improved intensity transmission and distribution calculation method for conventional optical microscopes was proposed; furthermore, a simple mathematical relation between the blur degree and camera parameters was achieved based on the proposed method. First, the light intensity distribution and propagation characteristics of a conventional optical microscope were modeled based on the property of the Fresnel circular hole diffraction combined with the practical optical parameters. Second, by analyzing the property of intensity distribution and blurring imaging, a quantitative simplified mathematical relationship between the blur degree and camera parameters in optical microscope imaging was obtained, and the three-dimensional (3D) blur property in the optical imaging process was analyzed under different conditions. Third, the connection between diffractive optics and geometric optics was obtained by summarizing and generalizing the 3D blur property curve of each monochromatic light source. Finally, the proposed method was verified through a series of simulations and experiments.

3D Reconstruction and morphology analysis of coarse aggregate using optical laser triangulation and image processing technology

Digital image processing is commonly used in civil engineering to measure aggregate morphologies. However, it has limitations due to laying style and acquisition method, restricting analysis to two-dimensional geometric features. To overcome this, a new laser scanning-based method is proposed to reconstruct three-dimensional features of coarse aggregates. Three-dimensional morphology indexes are developed, with promising results. This method effectively captures complete surface information and the ratios between axes, surface area rate, and volume rate describe shape, texture, and angularity well. The proposed methodology accurately characterizes coarse aggregate morphologies, aiding asphalt mixture design. Incorporating the developed indexes enables effective quality control of asphalt pavements.

Engine combustion and emission analysis using optical methods: An overview

Measurements in combustion places important role in design of reciprocating and turbo hot engines. The available methods of combustion analysis of engines have gained much attention and are capable of giving good results in determining the fuel characteristics and engine design optimizations. However, the full understanding of engine combustion can be made through optical image processing in addition to the existing methods. Optical tools are used efficiently to predict the combustion phenomena in internal combustion engine. In order to observe typical regimes of combustion most widely used techniques are Schlieren and shadowgraph. The devices used in optical measurements are high speed CCD camera and Constant Volume Spray Chambers (CVSCs). The analysis is carried to know the spray characteristics at atmospheric and elevated pressure conditions.The luminosity techniques are widely used to capture the combustion flames with high speed camera in optically accessible engines. The present engines have to be redesigned for making optical access for the desired measurements. Several endoscopy techniques are used for combustion visualization. Phase Doppler Interferometer (PDI) is used for the quantitative spray measurements. Particle Image Velocimetry is an intrusive technique which is capable of characterizing instantaneous velocity fields in a fluid flow. This paper presents the works on different optical techniques available and is in use to diagnose the engine combustion especially for reciprocating engines.

Near-lifespan longitudinal tracking of brain microvascular morphology, topology, and flow in male mice

In age-related neurodegenerative diseases, pathology often develops slowly across the lifespan. As one example, in diseases such as Alzheimer’s, vascular decline is believed to onset decades ahead of symptomology. However, challenges inherent in current microscopic methods make longitudinal tracking of such vascular decline difficult. Here, we describe a suite of methods for measuring brain vascular dynamics and anatomy in mice for over seven months in the same field of view. This approach is enabled by advances in optical coherence tomography (OCT) and image processing algorithms including deep learning. These integrated methods enabled us to simultaneously monitor distinct vascular properties spanning morphology, topology, and function of the microvasculature across all scales: large pial vessels, penetrating cortical vessels, and capillaries. We have demonstrated this technical capability in wild-type and 3xTg male mice. The capability will allow comprehensive and longitudinal study of a broad range of progressive vascular diseases, and normal aging, in key model systems.

Micro-Scale Surface Recognition via Microscope System Based on Hu Moments Pattern and Micro Laser Line Projection

The surface engineering of metals develops high technology to detect microscale convex, concave and flat surface patterns. It is because the manufacturing industry requires technologies to recognize microscale surface features. Thus, it is necessary to develop microscopic vision technology to recognize microscale concave, convex and flat surfaces. This study addresses microscale concave, convex and flat surface recognition via Hu moments’ patterns based on micro-laser line contouring. In this recognition, a Hu moments’ pattern is generated from a Bezier model to characterize the surface recovered through microscopic scanning. The Bezier model is accomplished by employing a genetic algorithm and surface coordinates. Thus, the flat, convex and concave surfaces are recognized based on the Hu moments’ pattern of each one. The microscope system projects a 40 μm laser line on the object and a camera acquires the object’s contour reflection to retrieve topographic coordinates. The proposed technique enhances the microscale convex, concave, flat, and surface recognition, which is performed via optical microscope systems. The contribution of microscopic shape recognition based on the Hu moments’ pattern and microscopic laser line is elucidated by a discussion based on the microscopic shape recognition performed through the optical microscopic image processing.

Prediction of Disease from Blood Microscopic Analysis Classification using Deep Learning Algorithm

Traditional blood disease detection based on visual inspection of blood smears using a microscope is time consuming, error-prone, and limited by the physical acuity of the hematologist. To facilitate clinical decision-making, an automated optical image processing system is required. Leukemia is a kind of cancer that is distinguished by the aberrant development of immature, abnormally shaped white blood cells (WBC) known as "blasts." Leukemia is a cancer of the white blood cells (WBCs) that affects the bone marrow and/or blood. A timely, safe, and accurate diagnosis of leukemia at an early stage is critical to treating and preserving patients' lives. The white blood cells in a blood smear are often examined under a microscope to determine the diagnosis.Many machine learning have been created to diagnose various illnesses, such as leukemia, and to deliver a high mis-classification error rate. As a result, we may use a deep learning system to identify microscope pictures for White Blood Cell Count study. The WBC differential counting system was divided into two modules: detection and classification. The detection module first processed the raw bone smear pictures, detecting all WBCs including red blood cells, platelet counts, staining impurities, and so on. The discovered cells were then sent into the categorization module.The categorization module was divided into two phases. In the first step, we separated a large number of cells, such as crushed cells, degenerated cells, and so on, that are not employed in the diagnosis for leukemia. The countable WBCs were then presented for multi-class separation using the Convolutional neural network technique in the second step

Tunable nonlocal metasurfaces based on graphene for analogue optical computation

Meta-optical devices have recently emerged as ultra-compact candidates for real-time computation in the spatial domain. The use of meta-optics for applications in image processing and wavefront sensing could enable an order of magnitude increase in processing speed and data throughput, while simultaneously drastically reducing the footprint of currently available solutions to enable miniaturisation. Most research to date has focused on static devices that can perform a single operation. Dynamically tunable devices, however, offer increased versatility. Here we propose graphene covered subwavelength silicon carbide gratings as electrically tunable optical computation and image processing devices at mid-infrared wavelengths.

Improved design method for suppressing optical focal spots and experimental verification

Pupil filtering is an effective method to modulate the 3D point spread function distribution of an optical system. Based on vector Debye-Wolf diffraction integral and a new fusion optimization algorithm, an improved method is described for efficiently designing the most widely-used concentric multi-belt pupil filters. The main advantage of this method is that it actively selects the minimum belt width and the total belt number for facilitating low-cost batch processing. Pupil filters for visible and near-infrared bands have been separately designed for improving the spatial resolution of optical imaging and ultrafast laser processing. The method has been experimentally demonstrated for suppressing the optical focal spot. The proposed method can be readily modified for designing more sophisticated multi-level pupil filters for practical applications in high-resolution imaging and fine laser micromachining.

Optical coherence tomography and digital image processing for scaling and Co-precipitation investigation on reverse osmosis membrane

Scaling known as the precipitation of inorganic salts is a main concern in the reverse osmosis (RO) process due to its effects on the process performance and life-time of membranes. Although single salt precipitation has been continuously studied, the co-precipitation of salts is rarely investigated due to its complexity, especially under process hydrodynamic conditions. Recently, optical coherence tomography (OCT) has been adopted as an in situ and rapid technique for monitoring the fouling in membrane processes. This study aims to develop an approach using OCT and digital image processing for qualitative and quantitative analysis of single salt precipitation and co-precipitation on RO membrane under hydrodynamic conditions. The effects of supersaturation, sodium chloride (NaCl), flow rate, and applied pressure are investigated. The developed approach enables the detection of crystal growth under the OCT. The morphology of precipitated salt, i.e CaSO4, including crystal shape and size obtained from OCT acquisition, agrees with the images from a scanning electron microscope (SEM). The growth of each crystal in co-precipitation of CaSO4 and CaCO3 is well detected in appearance time and growth pattern. This is beneficial to differentiate crystal morphology in the co-precipitation system. The quantitative surface coverage proportion and total volume are in well correlated the changes of RO performance parameters of permeate flux, conductivity (feed, concentrate, and permeate), and Ca2+ concentrate profile by time, which allows the scaling progress to be directly evaluated over time. This study reveals the high potential of combining OCT acquisition and digital image processing for direct observations and studying single salt and co-precipitation under RO hydrodynamic conditions, which can be extended for another type of composite fouling.

Metasurface with Directional‐Controlled Asymmetric Transmissions

The diverse design freedom and exotic electromagnetic properties of metasurfaces enable them to achieve unprecedented capability and superiority with regard to conventional optical elements. Herein, a bifunctional microwave metasurface with an asymmetric bidirection transmission phase that can be controlled is reported. The proposed metasurface is composed of bilayered double‐arrow‐shaped structures sandwiching a subwavelength metal grating. Such design can achieve not only linear polarization (LP) conversion for two oppositely directed wave incidences but also phase control independently by introducing a rotation and adjusting the geometrical parameters of the double‐arrow‐shaped structures. The metasurface breaks the symmetry and realizes different functions for opposite propagation directions, such as deflection and focusing, orbital angular momentum (OAM) mode generation, and holographic imaging. The experimental results suggest that the proposed Janus metasurface can be further exploited as an attractive platform for various applications ranging from microwave reconfigurable beam antenna and multifunctional polarization control to optical imaging, sensing, and information processing.

The use of a neural network in solving problems of recognition and classification of spacecraft by their optical images

In recent years, the number of space objects located in near-Earth outer space, especially in the near operational zone, has increased significantly due to the build-up of space groupings, including dual-use (for example, Starlink) and the remnants of their vital activity (space debris). This factor increases the importance of the task of recognizing and classifying space objects by type in the shortest possible time and entering them into the main catalog of space objects. The developed methodology allows automated analysis of optical images of space objects using software to solve the problem of their recognition and classification by type using a convolutional neural network. The purpose of the study is to increase the efficiency of processing and analysis of optical images of spacecraft. The experimental results of the study confirm the achievement of the research goal. The developed methodology contributes to the development of software and hardware for image processing and can be used in calculations and data preparation for information support of interested officials. For the first time, a training set for a convolutional neural network has been prepared using real optical images of spacecraft obtained in the visible range.

Developing an intelligent cloud attention network to support global urban green spaces mapping

Urban green spaces (UGS) play an important role in understanding of urban ecosystems, climate, environment, and public health concerns. Satellite derived UGS maps provide an efficient and effective tool for urban studies and contribute to targets and indicators of the sustainable development goals, at the global level, set by the United Nations. However, clouds create a challenging issue in optical satellite image processing, leading to significant uncertainty in UGS mapping. In this study, we propose an automatic UGS mapping method by integrating satellite images with crowdsourced geospatial data while aiming to reduce the uncertainty caused by cloud contamination. The proposed method consists of three parts: (1) auxiliary data pre-processing module; (2) cloud attention intelligent network (CAI-net); and (3) non-cloud scenes classification module. The auxiliary data pre-processing module was used to convert crowdsourcing geospatial data into auxiliary maps. The CAI-net was proposed to retrieve detailed UGS classes within clouds from satellite image patches and auxiliary maps, while non-cloud scenes classification module was used to extract UGS from satellite image patches. The proposed method was applied to generate spatial continuous global UGS map products, considering the uncertainty caused by cloud contamination. The results show the proposed method yielded a high-quality global UGS map with average overall accuracy as high as 92.96% when satellite images had cloud coverage ranging from 0% to 50%. The geospatial AI, specifically CAI-net, can provide more accurate UGS mapping regardless of different geographical and climatic conditions of the study areas, which is especially significant for humid tropical and subtropical regions with frequent clouds and rains.

Self-powered dual-wavelength polarization-sensitive photodetectors based on ZnO/BiFeO3 heterojunction

The ferroelectric-photovoltaic (FE-PV) devices have received growing attention for the FE polarization-induced internal electric field to facilitate the separation and transport of the photogenerated electron-hole (e-h) pairs in the FE semiconductors. However, the low photocurrent output of the FE-PV devices is still a key challenge for the application and development of these devices. Here, ZnO/BiFeO3 (BFO) heterojunction photodetectors (PDs) are designed to utilize the built-in electric field at the heterojunction interface of ZnO nanoparticles (NPs)/BFO and the spontaneous FE polarization field of BFO to promote the separation and transport of the photogenerated carriers. ZnO nanorod arrays (NRs) grown on ZnO NPs provide a directional transport channel for the photogenerated electrons. ZnO/BFO heterojunction devices demonstrate a self-powered photoresponse in the ultraviolet (UV) and visible (vis) dual-wavelength. The devices show a responsivity of 1.32 mA/W, response time (τrise/τdecay) of 12.6/44.9 ms at 420 nm light illumination. In addition, the polarization imaging function of ZnO/BFO PDs is explored by extending the polarization photocurrent mapping values of a single PD unit to a complex array of optical information image processing utilizing the polarization-sensitive properties of BFO without external bias.

Inclusion analysis in a dual-phase automotive strip steel

ABSTRACT A holistic assessment of inclusions present within a cold rolled DP800 steel was performed. The assessment comprised a variety of methods including optical microscopy and image processing, alongside quantitative, statistical analysis. The ‘statistics of extreme values’ approach was applied, utilising outputs from microstructural analysis. A qualitative analysis followed via the use of SEM and EDS to determine the morphologies and compositions of inclusions within the material. This has allowed for cross-coil examination; identifying the variation in behaviour of inclusions through the width of a cold rolled strip, which can be correlated with slab solidification as well as final product performance. This understanding is also critical in informing sample extraction for microstructural characterisation and mechanical property assessments. The research has provided an accelerated methodology of inclusion analysis that can be applied to DP800 effectively, allowing for manufacture-microstructure relationships to be determined.

Experimental application and accuracy assessment of 2D-DIC in meso-direct-shear test of sandy soil

The examination of the meso-mechanical properties of soils is fundamental to understand its macro-behaviour. This paper aims to evaluate the potential application of Digital Image Correlation (DIC), an optical image processing approach, in direct shear test (DST) for soils. Hence, a new shear box was designed to investigate the close behaviour of granular materials. The noise-floor and shear displacement accuracy were discussed to assess the reliability of DIC, besides the effectiveness of the new box was also examined. The distribution of the displacements, strains, and shear angle were measured under four different normal stresses. From the results, the immediate settlement, and the dilative behaviour of the sand during shearing were observed. Furthermore, the investigation revealed a strain concentration at the interface between the boxes, where a shear band was formed. The use of DIC opens up new ways in soil mechanics, overcoming some limitations of the conventional DST.