Edge-enhanced Images Research Articles

Nonlocal Huygens’ meta-lens for high-quality-factor spin-multiplexing imaging

Combining bright-field and edge-enhanced imaging affords an effective avenue for extracting complex morphological information from objects, which is particularly beneficial for biological imaging. Multiplexing meta-lenses present promising candidates for achieving this functionality. However, current multiplexing meta-lenses lack spectral modulation, and crosstalk between different wavelengths hampers the imaging quality, especially for biological samples requiring precise wavelength specificity. Here, we experimentally demonstrate the nonlocal Huygens’ meta-lens for high-quality-factor spin-multiplexing imaging. Quasi-bound states in the continuum (q-BICs) are excited to provide a high quality factor of 90 and incident-angle dependence. The generalized Kerker condition, driven by Fano-like interactions between q-BIC and in-plane Mie resonances, breaks the radiation symmetry, resulting in a transmission peak with a geometric phase for polarization-converted light, while unconverted light exhibits a transmission dip without a geometric phase. Enhanced polarization conversion efficiency of 65% is achieved, accompanied by a minimal unconverted value, surpassing the theoretical limit of traditional thin nonlocal metasurfaces. Leveraging these effects, the output polarization-converted state exhibits an efficient wavelength-selective focusing phase profile. The unconverted counterpart serves as an effective spatial frequency filter based on incident-angular dispersion, passing high-frequency edge details. Bright-field imaging and edge detection are thus presented under two output spin states. This work provides a versatile framework for nonlocal metasurfaces, boosting biomedical imaging and sensing applications.

A novel deep unsupervised approach for super-resolution of remote sensing hyperspectral image using gompertz-function convergence war accelerometric-optimization generative adversarial network (GF-CWAO-GAN)

Hyperspectral remote sensing images obtained from cameras are characterized by high-dimensions and low quality, which makes them unfavorable for various analytics purposes. This is due to the presence of visible and invisible frequencies of the reflected light making it poorly reveal the spectral signatures of the image. Visual communication advancement has paved the need for Image Super-Resolution (SR) which recovers high-resolution images from low-resolution images. Several works were carried out earlier on image SR using variants of supervised and unsupervised models that still lack accuracy. In this paper, we propose an unsupervised learning model titled Gompertz Function–based Convergence War Accelerometric Optimization–GAN framework for generating of High-Resolution (HR) images. The framework comprises a pre-processing stage, where the incoming Low-Resolution (LR) image is preprocessed for noise removal by applying Shannon-Gaussian Filter (S-GF). Following is the Gradient Domain Approach based Tone-Mapping (TM). Skew correction is done to remove distortion and maintain original resolution that may change during TM stage. The next stage comprises the boundary and edge enhancement of the resulting preprocessed image generated by the method of Inverse Gradient Mapping (IGM) followed by patch extraction to extract minute low-frequency information from the resulting boundary and edge-enhanced image. The contrast of the enhanced patches is improved by removing blurriness effect. The preprocessed image patches are then fed into the Gompertz Function-based Convergence War Accelerometric Optimization – GAN for feature mapping on the trained SR Image features that are clustered using Krzanowski and Li- Kantorovich Metric-K-Means clustering Algorithm (KL-KM-KMA) for effective generation of SR image. The developed model is validated for both qualitative and quantitative measurements. Comparisons are made with several other state-of -the-art methods for accuracy of 98.05%, precision of 97.98%, inception score of 8.71, Fréchet Inception Distance of 36.4 with reduced clustering and training time proving the efficiency of the proposed model.

Mid-infrared edge-enhanced imaging via angle-selective nonlinear filtering.

We propose a novel, to the best of our knowledge, scheme for mid-infrared upconversion imaging with high tunability between bright-field and edge-enhanced modalities. The involved engineering of the nonlinear process favors shaping the optical transfer function of the imaging system. Consequently, a nonlinear angle-selective filter can be configured to perform an all-optical Fourier processing of the image, which highly depends on phase-matching parameters. We numerically demonstrate the ability to switch modalities between the bright-field and edge-enhanced imaging by tuning the crystal temperature and simultaneously acquiring both information by dichromatic illumination. Notably, the achieved reconfigurability is realized without changing the imaging settings, which contrasts with previous instantiations based on pump adaptation. Therefore, the proposed architecture of upconversion imagers would pave a novel way to implement layout-compact and all-optical processing for infrared images.

Polarization-multiplexed zoom Moiré metalens for edge-enhanced imaging.

Optical image processing with high operational efficiency has been applied as a pre-processing imaging system for image recognition. Edge-enhanced imaging as a high-efficiency optical image processing method is of great significance for feature extraction and target recognition. However, the edge-enhanced imaging system based on the 4F system and the spatial filter transforms mainly work under coherent light illumination conditions, without continuously zooming to track the spatial position of the target. Here, we demonstrate a polarization-multiplexed zoom Moiré metalens for edge-enhanced imaging under incoherent light illumination. Metalens is designed to generate polarization-dependent optical transfer functions that produce edge-enhanced images with a resolution of 1.2 µm by digital subtraction. Furthermore, continuous zoom at the range of 1-2× is realized by constructing a Moiré metalens composed of cascaded metasurfaces. The cascaded metasurfaces consist of two center-aligned dielectric metasurfaces, each with a Moiré phase sensitive to the rotation angle. By rotating the metasurface, the phase profile of the cascaded metasurfaces changes, and the effect of continuous zoom is realized. The focal length can be actively changed from 38 µm to 77 µm with the focusing efficiency of 50.3%. This metalens can be applied to machine vision, microscopic imaging, and promotes the development of multi-functional integrated optical systems.

Arbitrary Tunable Edge-Enhanced Imaging Based on Curvilinear Vortex Filter

Arbitrary Tunable Edge-Enhanced Imaging Based on Curvilinear Vortex Filter

Visual augmentation of live-streaming images in virtual reality to enhance teleoperation of unmanned ground vehicles

First-person view (FPV) technology in virtual reality (VR) can offer in-situ environments in which teleoperators can manipulate unmanned ground vehicles (UGVs). However, non-experts and expert robot teleoperators still have trouble controlling robots remotely in various situations. For example, obstacles are not easy to avoid when teleoperating UGVs in dim, dangerous, and difficult-to-access areas with environmental obstacles, while unstable lighting can cause teleoperators to feel stressed. To support teleoperators’ ability to operate UGVs efficiently, we adopted construction yellow and black lines from our everyday life as a standard design space and customised the Sobel algorithm to develop VR-mediated teleoperations to enhance teleoperators’ performance. Our results show that our approach can improve user performance on avoidance tasks involving static and dynamic obstacles and reduce workload demands and simulator sickness. Our results also demonstrate that with other adjustment combinations (e.g., removing the original image from edge-enhanced images with a blue filter and yellow edges), we can reduce the effect of high-exposure performance in a dark environment on operation accuracy. Our present work can serve as a solid case for using VR to mediate and enhance teleoperation operations with a wider range of applications.

Cell classification with phase-imaging meta-sensors.

The development of photonic technologies for machine learning is a promising avenue toward reducing the computational cost of image classification tasks. Here we investigate a convolutional neural network (CNN) where the first layer is replaced by an image sensor array consisting of recently developed angle-sensitive metasurface photodetectors. This array can visualize transparent phase objects directly by recording multiple anisotropic edge-enhanced images, analogous to the feature maps computed by the first convolutional layer of a CNN. The resulting classification performance is evaluated for a realistic task (the identification of transparent cancer cells from seven different lines) through computational-imaging simulations based on the measured angular characteristics of prototype devices. Our results show that this hybrid optoelectronic network can provide accurate classification (>90%) similar to its fully digital baseline CNN but with an order-of-magnitude reduction in the number of calculations.

Fast selective edge-enhanced imaging with topological chiral lamellar superstructures.

Edge detection is a fundamental operation for feature extraction in image processing. The all-optical method has aroused growing interest owing to its ultra-fast speed, low energy consumption and parallel computation. However, current optical edge detection methods are generally limited to static devices and fixed functionality. Herein, we propose a fast-switchable scheme based on a ferroelectric liquid crystal topological structure. The self-assembled chiral lamellar superstructure, directed by the azimuthally variant photo-alignment agent, can be dynamically controlled by the polarity of the external electric field and respectively generates the vector beams with nearly orthogonal polarization distribution. Even after thousands of cycles, the horizontal and vertical edges of the object are selectively enhanced with an ultra-fast switching time of ∼57 μs. Broadband edge-enhanced imaging is efficiently demonstrated. This work extends the ingenious building of topological heliconical superstructures and offers an important glimpse into their potential in the emerging frontiers of optical computing for artificial intelligence.

Short-wavelength-infrared upconversion edge enhancement imaging based on a Laguerre-Gaussian composite vortex filter

Edge-enhanced imaging by spiral phase contrast has proven instrumental in revealing phase or amplitude gradients of an object, with notable applications spanning feature extraction, target recognition, and biomedical fields. However, systems deploying spiral phase plates encounter limitations in phase mask modulation, hindering the characterization of the modulation function during image reconstruction. To address this need, we propose and demonstrate an innovative nonlinear reconstruction method using a Laguerre-Gaussian composite vortex filter, which modulates the spectrum of the target. The involved nonlinear process spectrally transforms the incident short-wavelength-infrared (SWIR) signal from 1550 to 864 nm, subsequently captured by a silicon charge-coupled device. Compared with conventional schemes, our novel filtering method effectively suppresses the diffraction noise, significantly enhancing image contrast and resolution. By loading specific phase holograms on the spatial light modulator, bright-field imaging, isotropic, amplitude-controlled anisotropic, and directional second-order edge-enhanced imaging are realized. Anticipated applications for the proposed SWIR edge-enhanced imaging system encompass domains such as artificial intelligence recognition, deep tissue medical diagnostics, and non-destructive defect inspection. These applications underscore the valuable potential of our cutting-edge methodology in furthering both scientific exploration and practical implementations.

Polarization-independent edge detection based on the spin–orbit interaction of light

In previous edge detection schemes based on the spin-orbit interaction of light, the direction and intensity of the edge-enhanced images are influenced by the incident polarization state. In this study, we develop an edge detection strategy that is insensitive to changes in both the incident polarization and the incident angle. The output intensity and transfer function remain entirely impervious to changes in incident polarization, being explicitly formulated as functions of the incident angle, specifically in terms of cot2⁡θ i and cot⁡θ i , respectively. This behavior is attributed to the opposing nature of the polarization components E~ r H−H and E~ r V−V in the x-direction after undergoing mapping through the Glan polarizer, while the sum of polarization components E~ r H−V and E~ r V−H in the y-direction can be simplified to terms independent of incident polarization. Furthermore, we propose a metasurface design to achieve the required optical properties in order to realize the derived edge detection scheme.

Optical spatial differentiation enabled layer sensing of two-dimensional atomic crystals

Zero-thickness model and slab model are two important models in the description of optical behaviors in two-dimensional atomic crystals. The predicted difference in optical behaviors between the two models is very small, which is difficult to distinguish by established measurement methods. Here, we present an optical spatial differentiation method to examine the difference in edge images of different graphene layers. The theoretical results show that the edge imaging is significantly different between the two different models. When the beam reflection is at the Brewster angle, different graphene layers are used to adjust the spatial differentiation. It is shown that the slab model is more sensitive to the number of graphene layers. The zero-thickness model is more suitable for one-dimensional optical differential operation. Moreover, the spatial differentiation plays the role of a band-pass filter. The high-frequency edge information components will pass through the filter, thus realizing layer-sensitive edge-enhanced imaging. In addition, we do not focus on the verification of the exact model, but only provide an alternative method to characterize the number of graphene layers based on two models, and also provide possibilities for achieving imaging edge detection by graphene differential operators. This study may provide a possible method for the optical characterization of two-dimensional atomic crystals.

Compact meta-differentiator for achieving isotropically high-contrast ultrasonic imaging

Ultrasonic imaging is crucial in the fields of biomedical engineering for its deep penetration capabilities and non-ionizing nature. However, traditional techniques heavily rely on impedance differences within objects, resulting in poor contrast when imaging acoustically transparent targets. Here, we propose a compact spatial differentiator for underwater isotropic edge-enhanced imaging, which enhances the imaging contrast without the need for contrast agents or external physical fields. This design incorporates an amplitude meta-grating for linear transmission along the radial direction, combined with a phase meta-grating that utilizes focus and spiral phases with a first-order topological charge. Through theoretical analysis, numerical simulations, and experimental validation, we substantiate the effectiveness of our technique in distinguishing amplitude objects with isotropic edge enhancements. Importantly, this method also enables the accurate detection of both phase objects and artificial biological models. This breakthrough creates new opportunities for applications in medical diagnosis and nondestructive testing.

A dual-branch joint learning network for underwater object detection

Underwater object detection (UOD) is crucial for developing marine resources, environmental monitoring, and ecological protection. However, the degradation of underwater images limits the performance of object detectors. Most existing schemes treat underwater image enhancement (UIE) and UOD as two independent tasks, which take UIE as a preprocessing step to reduce the degradation problem, thus being unable to improve the detection accuracy effectively. Therefore, in this paper, we propose a dual-branch joint learning network (DJL-Net) that combines image processing and object detection through multi-task joint learning to construct an end-to-end model for underwater detection. With the dual-branch structure, DJL-Net can use the enhanced images generated by the image-processing module to supplement the features lost due to the degradation of the original underwater images. Specifically, DJL-Net first employs an image decolorization module governed by the detection loss, generating gray images to eliminate color disturbances stemming from underwater light absorption and scattering effects. An improved edge enhancement module is utilized to enhance the shape and texture expression in gray images and improve the recognition of object boundary features. Then, the generated edge-enhanced gray images and their original underwater images are input into the two branches to learn different types of features. Finally, a tridimensional adaptive gated feature fusion module is proposed to effectively fuse the complementary features learned from the two branches. Comprehensive experiments on four UOD datasets, including some scenes with challenging underwater environments, demonstrate the effectiveness and robustness of the proposed DJL-Net.

Electrically-switched differential microscopy based on computing liquid-crystal platforms.

Detection of transparent phase specimens especially biological cells with desired contrasts is of great importance in visual display and medical diagnosis. Due to the pure-phase nature, conventional detection approaches may damage samples or require complex operations. Computing liquid crystal (LC) is a thin and flat optical element with excellent capability in optical field modulation, which gives a feasible way to this issue from the perspective of analog optical computing. We here propose and experimentally implement an electrically switched two-dimensional (2D) differential microscopy based on computing LC platforms. The Pancharatnam-Berry phase LC polarization grating induces light's spin separation to promote the 2D differential operation. Using the electrically tunable LC plate as the system phase retardance provider, the detecting mode can be flexibly switched from bright-field images to edge-enhanced images with desired contrasts. Remarkably, owing to the wavelength-independent feature closely related to the geometric phases, our proposed scheme is demonstrated to be applicable to the multi-wavelength microscopy imaging. These results open avenues to form real-time all-optical image processing and may facilitate multifunctional differential microscopy.

Co-Mask R-CNN: collaborative learning-based method for tooth instance segmentation.

Traditional tooth image analysis methods primarily focus on feature extraction from individual images, often overlooking critical tooth shape and position information. This paper presents a novel computer-aided diagnosis method, Collaborative learning with Mask Region-based Convolutional Neural Network (Co-Mask R-CNN), designed to enhance tooth image analysis by leveraging the integration of complementary information. First, image enhancement is employed to generate an edge-enhanced tooth edge image. Then, a collaborative learning strategy combined with Mask R-CNN is introduced, where the original and edge images are input simultaneously, and a two-stream encoder extracts feature maps from complementary images. By utilizing an attention mechanism, the output features from the two branches are dynamically fused, quantifying the relative importance of the two complementary images at different spatial positions. Finally, the fused feature map is utilized for tooth instance segmentation. Extensive experiments are conducted using a proprietary dataset to evaluate the effectiveness of Co-Mask R-CNN, and the results are compared against those of an alternative segmentation network. The results demonstrate that Co-Mask R-CNN outperforms the other networks in terms of both segmentation accuracy and robustness. Consequently, this method holds considerable promise for providing medical professionals with precise tooth segmentation results, establishing a reliable foundation for subsequent tooth disease diagnosis and treatment.

Enhancing computational holography with spiral phase coding

In this Letter, we propose a new, to the best of our knowledge, approach to generate computer-generated holograms (CGHs) utilizing spiral phase coding. This method can be applied to generate an array spiral phase plate that can generate array vortex spots with a high compression ratio. Moreover, the method extends its applicability to the generation of Fresnel holograms and kinoforms, resulting in edge-enhanced imaging. Theoretical analysis and experimental results demonstrate the potential of spiral phase-encoded CGHs in laser processing and image enhancement.

Intelligent Phase Contrast Meta-Microscope System.

Phase contrast imaging techniques enable the visualization of disparities in the refractive index among various materials. However, these techniques usually come with a cost: the need for bulky, inflexible, and complicated configurations. Here, we propose and experimentally demonstrate an ultracompact meta-microscope, a novel imaging platform designed to accomplish both optical and digital phase contrast imaging. The optical phase contrast imaging system is composed of a pair of metalenses and an intermediate spiral phase metasurface located at the Fourier plane. The performance of the system in generating edge-enhanced images is validated by imaging a variety of human cells, including lung cell lines BEAS-2B, CLY1, and H1299 and other types. Additionally, we integrate the ResNet deep learning model into the meta-microscope to transform bright-field images into edge-enhanced images with high contrast accuracy. This technology promises to aid in the development of innovative miniature optical systems for biomedical and clinical applications.

Coarse-to-fine prior-guided attention network for multi-structure segmentation on dental panoramic radiographs

Objective. Accurate segmentation of various anatomical structures from dental panoramic radiographs is essential for the diagnosis and treatment planning of various diseases in digital dentistry. In this paper, we propose a novel deep learning-based method for accurate and fully automatic segmentation of the maxillary sinus, mandibular condyle, mandibular nerve, alveolar bone and teeth on panoramic radiographs. Approach. A two-stage coarse-to-fine prior-guided segmentation framework is proposed to segment multiple structures on dental panoramic radiographs. In the coarse stage, a multi-label segmentation network is used to generate the coarse segmentation mask, and in the fine-tuning stage, a prior-guided attention network with an encoder-decoder architecture is proposed to precisely predict the mask of each anatomical structure. First, a prior-guided edge fusion module is incorporated into the network at the input of each convolution level of the encode path to generate edge-enhanced image feature maps. Second, a prior-guided spatial attention module is proposed to guide the network to extract relevant spatial features from foreground regions based on the combination of the prior information and the spatial attention mechanism. Finally, a prior-guided hybrid attention module is integrated at the bottleneck of the network to explore global context from both spatial and category perspectives. Main results. We evaluated the segmentation performance of our method on a testing dataset that contains 150 panoramic radiographs collected from real-world clinical scenarios. The segmentation results indicate that our proposed method achieves more accurate segmentation performance compared with state-of-the-art methods. The average Jaccard scores are 87.91%, 85.25%, 63.94%, 93.46% and 88.96% for the maxillary sinus, mandibular condyle, mandibular nerve, alveolar bone and teeth, respectively. Significance. The proposed method was able to accurately segment multiple structures on panoramic radiographs. This method has the potential to be part of the process of automatic pathology diagnosis from dental panoramic radiographs.

Polarization-multiplexed metasurface enabled tri-functional imaging.

Diffraction-limited focusing imaging, edge-enhanced imaging, and long depth of focus imaging offer crucial technical capabilities for applications such as biological microscopy and surface topography detection. To conveniently and quickly realize the microscopy imaging of different functions, the multifunctional integrated system of microscopy imaging has become an increasingly important research direction. However, conventional microscopes necessitate bulky optical components to switch between these functionalities, suffering from the system's complexity and unstability. Hence, solving the problem of integrating multiple functions within an optical system is a pressing need. In this work, we present an approach using a polarization-multiplexed tri-functional metasurface, capable of realizing the aforementioned imaging functions simply by changing the polarization state of the input and output light, enhancing the system structure's compactness and flexibility. This work offers a new avenue for multifunctional imaging, with potential applications in biomedicine and microscopy imaging.

Optical edge-enhanced imaging based on dielectric metasurfaces

Optical edge-enhanced imaging is an effective method for boundary extraction, and it is also a cutting-edge technology for object detection in image processing. With the rapid development of metamaterial and metasurface fields, the miniaturization and lightweight of optical devices have reached a new scale. At present, the experimental research of optical edge-enhanced imaging using metasurface is insufficient and lags behind the theoretical research. In this paper, we propose optical edge-enhanced imaging based on a Huygens phase metasurface in the near-infrared (NIR). The entire optical edge-enhanced process is simulated, and the feasibility of the method is verified theoretically. Then, the edge-enhanced of numbers, letters, and biological tissues is carried out experimentally, and the results are consistent with the simulation results. This study proposes a near-infrared dielectric metasurface-based edge-enhanced imaging that can find important applications in compact optical platforms such as image processing.