Computer Image Research Articles

2D surface optical reflectance for use in harsh reactive environments.

In recent years, studies of surfaces at more realistic conditions has advanced significantly, leading to an increased understanding of surface dynamics under reaction conditions. The development has mainly been due to the development of new experimental techniques or new experimental approaches. Techniques such as High Pressure Scanning Tunneling/Force Microscopy, Ambient Pressure x-ray Photo emission Spectroscopy, Surface x-ray Diffraction, Polarization-Modulation InfraRed Reflection Absorption Spectroscopy and Planar Laser Induced Fluorescence at semi-realistic conditions has been used to study planar model catalysts or industrial materials under operating conditions. 2D-Surface Optical Reflectance has recently received attention as a useful experimental tool used in gaseous and liquid harsh conditions by providing complementary experimental information on planar model samples as well as being a powerful experimental tool on its own. The simplicity of the approach and the cost of the equipment makes it an attractive alternative and useful tool for surface science studies under reaction conditions. In this topical review, we review some recent studies that have been promoted by the technical development in optical components, image acquisition and computational image analysis.

Deep plug-and-play HIO approach for phase retrieval

In the phase retrieval problem, the aim is the recovery of an unknown image from intensity-only measurements such as Fourier intensity. Although there are several solution approaches, solving this problem is challenging due to its nonlinear and ill-posed nature. Recently, learning-based approaches have emerged as powerful alternatives to the analytical methods for several inverse problems. In the context of phase retrieval, a novel plug-and-play approach, to our knowledge, that exploits learning-based prior and efficient update steps has been presented at the Computational Optical Sensing and Imaging topical meeting, with demonstrated state-of-the-art performance. The key idea was to incorporate learning-based prior to the Gerchberg-Saxton type algorithms through plug-and-play regularization. In this paper, we present the mathematical development of the method including the derivation of its analytical update steps based on half-quadratic splitting and comparatively evaluate its performance through extensive simulations on a large test dataset. The results show the effectiveness of the method in terms of image quality, computational efficiency, and robustness to initialization and noise.

Bidirectional‐Sensitive Dual‐Narrowband Self‐Powered Single Perovskite Photodetector for Fast Computational Imaging

AbstractNarrowband photodetectors (PDs) are extensively utilized in fields such as image sensing, machine vision, and optical communications. Traditional approaches to achieving wavelength‐selective light detection typically involve methods such as voltage‐modulated charge collection or injection the incorporation of optical filters, or the use of multi‐stacked active layer materials. Herein, an innovative dual‐mode perovskite PD characterized by unbalanced charge carrier transport (UCCT) properties are introduced. By altering the direction of light incidence, the PD exhibits two distinct narrowband spectral responses, thereby significantly simplifying conventional strategies that complicate PD fabrication. This novel approach also mitigates issues related to slow response times, limited band selectivity, and low external quantum efficiency (EQE). In the Red (R)‐Mode, a detection peak at 700 nm is identified, with a full‐width at half‐maximum (FWHM) of 30 nm, a specific detectivity (D*) of 2.6 × 1010 Jones, and a responsivity (R) of 179 mA W−1. Conversely, in Green (G‐Mode, a narrowband response is observed with a peak at 500 nm, a FWHM of 75 nm, a D* of 1.1 × 1010 Jones, and an R of 75 mA W−1. Furthermore, a dual‐mode narrowband single‐PD imaging prototype, integrated with an advanced Fourier imaging algorithm, has been developed, enabling the reconstruction of high‐quality images with 128 × 128 pixels within a time frame of 100 seconds. This research represents a significant advancement in the realm of filter‐free dual‐mode narrowband PDs and lays the groundwork for the evolution of next‐generation optoelectronic applications.

VR and computer vision based facade complexity analysis for building design

ABSTRACT Architectural practice is evolving through digital fabrication, enabling complex designs that challenge the uniformity of barren walls and fully glazed facades that often dominate contemporary streetscapes. This paper addresses the challenge of quantifying complexity in architectural facade design. It asks whether a Virtual Reality (VR) and Computer Vision (CV) approach can effectively measure facade complexity and align with user perceptions. The study employs the Computational Image Complexity Analysis (CICA) system, integrating VR and CV algorithms, to assess reactions to various facade complexities. Results reveal an average standard deviation of 9% between the system’s complexity measurements and participants’ perceptions, with a preference for moderate complexity. These findings highlight the importance of aligning architectural complexity with user preferences to enhance sustainability and satisfaction and the potential of this approach to quantify complexity and guide data-driven building design processes. Future research should explore the long-term impact of complex facades on user well-being and environmental sustainability.

High-fidelity lens-free on-chip microscopy via dual-channel noise separation

Lens-free on-chip microscopy (LFOCM) is a high-throughput computational imaging technique that enables high-resolution, label-free imaging without requiring complex optical systems. However, LFOCM encounters significant challenges in achieving high-resolution reconstructions due to noise accumulation. We propose a high-fidelity LFOCM method that integrates pixel super-resolution (PSR) with dynamic dual-channel noise separation (DCNS). This approach simultaneously separates the amplitude and phase noise during the reconstruction process, thereby improving noise robustness and enhancing the dynamic range of quantitative amplitude and phase imaging. Experimental validation across various sample types demonstrated the effectiveness of our method. DCNS achieves a resolution that exceeds 34.1% of the Nyquist–Shannon sampling limit, with a full field of view (FOV) of 28.6 mm2, improving the dynamic range of phase reconstruction and effectively suppressing artifacts that degrade the reconstruction quality, thus resolving the trade-off between noise reduction and resolution.

Does a Greater Hamstring Muscle Thickness Mean a Greater Aponeurosis Thickness?

The aim of the present study was to assess the association between muscle thickness and the superficial and deep aponeurosis thickness of the hamstrings. Ultrasound images were captured from the semitendinosus (ST) and biceps femoris long head (BFlh) of fifty young individuals (28 males and 22 females). Measurements were taken at six sites along the thigh in prone position. Superficial and deep aponeurosis and muscle thickness were evaluated across the entire length of the ST and BFlh using a computational image segmentation approach, which generated approximately ~360 data points per participant. There was significant interindividual variability in superficial (coefficient of variation, CV: 19.37–31.40%) and deep aponeurosis thickness (CV: 21.22–31.82%). Correlation analysis revealed a limited association between muscle and superficial aponeurosis thickness, with no significant correlations (r = −0.21 to 0.22, p > 0.05) observed across regions. However, moderate positive correlations were found for deep aponeurosis thickness in the proximal region of the BFlh (r = 0.43, p = 0.002) and in the middle (r = 0.34, p = 0.014) and distal–middle region of the ST (r = 0.35, p = 0.022). No consistent relation between muscle and aponeurosis thickness was observed, suggesting that aponeurosis morphology varies independently of muscle size. This variability may impact the mechanical strain distribution, emphasizing the need for individualized assessments in hamstring injury prediction.

Computational wide-field imaging of poplar embolism and wound-response with a deployable microscope.

Low-cost, minimally invasive microscopy for tracking cellular dynamics in living plants within their natural ecosystems is crucial for addressing fundamental questions in plant ecology and biology. However, existing solutions are constrained by coarse resolution, limited field-of-view (FoV), and poor deployability in natural settings. Here, we utilize a compact, portable microscope ("miniscope") for label-free (autofluorescence) imaging in living poplar wood. We systematically implement and evaluate multiple computational methods to enhance resolution and FoV. Our optimal computational pipeline, comprising maximal intensity projection, deconvolution, and flat-field correction, increases resolution by up to 39% on-axis and up to 49% at the field edges, resolving features of 2.87 μm, averaged over a FoV of ∼1mm (diameter), compared with a 4.34 μm baseline. We demonstrate microscopy within the tissue of a living poplar plant in our greenhouse, observing the embolism of vessel elements, wound response, and tissue deformation from moisture evaporation.

Lensless efficient snapshot hyperspectral imaging using dynamic phase modulation

Snapshot hyperspectral imaging based on a diffractive optical element (DOE) is increasingly featured in recent progress in deep optics. Despite remarkable advances in spatial and spectral resolutions, the limitations of current photolithography technology have prevented the fabricated DOE from being designed at ideal heights and with high diffraction efficiency, diminishing the effectiveness of coded imaging and reconstruction accuracy in some bands. Here, we propose, to our knowledge, a new lensless efficient snapshot hyperspectral imaging (LESHI) system that utilizes a liquid-crystal-on-silicon spatial light modulator (LCoS-SLM) to replace the traditionally fabricated DOE, resulting in high modulation levels and reconstruction accuracy. Beyond the single-lens imaging model, the system can leverage the switch ability of LCoS-SLM to implement distributed diffractive optics (DDO) imaging and enhance diffraction efficiency across the full visible spectrum. Using the proposed method, we develop a proof-of-concept prototype with an image resolution of 1920×1080 pixels, an effective spatial resolution of 41.74 μm, and a spectral resolution of 10 nm, while improving the average diffraction efficiency from 0.75 to 0.91 over the visible wavelength range (400–700 nm). Additionally, LESHI allows the focal length to be adjusted from 50 mm to 100 mm without the need for additional optical components, providing a cost-effective and time-saving solution for real-time on-site debugging. LESHI is the first imaging modality, to the best of our knowledge, to use dynamic diffractive optics and snapshot hyperspectral imaging, offering a completely new approach to computational spectral imaging and deep optics.

Phase contrast ghost imaging using pseudo-thermal light

Ghost imaging and ghost holography enable non-local imaging of amplitude and phase objects. Recent studies have indicated that dual-arm phase-contrast imaging can be achieved by extracting only the edges of the target, thereby obviating the need for a full image reconstruction. However, existing approaches still depend on either cameras or complex scanning procedures. In this paper, we combine spiral phase contrast imaging with ghost imaging and propose a computational dual-arm phase contrast imaging scheme based on Fourier basis modulation under speckle illumination, with single-point detection used in both arms. Experimental and simulation results validate the feasibility of the proposed scheme and demonstrate its effective implementation under compressed sampling. Furthermore, we also experimentally investigate the impact of the lateral coherence length of the speckle on edge extraction accuracy, as well as the effect of the number of speckles on the imaging signal-to-noise ratio.

Review of AI Image Enhancement Techniques for In-Vehicle Vision Systems Under Adverse Weather Conditions

Nowadays, rapid advancements in computer vision, image processing, and artificial intelligence (AI) have significantly benefited autonomous vehicles. Visual perception is crucial for enhancing the functionality and safety of self-driving technology. However, adverse weather and illumination conditions can impair visual capabilities, affecting environmental awareness, decision-making, and safe navigation. This work provides a comprehensive review of AI image enhancement methods and benchmark datasets, including deblurring, deraining, dehazing, and low-light enhancement, along with the integration of multiple image enhancement techniques in computer vision tasks. Specifically, this review focuses on advancements for real-world applications and summarizes performance metrics for real-time operation in automotive vision systems. Furthermore, the paper highlights efforts and challenges in real-world testing to ensure the effectiveness and reliability of these solutions in practical applications, which is essential for enabling autonomous vehicles to operate safely and efficiently under various challenging conditions, thereby contributing to the future of intelligent transportation systems.

In-situ quality monitoring during embedded bioprinting using integrated microscopy and classical computer vision

Despite significant advances in bioprinting technology, current hardware platforms lack the capability for process monitoring and quality control. This limitation hampers the translation of the technology into industrial GMP-compliant manufacturing settings. As a key step towards a solution, we developed a novel bioprinting platform integrating a high-resolution camera forin-situmonitoring of extrusion outcomes during embedded bioprinting. Leveraging classical computer vision and image analysis techniques, we then created a custom software module for assessing print quality. This module enables quantitative comparison of printer outputs to input points of the computer-aided design model's 2D projections, measuring area and positional accuracy. To showcase the platform's capabilities, we then investigated compatibility with various bioinks, dyes, and support bath materials for both 2D and 3D print path trajectories. In addition, we performed a detailed study on how the rheological properties of granular support hydrogels impact print quality during embedded bioprinting, illustrating a practical application of the platform. Our results demonstrated that lower viscosity, faster thixotropy recovery, and smaller particle sizes significantly enhance print fidelity. This novel bioprinting platform, equipped with integrated process monitoring, holds great potential for establishing auditable and more reproducible biofabrication processes for industrial applications.

Point spread function modeling and engineering in black-box lens systems

Point spread function (PSF) engineering in an imaging system involves the introduction of additional optical components to efficiently encode object information. It typically relies on a well-established mapping relation between object space, PSF engineering plane, and image plane. However, this reliance can limit its application in imaging systems with unknown (“black box”) attributes, e. g. commercial photographic lenses, where a theoretical imaging model is not readily available. To address this limitation, we conduct wave-optics analysis of general lens systems, explicitly derive a general imaging model based on scaled Fourier transform, and develop an overall pipeline for automatic parameter determination of the model. When applied to a photographic lens, this pipeline demonstrates high PSF modeling accuracy, showcasing the capability of “seeing what you design”. We then implemented PSF engineering in this lens to find its optimal PSF for 3D localization. This work improves computational imaging by enabling accurate wavefront shaping design in arbitrary imaging systems.

Identification of Marine Oil Spills Using Neural Network Technologies

Introduction. Detecting oil spills is a critical task in monitoring the marine ecosystem, protecting it, and minimizing the consequences of emergency situations. The development of fast and accurate methods for detecting and mapping oil spills at sea is essential for prompt assessment and response to emergencies. High-resolution aerial photography provides researchers with a tool for remote monitoring of water discoloration. Artificial intelligence technologies contribute to improving and automating the interpretation and analysis of such images. This study aims to develop approaches for identifying oil spilled on water surfaces using neural networks and machine learning techniques.Materials and Methods. Algorithms capable of automatically identifying marine oil spills were developed using computer image analysis and machine learning methods. The U-Net convolutional neural network was employed for image segmentation tasks. The neural network architecture was designed using the PyTorch library implemented in Python. The AdamW optimizer was chosen for training the network. The neural network was trained on a dataset comprising 8,700 images.Results. The performance of oil spill detection on water surfaces was evaluated using metrics such as IoU, Precision, Recall, Accuracy, and F1 score. Calculations based on these metrics demonstrated identification accuracy of approximately 83–88%, confirming the efficiency of the algorithms used.Discussion and Conclusion. The U-Net convolutional network was successfully trained and demonstrated high accuracy in detecting marine oil spills on the given dataset. Future work will focus on developing algorithms using more advanced neural network models and image augmentation methods.

Computational spectral imaging reconstruction via a spatial–spectral cross-attention-driven network

Compared with traditional hyperspectral imaging, computational spectral imaging (CSI) has the advantage of snapshot imaging with high spatial and temporal resolution, which has attracted considerable attention. The core challenge of CSI is to achieve computational imaging reconstruction from a single 2D measurement image to the corresponding 3D spatial–hyperspectral image (HSI). Existing reconstruction methods still face problems in exploring spatial–spectral cross correlation, leading to significant spatial–spectral distortion. Furthermore, due to neglect of multi-scale feature reconstruction, their reconstruction quality still needs to be improved. In this paper, to solve the above problems, we propose a spatial–spectral cross-attention-driven network (SSCA-DN). In SSCA, a proposed multi-scale feature aggregation (MFA) module and a spectral-wise transformer (SpeT) are used for multi-scale spatial feature reconstruction and long-range spectral feature reconstruction, respectively. Using spatial attention and spectral attention to interactively guide the reconstruction of the target HSI in spectral and spatial dimensions, the proposed SSCA models spatial–spectral cross correlation with considering multi-scale features. Using the SSCA as a basic module, a novel SSCA-DN network is constructed, in which a proposed supervised preliminary reconstruction subnetwork (SPRNet) learns the generalized prior, and a proposed unsupervised multi-scale feature fusion and refinement subnetwork (UMFFRNet) learns the specific prior. The SSCA module ensures that the learned generalized and specific priors can capture the spatial–spectral cross correlation while considering multi-scale features. In addition, in UMFFRNet, driven by MFA and SSCA, a novel multi-scale fusion and refinement mechanism for multi-level adjacent features is proposed to effectively model the correlation between adjacent level features and the multi-scale spatial–spectral cross correlation, which further improves the reconstruction accuracy. Extensive experiments show that our method achieves state-of-the-art performance on both simulated and real datasets.

Programmable beam steering and quasi-continuous phase shift on a 2-bit terahertz coding metasurface with HEMT-switched multimodal modulation.

Terahertz reconfigurable intelligent surfaces (RIS) stand out from conventional phased arrays thanks to their unique electromagnetic properties and intelligent interconnect paradigms. They are a vital technology for terahertz wireless communication and radar detection systems. Compared with 1-bit coding metasurfaces, 2-bit coding metasurfaces offer significant advantages such as single beam steering and reduced quantization errors. Electrically controlled switchable coding metasurfaces have been restricted by the switch performance and integration technology, limiting reported devices to 1-bit coding. Additionally, metasurfaces that offer quasi-continuous phase modulation across a 2π range have garnered extensive attention for their higher phase shift precision and regulation freedom. This paper proposes a 2-bit multimodal THz quasi-continuous phase shift coding metasurface based on asymmetrically configured high electron mobility transistor (HEMT)-dipoles. The meta-device consists of two asymmetrically combined HEMT-dipole resonant structures, wherein the versatile density variations of two-dimensional electron gas (2DEG) introduce subtly local-field modulation under different external voltage combinations, rendering the quasi-continuous phase shift of 0° to 350° at 0.376 THz with a minimum incremental step of 6°. Moreover, selecting optimal 2-bit coding states within the quasi-continuous phase shift loop constructed different array phase gradients based on Snell's law, enabling real-time beam steering from 21° to 48° within the 0.365-0.385 THz frequency range. The simulation results align closely with the experimental findings, marking the first achievement of real-time programmable phase shift control and 2-bit beam steering using electrically controlled switches in the terahertz domain. This research provides a practical approach for real-time quasi-optical continuous phase modulation control and programmable single-beam electronic steering, with promising applications in terahertz wireless communication, radar detecting, and computational imaging, among other fields.

Real-time high-quality single-lens computational imaging via enhancing lens modulation transfer function consistency

Real-time high-quality single-lens computational imaging via enhancing lens modulation transfer function consistency

Long-term tracking of neural and oligodendroglial development in large-scale human cerebral organoids by noninvasive volumetric imaging

Human cerebral organoids serve as a quintessential model for deciphering the complexities of brain development in a three-dimensional milieu. However, imaging these organoids, particularly when they exceed several millimeters in size, has been curtailed by the technical impediments such as phototoxicity, slow imaging speeds, and inadequate resolution and imaging depth. Addressing these pivotal challenges, our study has pioneered a high-speed scanning microscope, synergistically coupled with advanced computational image processing. This ensemble has empowered us to monitor the intricate dynamics of neuron and oligodendrocyte development within cerebral organoids across a trajectory of approximately two months. Line-shaped illumination mitigates photodamage and, alongside refined spatial gating, maximizes signal collection through integrating with computational processing. The integration of deconvolution and compressive sensing has improved image contrast by 6-fold, elucidating fine features of the neurites. Thus, noninvasive imaging enabled us to perform long-term tracking of neural and oligodendroglial development in the large-scale human cerebral organoid. Furthermore, our sophisticated volumetric segmentation algorithm has yielded a robust four-dimensional quantitative analysis, encapsulating both neuronal and oligodendroglial maturation. Collectively, these advances mark a significant advancement in the field of neurodevelopment, providing a powerful tool for in-depth study of complex brain organoid systems.

An Encoded Photodetector Array for High-Frame-Rate Compressed Sensing and Computational Imaging

An Encoded Photodetector Array for High-Frame-Rate Compressed Sensing and Computational Imaging

Research on Innovative Apple Grading Technology Driven by Intelligent Vision and Machine Learning.

In the domain of food science, apple grading holds significant research value and application potential. Currently, apple grading predominantly relies on manual methods, which present challenges such as low production efficiency and high subjectivity. This study marks the first integration of advanced computer vision, image processing, and machine learning technologies to design an innovative automated apple grading system. The system aims to reduce human interference and enhance grading efficiency and accuracy. A lightweight detection algorithm, FDNet-p, was developed to capture stem features, and a strategy for auxiliary positioning was designed for image acquisition. An improved DPC-AWKNN segmentation algorithm is proposed for segmenting the apple body. Image processing techniques are employed to extract apple features, such as color, shape, and diameter, culminating in the development of an intelligent apple grading model using the GBDT algorithm. Experimental results demonstrate that, in stem detection tasks, the lightweight FDNet-p model exhibits superior performance compared to various detection models, achieving an mAP@0.5 of 96.6%, with a GFLOPs of 3.4 and a model size of just 2.5 MB. In apple grading experiments, the GBDT grading model achieved the best comprehensive performance among classification models, with weighted Jacard Score, Precision, Recall, and F1 Score values of 0.9506, 0.9196, 0.9683, and 0.9513, respectively. The proposed stem detection and apple body classification models provide innovative solutions for detection and classification tasks in automated fruit grading, offering a comprehensive and replicable research framework for standardizing image processing and feature extraction for apples and similar spherical fruit bodies.

An Immunocytochemistry Method to Investigate the Translationally Active HIV Reservoir.

Despite the success of combination antiretroviral therapy (cART) to suppress HIV replication, HIV persists in a long-lived reservoir that can give rise to rebounding viremia upon cART cessation. The translationally active reservoir consists of HIV-infected cells that continue to produce viral proteins even in the presence of cART. These active reservoir cells are implicated in the resultant viremia upon cART cessation and likely contribute to chronic immune activation in people living with HIV (PLWH) on cART. Methodologies to quantify the active reservoir are needed. Here, an automated immunocytochemistry (ICC) assay coupled with computational image analysis to detect and quantify intracellular Gag capsid protein (CA) is described (CA-ICC). For this purpose, fixed cells were deposited on microscopy slides by the cytospin technique and stained with antibodies against CA by an automated stainer, followed by slide digitization. Nuclear staining was used to count the number of cells in the specimen, and the chromogenic signal was quantified to determine the percentage of CA-positive cells. In comparative analyses, digital ELISA, qPCR, and flow cytometry were used to validate CA-ICC. The specificity and sensitivity of CA-ICC were assessed by staining a cell line that expresses CA (MOLT IIIB) alongside a control cell line (Jurkat) devoid of this marker, as well as peripheral blood mononuclear cells (PBMCs) from HIV seronegative donors before or after ex vivo infection with an HIV laboratory strain. The sensitivity of CA-ICC was further assayed by spiking MOLT IIIB cells into uninfected Jurkat cells in limiting dilutions. In those analyses, CA-ICC could detect down to 10 CA-positive cells per million with a sensitivity superior to flow cytometry. To demonstrate the application of CA-ICC in pre-clinical research, bulk PBMCs obtained from mouse and non-human primate animal models were stained to detect HIV CA and SIV p27, respectively. The level of intracellular CA quantified by CA-ICC in PBMCs obtained from animal models was associated with plasma viral loads and cell-associated CA measured by qPCR and ELISA, respectively. The application of CA-ICC to evaluate the activity of small-molecule targeted activator of cell-kill (TACK) in clinical specimens is presented. Overall, CA-ICC offers a simple imaging method for specific and sensitive detection of CA-positive cells in bulk cell preparations.