Classical Image Research Articles

Deep Search for a Scattered Light Dust Halo Around Vega with the Hubble Space Telescope

We present a provisory scattered-light detection of the Vega debris disk using deep Hubble Space Telescope (HST) coronagraphy (PID 16666). At only 7.7 pc, Vega is immensely important in debris disk studies both for its prominence and also because it allows the highest physical resolution among all debris systems relative to temperature zones around the star. We employ the STIS coronagraph’s widest wedge position and classical reference differential imaging to achieve among the lowest surface-brightness sensitivities to date ( ∼4μJyarcsec−2 ) at wide separations using 32 orbits in Cycle 29. We detect a halo extending from the inner edge of our effective inner working angle at 10.″5 out to the photon noise floor at 30″ (80–230 au). The face-on orientation of the system and the lack of a perfectly color-matched point-spread function star have posed significant challenges to the reductions, particularly regarding artifacts from the imperfect color matching. However, we find that a halo of small dust grains provides the best explanation for the observed signal. Unlike Fomalhaut (a close twin to Vega in luminosity, distance, and age), there is no clear distinction in scattered light between the parent planetesimal belt observed with the Atacama Large Millimeter/submillimeter Array and the extended dust halo. These HST observations complement JWST GTO Cycle 1 observations of the system with NIRCam and MIRI.

Permutation-equivariant quantum convolutional neural networks

Abstract The Symmetric group $S_{n}$ manifests itself in large classes of quantum systems as the invariance of certain characteristics of a quantum state with respect to permuting the qubits. Subgroups of $S_{n}$ arise, among many other contexts, to describe label symmetry of classical images with respect to spatial transformations, such as reflection or rotation. Equipped with the formalism of geometric quantum machine learning, in this study we propose the architectures of equivariant quantum convolutional neural networks (EQCNNs) adherent to $S_{n}$ and its subgroups. We demonstrate that a careful choice of pixel-to-qubit embedding order can facilitate easy construction of EQCNNs for small subgroups of $S_{n}$. Our novel EQCNN architecture corresponding to the full permutation group $S_{n}$ is built by applying all possible QCNNs with equal probability, which can also be conceptualized as a dropout strategy in quantum neural networks. For subgroups of $S_{n}$, our numerical results using MNIST datasets show better classification accuracy than non-equivariant QCNNs. The $S_{n}$-equivariant QCNN architecture shows significantly improved training and test performance than non-equivariant QCNN for classification of connected and non-connected graphs. When trained with sufficiently large number of data, the $S_{n}$-equivariant QCNN shows better average performance compared to $S_{n}$-equivariant QNN . These results contribute towards building powerful quantum machine learning architectures in permutation-symmetric systems.

Euler Kernel Mapping for Hyperspectral Image Clustering via Self-Paced Learning

Clustering, as a classical unsupervised artificial intelligence technology, is commonly employed for hyperspectral image clustering tasks. However, most existing clustering methods designed for remote sensing tasks aim to solve a non-convex objective function, which can be optimized iteratively, beginning with random initializations. Consequently, during the learning phase of the clustering model, it may easily fall into bad local optimal solutions and finally hurt the clustering performance. Additionally, prevailing approaches often exhibit limitations in capturing the intricate structures inherent in hyperspectral images and are very sensitive to noise and outliers that widely exist in remote sensing data. To address these issues, we proposed a novel Euler kernel mapping for hyperspectral image clustering via self-paced learning (EKM-SPL). EKM-SPL first employs self-paced learning to learn the clustering model in a meaningful order by progressing samples from easy to complex, which can help to remove bad local optimal solutions. Secondly, a probabilistic soft weighting scheme is employed to measure complexity across the data sample, which makes the optimization process more reasonable. Thirdly, in order to more accurately characterize the intricate structure of hyperspectral images, Euler kernel mapping is used to convert the original data into a reproduced kernel Hilbert space, where the nonlinearly inseparable clusters may become linearly separable. Moreover, we innovatively integrate the coordinate descent technique into the optimization algorithm to circumvent the computational inefficiencies and information loss typically associated with conventional kernel methods. Extensive experiments conducted on classic benchmark hyperspectral image datasets illustrate the effectiveness and superiority of our proposed model.

Mitigation of DMM-induced stripe patterns in synchrotron X-ray radiography through dynamic tilting.

In synchrotron X-ray radiography, achieving high image resolution and an optimal signal-to-noise ratio (SNR) is crucial for the subsequent accurate image analysis. Traditional methods often struggle to balance these two parameters, especially in situ applications where rapid data acquisition is essential to capture specific dynamic processes. For quantitative image data analysis, using monochromatic X-rays is essential. A double multilayer monochromator (DMM) is successfully used for this aim at the BAMline, BESSY II (Helmholtz Zentrum Berlin, Germany). However, such DMMs are prone to producing an unstable horizontal stripe pattern. Such an unstable pattern renders proper signal normalization difficult and thereby causes a reduction of the SNR. We introduce a novel approach to enhance SNR while preserving resolution: dynamic tilting of the DMM. By adjusting the orientation of the DMM during the acquisition of radiographic projections, we optimize the X-ray imaging quality, thereby enhancing the SNR. The corresponding shift of the projection during this movement is corrected in post-processing. The latter correction allows a good resolution to be preserved. This dynamic tilting technique enables the homogenization of the beam profile and thereby effectively reduces noise while maintaining high resolution. We demonstrate that data captured using this proposed technique can be seamlessly integrated into the existing radiographic data workflow, as it does not need hardware modifications to classical X-ray imaging beamline setups. This facilitates further image analysis and processing using established methods.

Intelligent Manufacturing in Wine Barrel Production: Deep Learning-Based Wood Stave Classification

Innovative wood inspection technology is crucial in various industries, especially for determining wood quality by counting rings in each stave, a key factor in wine barrel production. (1) Background: Traditionally, human inspectors visually evaluate staves, compensating for natural variations and characteristics like dirt and saw-induced aberrations. These variations pose significant challenges for automatic inspection systems. Several techniques using classical image processing and deep learning have been developed to detect tree-ring boundaries, but they often struggle with woods exhibiting heterogeneity and texture irregularities. (2) Methods: This study proposes a hybrid approach combining classical computer vision techniques for preprocessing with deep learning algorithms for classification, designed for continuous automated processing. To enhance performance and accuracy, we employ a data augmentation strategy using cropping techniques to address intra-class variability in individual staves. (3) Results: Our approach significantly improves accuracy and reliability in classifying wood with irregular textures and heterogeneity. The use of explainable AI and model calibration offers a deeper understanding of the model’s decision-making process, ensuring robustness and transparency, and setting confidence thresholds for outputs. (4) Conclusions: The proposed system enhances the performance of automatic wood inspection technologies, providing a robust solution for industries requiring precise wood quality assessment, particularly in wine barrel production.

Research on Improved Image Segmentation Algorithm Based on GrabCut

The classic interactive image segmentation algorithm GrabCut achieves segmentation through iterative optimization. However, GrabCut requires multiple iterations, resulting in slower performance. Moreover, relying solely on a rectangular bounding box can sometimes lead to inaccuracies, especially when dealing with complex shapes or intricate object boundaries. To address these issues in GrabCut, an improvement is introduced by incorporating appearance overlap terms to optimize segmentation energy function, thereby achieving optimal segmentation results in a single iteration. This enhancement significantly reduces computational costs while improving the overall segmentation speed without compromising accuracy. Additionally, users can directly provide seed points on the image to more accurately indicate foreground and background regions, rather than relying solely on a bounding box. This interactive approach not only enhances the algorithm’s ability to accurately segment complex objects but also simplifies the user experience. We evaluate the experimental results through qualitative and quantitative analysis. In qualitative analysis, improvements in segmentation accuracy are visibly demonstrated through segmented images and residual segmentation results. In quantitative analysis, the improved algorithm outperforms GrabCut and min_cut algorithms in processing speed. When dealing with scenes where complex objects or foreground objects are very similar to the background, the improved algorithm will display more stable segmentation results.

Full-Waveform Inversion Imaging of Cortical Bone Using Phased Array Tomography.

Classic ultrasound bone imaging modalities usually demand either a prior knowledge or an advanced estimation on speed of sound (SoS), which not only renders to a burdensome imaging process but also supplies a limited resolution. To overcome these drawbacks, this article proposed a frequency-domain full-waveform inversion (FDFWI) modality using phased array tomography for high-accuracy cortical bone imaging. A transmission scenario of ultrasound wave in 2-D space was presented in the frequency domain to simulate the forward wavefield propagation. Iterations in the inversion process were performed by matching the simulation wavefield to the experimental one from low to high discrete frequency points. Moreover, the association between the maximum initial frequency and the initial SoS model was explored to prevent the occurrence of cycle-skipping phenomenon, which could lead to the outcomes being trapped in local minima. The feasibility and effectiveness of the proposed imaging scheme were testified by simulation, phantom, and ex-vivo studies, with mean relative errors of cortical part being 3.18%, 8.71%, and 9.36%, respectively. It is verified that the proposed FDFWI method is an effective way for parametric imaging of cortical bone without any prior knowledge of sound speed.

Deepest Limits on Scattered Light Emission from the Epsilon Eridani Inner Debris Disk with HST/STIS

Epsilon Eridani is one of the first debris disk systems detected by the Infrared Astronomical Satellite. However, the system has thus far eluded detection in scattered light with no components having been directly imaged. Its similarity to a relatively young solar system combined with its proximity makes it an excellent candidate to further our understanding of planetary system evolution. We present a set of coronagraphic images taken using the Space Telescope Imaging Spectrograph coronagraph on the Hubble Space Telescope at a small inner working angle to detect a predicted warm inner debris disk inside 1″. We used three different postprocessing approaches—nonnegative matrix factorization (NMF), Karhunen–Loève Image Processing (KLIP), and classical reference differential imaging, to best optimize reference star subtraction—and find that NMF performed the best overall while KLIP produced the absolute best contrast inside 1″. We present limits on scattered light from warm dust, with constraints on surface brightness at 6 mJy as−2 at our inner working angle of 0.″6. We also place a constraint of 0.5 mJy as−2 outside 1″, which gives us an upper limit on the brightness for outer disks and substellar companions. Finally, we calculated an upper limit on the dust albedo at ω < 0.487.

Contrast-preserving image smoothing via the truncated first-order rational function

The main task of image smoothing is to remove the insignificant details of the input image while preserving salient structural edges. In the fields of computer vision and graphics, image smoothing techniques are of great practical importance. In this paper, we investigate a new nonconvex variational optimization model for contrast-preserving image smoothing based on the truncated first-order rational (TFOR) penalty function. We employ an iterative numerical method that utilizes the half-quadratic minimization to effectively solve the proposed model. To validate the effectiveness of the proposed method, we compare it with some related state-of-the-art methods. Experimental results are given to show that the proposed method performs well in preserving the image contrast while maintaining the important edges and structures. We apply the proposed method on various classic image processing tasks such as clip-art compression artifact removal, detail enhancement, image denoising, image abstraction, flash and no-flash image restoration, and guided depth map upsampling.

Improving deep learning-based automatic cranial defect reconstruction by heavy data augmentation: From image registration to latent diffusion models

Modeling and manufacturing of personalized cranial implants are important research areas that may decrease the waiting time for patients suffering from cranial damage. The modeling of personalized implants may be partially automated by the use of deep learning-based methods. However, this task suffers from difficulties with generalizability into data from previously unseen distributions that make it difficult to use the research outcomes in real clinical settings. Due to difficulties with acquiring ground-truth annotations, different techniques to improve the heterogeneity of datasets used for training the deep networks have to be considered and introduced. In this work, we present a large-scale study of several augmentation techniques, varying from classical geometric transformations, image registration, variational autoencoders, and generative adversarial networks, to the most recent advances in latent diffusion models. We show that the use of heavy data augmentation significantly increases both the quantitative and qualitative outcomes, resulting in an average Dice Score above 0.94 for the SkullBreak and above 0.96 for the SkullFix datasets. The results show that latent diffusion models combined with vector quantized variational autoencoder outperform other generative augmentation strategies. Moreover, we show that the synthetically augmented network successfully reconstructs real clinical defects, without the need to acquire costly and time-consuming annotations. The findings of the work will lead to easier, faster, and less expensive modeling of personalized cranial implants. This is beneficial to numerous people suffering from cranial injuries. The work constitutes a considerable contribution to the field of artificial intelligence in the automatic modeling of personalized cranial implants.

Ultrasonic guided wave phased array based on reconstructed pulse-echo signals from phase information

Abstract Aiming at the inherent dispersion characteristics of ultrasonic guided waves and the problems of gratings and side lobes in classical total focusing method (TFM) imaging algorithms, this paper proposes a full focus imaging algorithm based on reconstruct signals from phase information. Utilizing the numerical solution of the dispersion curve in the wavenumber domain, the time domain signal of Lamb wave is propagated backward to revise the changes in waveform shape caused by dispersion, and the flight time of the pulse-echo is eliminated simultaneously. The phase of pulse-echo after dispersion compensation should be consistent with the excitation signal. And the pulse-echo should have a large waveform correlation coefficient with the excitation signal. For each imaging point, dispersion compensation for all collected signals is respectively implemented so that the sign coherence factor (SCF) and the waveform correlation factor (WCF) between pulse-echo and excitation waveform can be extracted to reconstruct scattering signals. The reconstructed signals only contain information about the pulse-echo from defect. TFM imaging of the reconstructed signal can effectively suppress the grating side lobes of classical TFM imaging and improve the signal-to-noise ratio (SNR).

An infrared-optical image registration method for industrial blower monitoring based on contour-shape descriptors

Industrial blowers are energy-efficient and widely used, but fault monitoring is challenging. Infrared and visible light sensors monitor their status, capturing temperature distribution and time evolution in images and videos. Due to the complexity of the industrial scene and the large difference between the characteristics of infrared and visible images, the existing registration methods are unable to accurately align the infrared and visible images. This paper proposes a Neural Network (NN) based registration method of infrared and visible images for industrial blowers using contours and Weight Global Shape Context Descriptor (W-GSCD), by considering the distance between different feature points as weights in our feature registration method which improves the accuracy of the registration of IR and visible images compared to classical image registration methods. Experiments on real data show that the proposed method effectively addresses registration challenges from complex shapes and heterogeneous backgrounds, achieving promising results on the blower dataset.

Deep learning-based instance segmentation for improved pepper phenotyping

Vegetable breeding companies invest a considerable amount of their resources in phenotyping. The advancement of computer vision technology has made it possible to digitalize these processes, leading to improved efficiency and quality. However, phenotyping activities often take place in outdoor fields or greenhouses, where the environmental/illumination conditions are constantly changing. This lack of standardization presents a problem for automatically isolating the relevant elements in the images, which is an important first step for phenotyping. Classical image analysis methods have shown not to be robust enough for that in these changing conditions. However, in the last years, deep learning models have demonstrated to be able to identify and learn meaningful features that are more robust and representative of the underlying patterns, enabling them to handle diverse and changeable conditions effectively.In this work, we propose a pepper instance segmentation solution based on deep learning after harvest under field conditions. We implement the method and validate it for three pepper varieties: Blocky Bell, Jalapeño and Lamuyo. We compare the performance of this new method for each variety with a previous solution based on classical image processing techniques, with the objective of measuring and demonstrating the superiority of deep learning-based instance segmentation over traditional methods as a first step for phenotyping.The instance segmentation deep learning based models outperform the results obtained by classical image processing algorithms for the three pepper varieties: in Blocky Bell mAP is increased from 0.63 to 0.97, in Jalapeño from 0.39 to 0.52 and in Lamuyo from 0.67 to 0.97.

Representation of classical painting images in an advertising product as a form of dialogue between elite and mass culture (using the example of Volkswagen AG’s 2008 advertising campaign)

Representation of classical painting images in an advertising product as a form of dialogue between elite and mass culture (using the example of Volkswagen AG’s 2008 advertising campaign)

Image restoration for digital line drawings using line masks

The restoration of digital images holds practical significance due to the fact that degradation of digital image data on the internet is common. State-of-the-art image restoration methods usually employ end-to-end trained networks. However, we argue that a network trained with diverse image pairs is not optimal for restoring line drawings which have extensive plain backgrounds. We propose a line-drawing restoration framework which takes a restoration neural network as backbone and processes an input degraded line drawing in two steps. First, a proposed mask-predicting network predicts a line mask which indicates the possible location of foreground and background in the potential original line drawing. Next, we feed the degraded input line drawing together with the predicted line mask into the backbone restoration network. The traditional L1 loss for the backbone restoration network is substituted with a masked Mean Square Error (MSE) loss. We test our framework on two classical image restoration tasks: JPEG restoration and super-resolution, and experiments demonstrate that our framework can achieve better quantitative and visual results in most cases.

An image information fusion based simple diffusion network leveraging the segment anything model for guided attention on thermal images producing colorized pedestrian masks

Because of the synergistic characteristics of the data they provide, thermal imagers are becoming more and more common as secondary data-collecting modules to complement classical optical imagers. Their application is especially beneficial when ambient elements like light, smoke, or other particulates need to be handled. As a result, fusion-based detection technology, like assistive driving systems, is trying to incorporate thermal imagers. In this work, we demonstrate that it is possible to create a mask for pedestrians in thermal imaging by building a basic Denoising Diffusion Probabilistic Model and using the results from the Segment Anything Model. Moreover, we show how the anticipated mask can fuse RGB and thermal pictures into a 3-channel output. Finally, we demonstrate how the loss function optimizes the end-to-end analysis in this field. Results from both the quantitative and qualitative analyses validate our methodology.

Curriculum learning for ab initio deep learned refractive optics

Deep optical optimization has recently emerged as a new paradigm for designing computational imaging systems using only the output image as the objective. However, it has been limited to either simple optical systems consisting of a single element such as a diffractive optical element or metalens, or the fine-tuning of compound lenses from good initial designs. Here we present a DeepLens design method based on curriculum learning, which is able to learn optical designs of compound lenses ab initio from randomly initialized surfaces without human intervention, therefore overcoming the need for a good initial design. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens in a cellphone-style form factor, with highly aspheric surfaces and a short back focal length.

Automated segmentation and deep learning classification of ductopenic parotid salivary glands in sialo cone-beam CT images.

This study addressed the challenge of detecting and classifying the severity of ductopenia in parotid glands, a structural abnormality characterized by a reduced number of salivary ducts, previously shown to be associated with salivary gland impairment. The aim of the study was to develop an automatic algorithm designed to improve diagnostic accuracy and efficiency in analyzing ductopenic parotid glands using sialo cone-beam CT (sialo-CBCT) images. We developed an end-to-end automatic pipeline consisting of three main steps: (1) region of interest (ROI) computation, (2) parotid gland segmentation using the Frangi filter, and (3) ductopenia case classification with a residual neural network (RNN) augmented by multidirectional maximum intensity projection (MIP) images. To explore the impact of the first two steps, the RNN was trained on three datasets: (1) original MIP images, (2) MIP images with predefined ROIs, and (3) MIP images after segmentation. Evaluation was conducted on 126 parotid sialo-CBCT scans of normal, moderate, and severe ductopenic cases, yielding a high performance of 100% for the ROI computation and 89% for the gland segmentation. Improvements in accuracy and F1 score were noted among the original MIP images (accuracy: 0.73, F1 score: 0.53), ROI-predefined images (accuracy: 0.78, F1 score: 0.56), and segmented images (accuracy: 0.95, F1 score: 0.90). Notably, ductopenic detection sensitivity was 0.99 in the segmented dataset, highlighting the capabilities of the algorithm in detecting ductopenic cases. Our method, which combines classical image processing and deep learning techniques, offers a promising solution for automatic detection of parotid glands ductopenia in sialo-CBCT scans. This may be used for further research aimed at understanding the role of presence and severity of ductopenia in salivary gland dysfunction.

Nonlinear sound-sheet microscopy: imaging opaque organs at the capillary and cellular scale.

Light-sheet fluorescence microscopy has revolutionized biology by visualizing dynamic cellular processes in three dimensions. However, light scattering in thick tissue and photobleaching of fluorescent reporters limit this method to studying thin or translucent specimens. Here we show that non-diffractive ultrasonic beams used in conjunction with a cross-amplitude modulation sequence and nonlinear acoustic reporters enable fast and volumetric imaging of targeted biological functions. We report volumetric imaging of tumor gene expression at the cm 3 scale using genetically encoded gas vesicles, and localization microscopy of currently uncharted cerebral capillary networks using intravascular microbubble contrast agents. Nonlinear sound-sheet microscopy provides a ∼64x acceleration in imaging speed, ∼35x increase in imaged volume and ∼4x increase in classical imaging resolution compared to the state-of-the-art in biomolecular ultrasound.

Qutrit representation of quantum images: new quantum ternary circuit design

Quantum computation is growing in significance and proving to be a powerful tool in meeting the high real-time computational demands of classical digital image processing. However, extensive research has been done on quantum image processing, mainly rooted in binary quantum systems. In this paper, we propose a new quantum ternary image circuit based on the analysis of the existing qutrit representation of quantum images. The proposed design utilizes ternary shift gates and ternary Muthukrishnan–Stroud gates, with the belief that this circuit can be used for ternary quantum image processing. This study makes a significant improvement compared to the existing counterpart in terms of quantum cost, the number of constant inputs, and garbage outputs, which are all essential parameters in quantum circuit design.