Image Processing Tasks Research Articles

How can geostatistics help us understand deep learning? An exploratory study in SAR-based aircraft detection

Deep Neural Networks (DNNs) have garnered significant attention across various research domains due to their impressive performance, particularly Convolutional Neural Networks (CNNs), known for their exceptional accuracy in image processing tasks. However, the opaque nature of DNNs has raised concerns about their trustworthiness, as users often cannot understand how the model arrives at its predictions or decisions. This lack of transparency is particularly problematic in critical fields such as healthcare, finance, and law, where the stakes are high. Consequently, there has been a surge in the development of explanation methods for DNNs. Typically, the effectiveness of these methods is assessed subjectively via human observation on the heatmaps or attribution maps generated by eXplanation AI (XAI) methods. In this paper, a novel GeoStatistics Explainable Artificial Intelligence (GSEAI) framework is proposed, which integrates spatial pattern analysis from Geostatistics with XAI algorithms to assess and compare XAI understandability. Global and local Moran’s I indices, commonly used to assess the spatial autocorrelation of geographic data, assist in comprehending the spatial distribution patterns of attribution maps produced by the XAI method, through measuring the levels of aggregation or dispersion. Interpreting and analyzing attribution maps by Moran’s I scattergram and LISA clustering maps provide an accurate global objective quantitative assessment of the spatial distribution of feature attribution and achieves a more understandable local interpretation. In this paper, we conduct experiments on aircraft detection in SAR images based on the widely used YOLOv5 network, and evaluate four mainstream XAI methods quantitatively and qualitatively. By using GSEAI to perform explanation analysis of the given DNN, we could gain more insights about the behavior of the network, to enhance the trustworthiness of DNN applications. To the best of our knowledge, this is the first time XAI has been integrated with geostatistical algorithms in SAR domain knowledge, which expands the analytical approaches of XAI and also promotes the development of XAI within SAR image analytics.

Efficient urinary stone type prediction: a novel approach based on self-distillation

Urolithiasis is a leading urological disorder where accurate preoperative identification of stone types is critical for effective treatment. Deep learning has shown promise in classifying urolithiasis from CT images, yet faces challenges with model size and computational efficiency in real clinical settings. To address these challenges, we developed a non-invasive prediction approach for determining urinary stone types based on CT images. Through the refinement and improvement of the self-distillation architecture, coupled with the incorporation of feature fusion and the Coordinate Attention Module (CAM), we facilitated a more effective and thorough knowledge transfer. This method circumvents the extra computational expenses and performance reduction linked with model compression and removes the reliance on external teacher models, markedly enhancing the efficacy of lightweight models. achieved a classification accuracy of 74.96% on a proprietary dataset, outperforming current techniques. Furthermore, our method demonstrated superior performance and generalizability on two public datasets. This not only validates the effectiveness of our approach in classifying urinary stones but also showcases its potential in other medical image processing tasks. These results further reinforce the feasibility of our model for actual clinical deployment, potentially assisting healthcare professionals in devising more precise treatment plans and reducing patient discomfort.

Design and Fabrication of SCARA for Image Processing in Industry

The scope of our project is the design and fabrication of SCARA (Selective Compliance Articulated Robot Arm) robots optimized for image processing tasks within industrial environments. This report presents a comprehensive review of the design, control, and applications of SCARA robots, focusing on their integration with image processing technologies to enhance industrial automation. We delve into the underlying principles governing the kinematics and dynamics of SCARA robots, elucidating their operational mechanisms and capabilities. Advancementsin end-effector design and sensor integration have enhanced the adaptability of SCARA robots, enabling them to tackle complex tasks in unstructured environments.

An improved algorithm for salient object detection of microscope based on U2-Net.

With the rapid advancement of modern medical technology, microscopy imaging systems have become one of the key technologies in medical image analysis. However, manual use of microscopes presents issues such as operator dependency, inefficiency, and time consumption. To enhance the efficiency and accuracy of medical image capture and reduce the burden of subsequent quantitative analysis, this paper proposes an improved microscope salient object detection algorithm based on U2-Net, incorporating deep learning technology. The improved algorithm first enhances the network's key information extraction capability by incorporating the Convolutional Block Attention Module (CBAM) into U2-Net. It then optimizes network complexity by constructing a Simple Pyramid Pooling Module (SPPM) and uses Ghost convolution to achieve model lightweighting. Additionally, data augmentation is applied to the slides to improve the algorithm's robustness and generalization. The experimental results show that the size of the improved algorithm model is 72.5MB, which represents a 56.85% reduction compared to the original U2-Net model size of 168.0MB. Additionally, the model's prediction accuracy has increased from 92.24 to 97.13%, providing an efficient means for subsequent image processing and analysis tasks in microscopy imaging systems.

Secret image restoration with interpolation and social network search

Image restoration is an essential task in image processing. Due to the secret image extracted from the corrupted stego-image and its pixels may suffer different levels of damage, the type of noise is particular. Most of the existing methods have a challenge in directly removing this type of noise. Hence, we propose a corrupted secret image restoration algorithm with interpolation and social network search. Firstly, a corrupted secret image is divided into several blocks, and the convex hull of each block is calculated. For the corrupted pixels in the convex hull, the natural neighbor interpolation and bi-harmonic spline interpolation are utilized to generate the estimation values of corrupted pixels according to the density of the trusted pixels. For the corrupted pixels outside the convex hull, the social network search algorithm is applied to generate the estimated values of corrupted pixels. Finally, all corrupted pixels are recovered by the ones of their candidate values closest to their estimated values. The performance of the various methodologies was evaluated using the USCSIPI database, employing peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indices, which demonstrated an enhancement of more than 0.78 dB and 0.0431, respectively. Experiments show that our algorithm outperforms other state-of-the-art (SOTA) algorithms.

Specular highlight removal using Quaternion transformer

Specular highlight removal is a very important issue, because specular highlight reflections in images with illumination changes can give very negative effects on various computer vision and image processing tasks. Numerous state-of-the-art networks for the specular removal use convolutional neural networks (CNN), which cannot learn global context effectively. They capture spatial information while overlooking 3D structural correlation information of an RGB image. To address this problem, we introduce a specular highlight removal network based on Quaternion transformer (QformerSHR), which employs a transformer architecture based on Quaternion representation. In particular, a depth-wise separable Quaternion convolutional layer (DSQConv) is proposed to enhance computational performance of QformerSHR, while efficiently preserving the structural correlation of an RGB image by utilizing the Quaternion representation. In addition, a Quaternion transformer block (QTB) based on DSQConv learns global context. As a result, QformerSHR consisting of DSQConv and QTB performs the specular removal from natural and text image datasets effectively. Experimental results demonstrate that it is significantly more effective than state-of-the-art networks for the specular removal, in terms of both quantitative performance and subjective quality.

RBP-DIP: Residual back projection with deep image prior for ill-posed CT reconstruction

The success of deep image prior (DIP) in a number of image processing tasks has motivated their application in image reconstruction problems in computed tomography (CT). In this paper, we introduce a residual back projection technique (RBP) that improves the performance of deep image prior framework in iterative CT reconstruction, especially when the reconstruction problem is highly ill-posed. The RBP-DIP framework uses an untrained U-net in conjunction with a novel residual back projection connection to minimize the objective function while improving reconstruction accuracy. In each iteration, the weights of the untrained U-net are optimized, and the output of the U-net in the current iteration is used to update the input of the U-net in the next iteration through the proposed RBP connection. The introduction of the RBP connection strengthens the regularization effects of the DIP framework in the context of iterative CT reconstruction leading to improvements in accuracy. Our experiments demonstrate that the RBP-DIP framework offers improvements over other state-of-the-art conventional IR methods, as well as pre-trained and untrained models with similar network structures under multiple conditions. These improvements are particularly significant in the few-view and limited-angle CT reconstructions, where the corresponding inverse problems are highly ill-posed and the training data is limited. Furthermore, RBP-DIP has the potential for further improvement. Most existing IR algorithms, pre-trained models, and enhancements applicable to the original DIP algorithm can also be integrated into the RBP-DIP framework.

Stability and Preservation of Fuzzy Membership Functions in Adaptive Gaussian Derivative Filters

This paper examines the stability and preservation of fuzzy membership functions in Gaussian filters, particularly focusing on the Adaptive Gaussian Derivative (AGD) filter. Gaussian filters are essential for smoothing and noise reduction in image processing. The AGD filter, which adapts based on local image statistics, outperforms traditional methods. Key concepts like fuzzy sets, Boundary Input Boundary Output (BIBO) stability, and convolution are defined, and the mathematical formulation of the Gaussian and its derivative is presented. The AGD filter's BIBO stability ensures bounded outputs for bounded inputs, guaranteeing consistent behavior. It also preserves fuzzy membership function properties, maintaining convexity and boundedness through linearity and continuity. Frequency response analysis using Fourier Transform confirms the AGD filter retains the Gaussian shape in the frequency domain, preserving image smoothness. Theorems and proofs validate the AGD filter's stability and its capability to preserve fuzzy membership functions, ensuring reliable processing in applications such as medical image analysis. These properties make the AGD filter a robust tool for advanced image processing tasks.

Contrast-preserving image smoothing via the truncated first-order rational function

The main task of image smoothing is to remove the insignificant details of the input image while preserving salient structural edges. In the fields of computer vision and graphics, image smoothing techniques are of great practical importance. In this paper, we investigate a new nonconvex variational optimization model for contrast-preserving image smoothing based on the truncated first-order rational (TFOR) penalty function. We employ an iterative numerical method that utilizes the half-quadratic minimization to effectively solve the proposed model. To validate the effectiveness of the proposed method, we compare it with some related state-of-the-art methods. Experimental results are given to show that the proposed method performs well in preserving the image contrast while maintaining the important edges and structures. We apply the proposed method on various classic image processing tasks such as clip-art compression artifact removal, detail enhancement, image denoising, image abstraction, flash and no-flash image restoration, and guided depth map upsampling.

Brainchop: Providing an Edge Ecosystem for Deployment of Neuroimaging Artificial Intelligence Models.

Deep learning has proven highly effective in various medical imaging scenarios, yet the lack of an efficient distribution platform hinders developers from sharing models with end-users. Here, we describe brainchop, a fully functional web application that allows users to apply deep learning models developed with Python to local neuroimaging data from within their browser. While training artificial intelligence models is computationally expensive, applying existing models to neuroimaging data can be very fast; brainchop harnesses the end user's graphics card such that brain extraction, tissue segmentation, and regional parcellation require only seconds and avoids privacy issues that impact cloud-based solutions. The integrated visualization allows users to validate the inferences, and includes tools to annotate and edit the resulting segmentations. Our pure JavaScript implementation includes optimized helper functions for conforming volumes and filtering connected components with minimal dependencies. Brainchop provides a simple mechanism for distributing models for additional image processing tasks, including registration and identification of abnormal tissue, including tumors, lesions and hyperintensities. We discuss considerations for other AI model developers to leverage this open-source resource.

SFP: Similarity-based filter pruning for deep neural networks

Convolutional neural networks have exhibited exceptional performance in various artificial intelligence domains, particularly in large-scale image processing tasks. However, the proliferation of network parameters and computational requirements has emerged as a significant bottleneck for the practical deployment of CNNs. In this paper, we propose a novel similarity-based filter pruning (SFP) approach for compressing convolutional neural networks, which is different from the traditional pruning method. The existing pruning methods eliminate the unimportant parameters but ignore the duplication of the reserved convolutional kernels. In the proposed SFP, kernels are clustered first according to their similarity, then the unimportant and redundant kernels are pruned in each class, which is more efficient than traditional pruning methods only based on the importance criterion. Furthermore, this paper introduces the concept of Kernel Dispersion to evaluate sparsity across distinct network layers, and proposes Distillation Fine-Tuning with Variable Temperature Coefficient to expedite convergence and enhance accuracy. The performance of the proposed similarity-based filter pruning approach is evaluated on different datasets, including CIFAR10, CIFAR100, ImageNet, and VOC. The experimental results indicate that the proposed SFP achieves approximately 1% higher accuracy at a comparable pruning rate compared to traditional state-of-the-art pruning methods.

Image Contrast Enhancement Using Artificial Hummingbird Algorithm

Abstract: Image enhancement plays an important role in medical imaging, satellite image processing, and computer vision. These fields require high-quality images to improve the visibility of details, which is essential for accurate analysis and interpretation. Contrast adjustment, histogram equalization, and gamma correction are the traditional image enhancement techniques that are widely used. However, these methods often fall short, particularly when applied uniformly across diverse images, as they may not always yield the best results. We provide a unique picture enhancing method based on the Artificial Hummingbird Algorithm (AHA) to overcome these drawbacks. The AHA is inspired by the foraging behaviour of hummingbirds, which are known for their efficient and adaptive search strategies. This bio-inspired algorithm leverages strategies to optimize the gamma correction parameter, aiming to enhance image contrast and overall visual quality. The aim is to optimize the gamma correction parameter to maximize fitness values derived from several image quality metrics, including Structural Similarity Index Measure (SSIM), Feature Similarity Index Measure (FSIM), and Peak Signal-to-Noise Ratio (PSNR). The AHA operates through a population-based approach, where each individual in the population represents a potential solution with a specific gamma value. Initially, a population of random gamma values is generated, and each value is used to enhance the image, followed by evaluating its quality using the combined fitness function. The algorithm uses three distinct foraging strategies to efficiently explore the solution space: guided, territorial and migration foraging. Guided foraging helps steer the search towards promising areas, while territorial foraging refines the local search. Migration foraging periodically resets some individuals to avoid premature convergence and ensure a comprehensive exploration of the solution space. Through iterative adaptation and evolution, the AHA continuously updates and refines the best gamma value found, leading to significantly enhanced images. Experimental results demonstrate that the AHA-based technique offers substantial improvements in image quality, with better contrast, sharper details, and higher scores in the quality metrics compared to traditional methods, making it a robust and efficient solution for high-quality image processing tasks.

Optimal quantum circuit generation for pixel segmentation in multiband images

A novel approach is proposed for multiband image processing via quantum models in real situations. Quantum circuits are automatically generated ad-hoc for each use case via multiobjective genetic algorithms. Using this universal method, image processing tasks such as segmentation can be carried out by considering the properties that constitute each pixel. The generated circuits present a low level of correlation between qubits, and thus can be considered quantum-inspired machine learning models. The effectiveness of this methodology has been validated by applying it to different segmentation use cases. Comparisons are made between optimized classical kernel methods and the generated quantum-inspired models to understand their behaviors. The results show that quantum models for multiband image processing achieve accuracies similar to those of classical methods.

Collaborative masking based speckle disentanglement for self-supervised optical coherence tomography image despeckling

Optical coherence tomography (OCT) has proven to be an effective and safe diagnostic tool in clinical settings due to its unique advantages. Nonetheless, the OCT images are unavoidably corrupted with speckle noise. Despeckling has become a vital preliminary step in OCT based image processing and analysis tasks. The supervised deep learning (DL) based denoising algorithms have recently attracted much attention due to their promising performance, but the lack of the noise-free OCT images renders it difficult to provide the effective supervision for DL model training. This paper has presented a self-supervised OCT image despeckling approach which does not depend on the reference clean image and combines the disentangled representation strategy with speckle noise estimation for the effective denoising. The proposed algorithm firstly disentangles the corrupted image into the speckle noise and clean image. Then, the multiple generated clean images are multiplied by the disentangled speckle noise to synthesize the speckle-corrupted image for self-supervised learning. To make the generated clean image and extracted speckle noise more statistically realistic, a collaborative masking strategy and a noise calibration module are applied to explore the spatial correlation of the clean image and refine speckle noise. The proposed method has been validated on two public datasets. Experimental results demonstrate that our method has superior speckle reduction performance to some traditional and DL-based despeckling approaches in terms of visual evaluation and quantitative metrics. Particularly, our method provides contrast-to-noise ratio (CNR) and signal-to-noise ratio (SNR) improvements of at least 0.35 dB and 14.37 dB over the compared methods, respectively. Additionally, the retinal image classification results provide further evidence for the exceptional despeckling performance of our approach against other compared methods. It is obvious that the developed method has significant advantages in keeping the fidelity of retinal structure and enhancing the speckle suppression level. Meanwhile, the presented method holds certain reference value for improving diagnostic efficiency in clinical.

Domain adaptive noise reduction with iterative knowledge transfer and style generalization learning

Low-dose computed tomography (LDCT) denoising tasks face significant challenges in practical imaging scenarios. Supervised methods encounter difficulties in real-world scenarios as there are no paired data for training. Moreover, when applied to datasets with varying noise patterns, these methods may experience decreased performance owing to the domain gap. Conversely, unsupervised methods do not require paired data and can be directly trained on real-world data. However, they often exhibit inferior performance compared to supervised methods. To address this issue, it is necessary to leverage the strengths of these supervised and unsupervised methods. In this paper, we propose a novel domain adaptive noise reduction framework (DANRF), which integrates both knowledge transfer and style generalization learning to effectively tackle the domain gap problem. Specifically, an iterative knowledge transfer method with knowledge distillation is selected to train the target model using unlabeled target data and a pre-trained source model trained with paired simulation data. Meanwhile, we introduce the mean teacher mechanism to update the source model, enabling it to adapt to the target domain. Furthermore, an iterative style generalization learning process is also designed to enrich the style diversity of the training dataset. We evaluate the performance of our approach through experiments conducted on multi-source datasets. The results demonstrate the feasibility and effectiveness of our proposed DANRF model in multi-source LDCT image processing tasks. Given its hybrid nature, which combines the advantages of supervised and unsupervised learning, and its ability to bridge domain gaps, our approach is well-suited for improving practical low-dose CT imaging in clinical settings. Code for our proposed approach is publicly available at https://github.com/tyfeiii/DANRF.

A high-quality self-supervised image denoising method based on SDDW-GAN and CHRNet

Image denoising remains a classic and crucial issue in the field of image processing, significantly impacting the outcomes of subsequent image processing tasks. For instance, the denoising network depends on “noise-clean” image pairs to train network effectively. However, it is often hampered by issues such as useful information loss, low training efficiency, and poor blind denoising. To address these challenges, this study proposes a novel image denoising network that integrates the complementary strengths of model-based and learning-based approaches, specifically leveraging the capabilities of both SDDW-GAN and CHRNet. Firstly, SDDW-GAN is designed to estimate the noise distribution on the input noisy images, and a fast-smoothing noisy block sampling algorithm is proposed to extract the noise blocks in noisy images in SDDW-GAN. Secondly, a network with dual generators and dual discriminators based on W-GAN is designed to estimate the noise distribution on the input noisy images and generate noise sample pairs with the same noise distribution, which solves the problem of relying on “noise-clean” image pairs. Thirdly, CHRNet is designed to compute the mapping relationship between the double-noise samples and the single-noise samples. In order to further improve the denoising effect, the dual-channel residual attention module is proposed for fusion learning of global and local features. Experimental results show that the proposed method has a better denoising effect in complex environments and outperforms existing denoising methods. Specifically, in comparison with the stand-alone denoising methods BM3D, DnCNN, Noise2Noise, and Blind2Unblind, the proposed method improves the average peak signal-to-noise ratio (PSNR) by 0.23 dB to 0.78 dB on two benchmarking datasets crossing different noise levels. Its denoising effect is also greater than other competitive stand-alone and combination methods. The proposed method can also extend to low-light image enhancement, deblurring, and super-resolution.

Topological Analysis of Image Reconstruction Based on Polar Complex Exponential Transform

Image moments are an important tool used for image reconstruction. An image can be represented in terms of image moments, which is known as image reconstruction from its moments. To construct an image from the moments, a question often arises that how many moments are required for the reconstruction of image. Theoretically, many image moments may be required for accurately reconstructing an image. However, since image moment construction can be computationally challenging, often in practice only a finite number of moments are used for the image reconstruction. The difference in reconstructed and original image for many numbers of images can lead to a satisfying answer. However, accurate reconstruction may not often be possible, and we are often relying on other approaches to find similarity between the original image and reconstructed image. We used a similarity based on a topological data analysis tool known as the persistence diagram, determining the bottleneck distance between the original and the reconstructed image as the measure of similarity. Our investigation was conducted on the common images utilized in image processing tasks. The findings indicate that there is no direct correlation between the number of moments and the quality of image reconstruction. It is necessary to choose an appropriate number of moments, which may be very small, instead of calculating a high number of image moments.

Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know.

The rapid development of deep learning in medical imaging has significantly enhanced the capabilities of artificial intelligence while simultaneously introducing challenges, including the need for vast amounts of training data and the labor-intensive tasks of labeling and segmentation. Generative adversarial networks (GANs) have emerged as a solution, offering synthetic image generation for data augmentation and streamlining medical image processing tasks through models such as cGAN, CycleGAN, and StyleGAN. These innovations not only improve the efficiency of image augmentation, reconstruction, and segmentation, but also pave the way for unsupervised anomaly detection, markedly reducing the reliance on labeled datasets. Our investigation into GANs in medical imaging addresses their varied architectures, the considerations for selecting appropriate GAN models, and the nuances of model training and performance evaluation. This paper aims to provide radiologists who are new to GAN technology with a thorough understanding, guiding them through the practical application and evaluation of GANs in brain imaging with two illustrative examples using CycleGAN and pixel2style2pixel (pSp)-combined StyleGAN. It offers a comprehensive exploration of the transformative potential of GANs in medical imaging research. Ultimately, this paper strives to equip radiologists with the knowledge to effectively utilize GANs, encouraging further research and application within the field.

Tailoring Space-Time Nonlocality for Event-Based Image Processing Metasurfaces.

Analog computation with passive optical components can enhance processing speeds and reduce power consumption, recently attracting renewed interest thanks to the opportunities enabled by metasurfaces. Basic image processing tasks, such as spatial differentiation, have been recently demonstrated based on engineered nonlocalities in metasurfaces, but next-generation computational schemes require more advanced capabilities. Here, by simultaneously tailoring the nonlocal electromagnetic response of a metasurface in space and time, we demonstrate a passive ultrathin silicon-based device that performs mixed spatiotemporal differentiation of input images, realizing event-based edge detection. The metasurface performs spatial differentiation only when the input image is evolving in time, resulting in spatiotemporal image processing on subpicosecond timescales. Moreover, the metasurface design can be tailored to selectively enhance objects moving at desired speeds. Our results point towards fully passive processing of spatiotemporal signals, for highly compact neuromorphic cameras.

Implementation and Efficient Analysis of Preprocessing Techniques in Deep Learning for Image Classification

Deep learning models have recently been preferred to perform certain image-processing tasks. Recently, with the increasing radiation, heat, and poor lighting conditions, the raw image samples may contain noisy and ambiguous information. To process these images, the deep learning model requires a large number of data samples to learn the missing information from other clear data samples. This necessitates training the neural network with a huge dataset. The researchers are now attempting to filter and improve such noisy images via preprocessing in order to provide valid and accurate feature information to the neural network layers. However, certain research studies claim that some useful information may be lost when the image is not preprocessed with an appropriate filter or enhancement technique. The MSA (meta-synthesis and analysis) approach is utilized in this work to present the impact of the image processing applications done with and without preprocessing steps. Also, this work summarizes the existing deep learning-based image processing models utilizing or not preprocessing steps in their implementation. This work has also found that 85% of the existing techniques involve a preprocessing step while developing a deep learning model. However, a maximum accuracy of 96.89% is observed on Sine-Net when it is implemented without a preprocessing and the same model gave 96.85% when implemented with preprocessing. This research provides various research insights on the requirement and non-requirement of preprocessing steps in a deep learning-based implementation.