Image Information Research Articles

A study of blind denoising algorithms for two-scale real images based on partial differential equations

In order to better preserve the details and texture information in the image, a dual scale real image blind denoising algorithm based on partial differential equations is studied. Input real image information, perform small-scale denoising based on differential curvature partial differential equations, and combine with a new diffusion coefficient function to distinguish the edges, flat areas, and noise of the image, allowing the algorithm to retain more subtle information such as weak edges and textures while removing noise. For the large-scale information in the input real image, a multi-stage partial differential equation is used for denoising processing. Based on the mixed denoising method of 8-neighborhood and implicit curvature, a weight function is constructed to control the proportion of the two types of equations in image denoising, effectively achieving the goal of large-scale image denoising. The experimental results show that in the MATLAB coding platform, this algorithm can eliminate different types of noise, preserve more edge and detail information of the image, and improve the registration and recognition accuracy in the image application process.

Learning Many-to-Many Mapping for Unpaired Real-World Image Super-Resolution and Downscaling.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

A fast differential network with adaptive reference sample for gaze estimation

Most non-invasive gaze estimation methods do not consider the inter-individual differences in anatomical structure, but directly regress the gaze direction from the appearance image information, which limits the accuracy of individual-independent gaze estimation networks. In addition, existing gaze estimation methods tend to consider only how to improve the model’s generalization performance, ignoring the crucial issue of efficiency, which leads to bulky models that are difficult to deploy and have questionable cost-effectiveness in practical use. This paper makes the following contributions: (1) A differential network for gaze estimation using adaptive reference samples is proposed, which can adaptively select reference samples based on scene and individual characteristics. (2) The knowledge distillation is used to transfer the knowledge structure of robust teacher networks into lightweight networks so that our networks can execute quickly and at low computational cost, dramatically increasing the prospect and value of applying gaze estimation. (3) Integrating the above innovations, a novel fast differential neural network (Diff-Net) named FDAR-Net is constructed and achieved excellent results on MPIIGaze, UTMultiview and EyeDiap.

Learning Disentangled Representation for One-Shot Progressive Face Swapping.

Although face swapping has attracted much attention in recent years, it remains a challenging problem. Existing methods leverage a large number of data samples to explore the intrinsic properties of face swapping without considering the semantic information of face images. Moreover, the representation of the identity information tends to be fixed, leading to suboptimal face swapping. In this paper, we present a simple yet efficient method named FaceSwapper, for one-shot face swapping based on Generative Adversarial Networks. Our method consists of a disentangled representation module and a semantic-guided fusion module. The disentangled representation module comprises an attribute encoder and an identity encoder, which aims to achieve the disentanglement of the identity and attribute information. The identity encoder is more flexible, and the attribute encoder contains more attribute details than its competitors. Benefiting from the disentangled representation, FaceSwapper can swap face images progressively. In addition, semantic information is introduced into the semantic-guided fusion module to control the swapped region and model the pose and expression more accurately. Experimental results show that our method achieves state-of-the-art results on benchmark datasets with fewer training samples.

Hybrid improved fuzzy C-means and watershed segmentation to classify Alzheimer’s using deep learning

Brain damage and deficits in interactions among brain cells are the primary causes of dementia and Alzheimer’s disease (AD). Despite ongoing research, no effective medications have yet been developed for these conditions. Therefore, early detection is crucial for managing the progression of these disorders. In this study, we introduce a novel tool for detecting AD using non invasive medical tests, such as magnetic resonance imaging (MRI). Our method employs fuzzy C-means clustering to identify features that enhance image accuracy. The standard fuzzy C-means algorithm has been augmented with fuzzy components to improve clustering performance. This enhanced approach optimizes segmentation by extracting image information and utilizing a sliding window to calculate center coordinates and establish a stable group matrix. These critical features are subsequently integrated with a two-phase watershed segmentation process. The resulting segmented images are then used to train an optimal convolutional neural network (CNN) for AD classification. Our methodology demonstrated a 98.20% accuracy rate in the detection and classification of segmented MRI brain images, highlighting its efficacy in identifying disease types.

Image Dehazing Technique Based on DenseNet and the Denoising Self-Encoder

The application value of low-quality photos taken in foggy conditions is significantly lower than that of clear images. As a result, restoring the original image information and enhancing the quality of damaged images on cloudy days are crucial. Commonly used deep learning techniques like DehazeNet, AOD-Net, and Li have shown encouraging progress in the study of image dehazing applications. However, these methods suffer from a shallow network structure leading to limited network estimation capability, reliance on atmospheric scattering models to generate the final results that are prone to error accumulation, as well as unstable training and slow convergence. Aiming at these problems, this paper proposes an improved end-to-end convolutional neural network method based on the denoising self-encoder-DenseNet (DAE-DenseNet), where the denoising self-encoder is used as the main body of the network structure, the encoder extracts the features of haze images, the decoder performs the feature reconstruction to recover the image, and the boosting module further performs the feature fusion locally and globally, and finally outputs the dehazed image. Testing the defogging effect in the public dataset, the PSNR index of DAE-DenseNet is 22.60, which is much higher than other methods. Experiments have proved that the dehazing method designed in this paper is better than other algorithms to a certain extent, and there is no color oversaturation or an excessive dehazing phenomenon in the image after dehazing. The dehazing results are the closest to the real image and the viewing experience feels natural and comfortable, with the image dehazing effect being very competitive.

Digital Architectural Form Generation Through Pixel System-Driven Image Feature Information

This study analyzes valuable information (image feature information) from pixel-based images to systematize the form generation process utilizing this information. Information in architecture is mainly used as an analytical tool for functional design optimization. However, the Fourth Industrial Revolution and advances in information and communication technology have positioned information as a core value-creating tool. Consequently, there is a growing need to develop methodologies for effectively applying information to architectural design generation. This study combines digital technology with images, which directly influence architectural design, transforming them into numerical information. By systematizing the three-dimensional form generation process using low-level image feature information, it analyzes the relationship between the characteristics of valuable information and the generated forms. This approach helps identify the interrelationships between the characteristics of valuable information and the generated forms, the complementarity needed to mitigate the limitations of valuable information, and the potential for architectural applications of the three-dimensional form generation methodology utilizing image information. This approach reduces labor and enhances architectural aesthetics by instantly generating diverse early design alternatives. Such research can provide a more logical basis for avoiding intuitive judgments during the design decision-making process. It can also lay the groundwork for information-centric design systems in architecture.

Early detection of verticillium wilt in eggplant leaves by fusing five image channels: a deep learning approach.

As one of the world's most important vegetable crops, eggplant production is often severely affected by verticillium wilt, leading to significant declines in yield and quality. Traditional multispectral disease-imaging equipment is expensive and complicated to operate. Low-cost multispectral devices cannot capture images and cover less information. The traditional approach to early disease diagnosis involves using multispectral disease-imaging equipment in conjunction with machine learning technology. However, this approach has significant limitations in early disease diagnosis, including challenges such as high costs, complex operation, and low model performance. The aim of this study was to combine low-cost multispectral cameras with deep learning technology to detect early stage Verticillium wilt in eggplant effectively. Using the Manual FS-3200T-10GE-NNC multispectral camera to perform multispectral imaging of the leaves of eggplant seedlings at the early infection stage, information fusion was performed on the collected multispectral images, and a five-channel image information fusion model was established. Image information fusion technology was combined with deep learning technology, among which the VGG16-triplet attention model performed the best, achieving a precision of 86.73% on the test set. Model validation on 48- and 72-hour data reached a precision of 75% and 82%, respectively, achieving an early diagnosis of Verticillium wilt. This highlighted the potential of multispectral cameras for early disease detection. In this study, we successfully developed a method for the non-destructive detection of the early stages of eggplant wilt disease by combining multispectral imaging technology with deep learning algorithms. While ensuring high accuracy, this method significantly reduces the cost of experimental equipment. The application of this method can reduce the cost of agricultural equipment and provide a scientific basis for agricultural production practices, helping to reduce losses caused by diseases.

The Impact of Movie Text Information on Movie Recommendation Systems based on Multi-Armed Bandit

Abstract. The multi-armed bandit problem has been extensively studied and applied in various fields such as recommendation systems and resource allocation. However, there is still a lack of research in the movie recommendation domain, such as the comparison of algorithm performance and the impact of movie text information. This paper uses the Movielens 100K movie review data set to compare and analyze the performance of classic algorithms-Greedy, UCB, and TS-in the multi-armed bandit problem. Additionally, to explore the influence of movie text information (such as movie summaries and user reviews) on system algorithms, this experiment utilizes web scraping technology through HTTP requests and HTML parsing, to extract data from the Douban website, obtaining the information and forming a new data set. In this movie review data set, the Greedy algorithm outperforms both UCB and TS algorithms. This study finds that incorporating movie text information improves the accuracy of all three algorithms, thus contributing to the enhancement of movie recommendation systems. This experiment did not consider larger datasets or movie image information, which could be addressed in future improvements.

Nonlinear multi-image optical authentication based on QR decomposition and Kramer-Kronig relations

Abstract In this paper, a new nonlinear optical multi-image authentication scheme is proposed based on Kramers-Kronig digital holography and orthogonal triangular decomposition or QR decomposition. Here, the complex light field carrying the information of multiple images is modulated by random phase masks and propagated at certain distance. Afterwards, the QR decomposition is applied to the complex wavefront to generate the private keys and to add the non-linearity in the scheme. Next, the product of orthogonal matrix and upper triangular matrix is processed further. The obtained output is modulated by different phase masks and interfered with reference beam to record the encrypted image. For decryption, the Kramer-Kronig relation is utilized to extract the plaintext images directly with only the positive frequency part. A series of numerical simulations are conducted to validate the efficacy and robustness of proposed image authentication scheme.

Role of visual information in multimodal large language model performance: an evaluation using the Japanese nuclear medicine board examination.

This study aimed to assess the performance of state-of-the-art multimodal large language models (LLMs), specifically GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro, on Japanese Nuclear Medicine Board Examination (JNMBE) questions and to evaluate the influence of visual information on the decision-making process. This study utilized 92 questions with images from the JNMBE (2019-2023). The LLMs' responses were assessed under two conditions: providing both text and images and providing only text. Each model answered all questions thrice, and the most frequent answer choice was considered the final answer. The accuracy and agreement rates among the model answers were evaluated using statistical tests. GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro exhibited no significant differences in terms of accuracy between the text-and-image and text-only conditions. GPT-4o and Claude 3 Opus demonstrated accuracies of 54.3% (95% CI: 44.2%-64.1%) each when provided with both text and images; however, they selected the same options as in the text-only condition for 71.7% of the questions. Gemini 1.5 Pro performed significantly worse than GPT-4o under text and image conditions. The agreement rates among the model answers ranged from weak to moderate. The influence of images on decision-making in nuclear medicine is limited to the latest multimodal LLMs, and their diagnostic ability in this highly specialized field remains insufficient. Improving the utilization of image information and enhancing the answer reproducibility are crucial for the effective application of LLMs in nuclear medicine education and practice. Further advancements in these areas are necessary to harness the potential of LLMs as assistants in nuclear medicine diagnosis.

Ultra-security optical image encryption using speckles through a multi-mode fiber

To realize optical image encryption for long-distance transmission while considering its security performance, an optical image encryption scheme is proposed in this paper. In the scheme, the pixel information of the plaintext image is first encoded by orbital angular momentum (OAM) holograms; then, the information-coded OAM beam is transmitted through a 1 km multimode fiber to generate speckles as ciphertexts for encryption; and finally, a pre-trained deep learning model capable of learning the relationship between the output speckles and the input information-coded OAM modes is used for decryption. The proposed scheme is not only able to achieve high fidelity recovered image but also a remarkable level of security. The high security stems from the combined use of three keys: the order key, the model key, and the coder key, during optical encryption. The image can therefore only be decrypted by authorized users who simultaneously know the three keys. We have experimentally demonstrated the high fidelity and high security encryption/decryption capabilities. Our work can provide a promising avenue for further research into long-distance optical image transmission and encryption with ultra-high security.

Combined query embroidery image retrieval based on enhanced CNN and blend transformer.

Embroidery images carry rich historical information and are an important form of embroidery art. In the field of combination query image retrieval, how to efficiently retrieve the embroidery image information required by users has become a current research challenge. In recent years, convolutional neural networks (CNNs) have achieved significant success in image feature extraction, but they tend to focus on local information, making it easy to ignore global context information when processing such textured embroidery images. Therefore, we propose a combination query retrieval method for embroidery images. First, we propose Blend-Transformer, which introduces Group External Attention (GEA). GEA can integrate feature information from three different dimensions, effectively capturing the local and global context information of embroidery images. Second, we propose Enhanced CNN, which introduces Shuffle Attention (SA), regrouping the reference image features extracted by CNN and reaggregating them by channel to enhance the richness of embroidery image feature information. Through experiments on the TCE-S and ICR2020 standard datasets, we verify the excellent performance of the proposed algorithm in embroidery image retrieval. Our method fills the gap in embroidery image retrieval research and provides a new perspective for the protection of embroidery art.

Feature Intensification Using Perception-Guided Regional Classification for Remote Sensing Image Super-Resolution

In recent years, super-resolution technology has gained widespread attention in the field of remote sensing. Despite advancements, current methods often employ uniform reconstruction techniques across entire remote sensing images, neglecting the inherent variability in spatial frequency distributions, particularly the distinction between high-frequency texture regions and smoother areas, leading to computational inefficiency, which introduces redundant computations and fails to optimize the reconstruction process for regions of higher complexity. To address these issues, we propose the Perception-guided Classification Feature Intensification (PCFI) network. PCFI integrates two key components: a compressed sensing classifier that optimizes speed and performance, and a deep texture interaction fusion module that enhances content interaction and detail extraction. This network mitigates the tendency of Transformers to favor global information over local details, achieving improved image information integration through residual connections across windows. Furthermore, a classifier is employed to segment sub-image blocks prior to super-resolution, enabling efficient large-scale processing. The experimental results on the AID dataset indicate that PCFI achieves state-of-the-art performance, with a PSNR of 30.87 dB and an SSIM of 0.8131, while also delivering a 4.33% improvement in processing speed compared to the second-best method.

Impact of RAW vs. RGB images on low-light object detection in autonomous driving

Abstract. In the field of autonomous driving, object detection in low-light conditions presents a critical challenge. Traditional detection algorithms, reliant on RGB imagery, often suffer from diminished performance under inadequate illumination. In contrast, RAW images, which retain a higher level of intrinsic image information, have the potential to enhance detection accuracy. Therefore, the paper aims to assess whether models trained on RAW images exhibit enhanced object detection performance in low-light scenarios. To this end, experiments are conducted utilizing the Low-light Object Detection (LOD) dataset, which includes paired RAW and RGB images, with variations in lighting simulated through different exposure times. Two distinct datasets are constructed: one consisting of RAW-normal and RAW-dark images, and another comprising RGB-normal and RGB-dark images. The models evaluated in this research include YOLOv8, Faster R-CNN (Region-based Convolutional Neural Network), and EfficientDet, selected to ensure robustness and generalizability of the findings. The results demonstrate that models trained on RAW images significantly outperform those trained on RGB images, thus exhibiting higher accuracy both in general and under low-light conditions. The preservation of detailed image information in RAW format facilitates enhanced feature extraction, thereby improving detection accuracy and resilience under low-light conditions. These findings provide novel insights into advancing object detection methodologies for autonomous driving systems operating in challenging lighting environments.

DIRformer: A Novel Image Restoration Approach Based on U-shaped Transformer and Diffusion Models

Image restoration (IR) involves the retrieval of missing or damaged image information and represents a significant challenge in the field of visual reconstruction. Currently, U-Net based Diffusion Models (DMs) display favorable results when utilized for IR tasks. However, the Diffusion Model (DM) based on U-Net demonstrates shortcomings in capturing the global context for IR. To address this issue, we propose a Novel Image Restoration Approach Based on U-shaped Transformer and Diffusion Models (DIRformer). DIRformer enhances the modeling capacity for long-range dependencies within DMs. In particular, DIRformer replaces the traditional U-Net downsampling with Patch merging, dedicated to improving detail preservation, and replaces upsampling with Dual up-sample, strategically designed to alleviate checkerboard artifacts. Besides, as a lightweight and versatile transformer-based solution for IR, DIRformer incorporates time and degradation mapping into the transformer design, all while preserving the fundamental U-shaped structural framework. We assess the efficacy of DIRformer in a multi-tasking IR setting across four datasets. The experimental performance illustrates that DIRformer achieves competitive performance on distortion metrics, including PSNR and SSIM. Remarkably, our proposed approach is almost 25× smaller and 2× faster than the existing methods while achieving comparable high performance.

Satellite Image Restoration via an Adaptive QWNNM Model

Due to channel noise and random atmospheric turbulence, retrieved satellite images are always distorted and degraded and so require further restoration before use in various applications. The latest quaternion-based weighted nuclear norm minimization (QWNNM) model, which utilizes the idea of low-rank matrix approximation and the quaternion representation of multi-channel satellite images, can achieve image restoration and enhancement. However, the QWNNM model ignores the impact of noise on similarity measurement, lacks the utilization of residual image information, and fixes the number of iterations. In order to address these drawbacks, we propose three adaptive strategies: adaptive noise-resilient block matching, adaptive feedback of residual image, and adaptive iteration stopping criterion in a new adaptive QWNNM model. Both simulation experiments with known noise/blurring and real environment experiments with unknown noise/blurring demonstrated that the effectiveness of adaptive QWNNM models outperformed the original QWNNM model and other state-of-the-art satellite image restoration models in very different technique approaches.

Speech recognition using an english multimodal corpus with integrated image and depth information

Traditional English corpora mainly collect information from a single modality, but lack information from multimodal information, resulting in low quality of corpus information and certain problems with recognition accuracy. To solve the above problems, this paper proposes to introduce depth information into multimodal corpora, and studies the construction method of English multimodal corpora that integrates electronic images and depth information, as well as the speech recognition method of the corpus. The multimodal fusion strategy adopted integrates speech signals and image information, including key visual information such as the speaker’s lip movements and facial expressions, and uses deep learning technology to mine acoustic and visual features. The acoustic model in the Kaldi toolkit is used for experimental research.Through experimental research, the following conclusions were drawn: Under 15-dimensional lip features, the accuracy of corpus A under monophone model was 2.4% higher than that of corpus B under monophone model when the SNR (signal-to-noise ratio) was 10dB, and the accuracy of corpus A under the triphone model at the signal-to-noise ratio of 10dB was 1.7% higher than that of corpus B under the triphone model at the signal-to-noise ratio of 10dB. Under the 32-dimensional lip features, the speech recognition effect of corpus A under the monophone model at the SNR of 10dB was 1.4% higher than that of corpus B under the monophone model at the SNR of 10dB, and the accuracy of corpus A under the triphone model at the SNR of 10dB was 2.6% higher than that of corpus B under the triphone model at the SNR of 10dB. The English multimodal corpus with image and depth information has a high accuracy, and the depth information helps to improve the accuracy of the corpus.

MSL-CCRN: Multi-stage self-supervised learning based cross-modality contrastive representation network for infrared and visible image fusion

Infrared and visible image fusion (IVIF) facing different information in two modal scenarios, the focus of research is to better extract different information. In this work, we propose a multi-stage self-supervised learning based cross-modality contrastive representation network for infrared and visible image fusion (MSL-CCRN). Firstly, considering that the scene differences between different modalities affect the fusion of cross-modal images, we propose a contrastive representation network (CRN). CRN enhances the interaction between the fused image and the source image, and significantly improves the similarity between the meaningful features in each modality and the fused image. Secondly, due to the lack of ground truth in IVIF, the quality of directly obtained fused image is seriously affected. We design a multi-stage fusion strategy to address the loss of important information in this process. Notably, our method is a self-supervised network. In fusion stage I, we reconstruct the initial fused image as the new view of fusion stage II. In fusion stage II, we use the fused image obtained in the previous stage to carry out three-view contrastive representation, thereby constraining the feature extraction of the source image. This makes the final fused image introduce more important information in the source image. Through a large number of qualitative, quantitative experiments and downstream object detection experiments, our propose method shows excellent performance compared with most advanced methods.

Predicting Immune Checkpoint Inhibitor-Related Pneumonitis via Computed Tomography and Whole-Lung Analysis Deep Learning

Background: Immune checkpoint inhibitor-related pneumonitis (ICI-P) is a fatal adverse event of immunotherapy. However, there is a lack of methods to identify patients who have a high risk of developing ICI-P in immunotherapy. Purpose: We aim at predicting the individualized risk of developing ICI-P by computed tomography (CT) images and deep learning to assist in personalized immunotherapy planning. Methods: We first explored the prognostic value of the commonly used clinical factors. Moreover, we proposed a novel whole-lung analysis deep learning (DL) model, which is constructed using a combination of Densely Connected Convolutional Networks (DenseNet) and Feature Pyramid Networks (FPN). This DL model mines global lung information from CT images for predicting the risk of developing ICI-P, and it is fully automated and does not require manually annotating images. Finally, 157 patients were collected and randomly divided into training and testing sets for performance evaluation. Results: In the testing set, the clinical model achieved an Area Under the Curve (AUC) of 0.710 and accuracy of 0.625. By mining global lung information, the DL model achieved AUC=0.780 and accuracy=0.729 in the testing set, where the DL score revealed a significant difference between ICI-P and non-ICI-P patients. Through deep learning visualization technique, we found that many areas outside of tumor (e.g., pleural retraction, pleural effusion, and the abnormalities in vessels) are important for predicting the risk of developing ICI-P in immunotherapy. Conclusions: The whole-lung analysis DL model provides an easy-to-use method for identifying patients at high risk of developing ICI-P by CT images, which is important for individualized treatment planning in immunotherapy. The performance improvement over the clinical model indicates that mining whole-lung information in CT images is effective for prognostic prediction in immunotherapy.