Image Information Research Articles

A chest imaging diagnosis report generation method based on dual-channel transmodal memory network

Medical imaging is critical for clinical diagnosis. In the past few years, medical imaging data has been steadily growing at an annual rate of 30%, while the annual growth rate in the number of radiologists is only 4. 1%. The traditional manual methods of chest imaging reports result in high information load and workload with low relevance of images and texts. Here we propose a chest CT imaging diagnosis report generation network to effectively integrate image and text feature information, improve the relevance of the cross-modal networks and automatically generate imaging diagnosis reports. The network includes three major modules: a novel ResNetII for information extraction, an image–text dual-channel transmodal memory network (DCTMN), and a dual-channel decoder module combined with LSTM gate. We validate the effectiveness of the proposed network on the IU X-ray and the MIMIC-CXR datasets. Our analysis indicates that the proposed method can strengthen the relationship between chest imaging and text information, ultimately achieving the automatic generation of chest imaging diagnosis reports.

A high-quality visual image encryption algorithm utilizing the conservative chaotic system and adaptive embedding

Regarding the issue of encrypted images in the channel attracting the attention of attackers and making them vulnerable to attacks. This paper proposes a high-quality visual image encryption algorithm utilizing the conservative chaotic system, two-dimensional compressive sensing, optimization local of binary patterns, and adaptive embedding method. Firstly, a conservative chaotic system with excellent performance and resistance to reconstruction attacks is proposed. Secondly, the pseudo-random sequences generated by the chaotic system dynamically generate measurement matrices, which are optimized before compressing the image. Then, during the encryption process, by utilizing the newly proposed composite DNA computing rules, chaotic sequences dynamically encode and compute image information, which enriches the coding criteria and improves the security of encryption algorithms. Finally, in the information embedding stage, the newly proposed OLBP algorithm can identify important textured and non-textured regions of the host image, prioritize embedding non-textured regions, and then embed textured regions, which can improve the amount of information embedding and reduce damage to the host image. Experimental simulation and analysis exhibit that the encryption algorithm has high security, strong robustness and high efficiency. Meanwhile the imperceptible analysis of the steganographic images is exceed 52 dB, so the algorithm presents a high-quality visual effect of imperceptibility.

APPLICATION OF MACHINE LEARNING TECHNIQUES TO INCREASE THE LEVEL OF ACCURACY OF OPTICAL CHARACTER RECOGNITION RESULTS

One of the most pervasive processes of modernity is undoubtedly digitalization, which has encompassed all key spheres of human life. The development of information technology has contributed to large-scale changes not only in the everyday aspect of life, but also more globally, automating complex business processes in the field of entrepreneurship, economics, and healthcare. The transition to digital data and documentation has provided greater accessibility to necessary information and has also enhanced the efficiency of its analysis and processing. Due to this fact, optical character recognition (OCR) technology has gained significant importance, enabling the identification and extraction of textual data from images. OCR systems play a pivotal role in the digital transformation of society as they eliminate the need for manual handling of textual information in images and are applicable in automating the majority of business processes associated with paper-based data processing, such as gathering statistical data from paper forms, reflecting paper documents in electronic document management systems, converting textual information into audio files, and so on. This paper is dedicated to describing optical character recognition technology, as well as providing an overview of machine learning techniques that are actively used in the context of its modern implementation, in order to enhance the quality of the obtained results. In addition, the paper presents the principles of operation of the described approaches, their capabilities, as well as some limitations that may be encountered when using them in various scenarios.

Hue Guidance Network for Single Image Reflection Removal.

Reflection from glasses is ubiquitous in daily life, but it is usually undesirable in photographs. To remove these unwanted noises, existing methods utilize either correlative auxiliary information or handcrafted priors to constrain this ill-posed problem. However, due to their limited capability to describe the properties of reflections, these methods are unable to handle strong and complex reflection scenes. In this article, we propose a hue guidance network (HGNet) with two branches for single image reflection removal (SIRR) by integrating image information and corresponding hue information. The complementarity between image information and hue information has not been noticed. The key to this idea is that we found that hue information can describe reflections well and thus can be used as a superior constraint for the specific SIRR task. Accordingly, the first branch extracts the salient reflection features by directly estimating the hue map. The second branch leverages these effective features, which can help locate salient reflection regions to obtain a high-quality restored image. Furthermore, we design a new cyclic hue loss to provide a more accurate optimization direction for the network training. Experiments substantiate the superiority of our network, especially its excellent generalization ability to various reflection scenes, as compared with state-of-the-arts both qualitatively and quantitatively. Source codes are available at https://github.com/zhuyr97/HGRR.

Infrared and visible image fusion based on hybrid multi-scale decomposition and adaptive contrast enhancement

Effectively fusing infrared and visible images enhances the visibility of infrared target information while capturing visual details. Balancing the brightness and contrast of the fusion image adequately has posed a significant challenge. Moreover, preserving detailed information in fusion images has been problematic. To address these issues, this paper proposes a fusion algorithm based on multi-scale decomposition and adaptive contrast enhancement. Initially, we present a hybrid multi-scale decomposition method aimed at extracting valuable information comprehensively from the source image. Subsequently, we advance an adaptive base layer optimization approach to regulate the brightness and contrast of the resultant fusion image. Lastly, we design a weight mapping rule grounded in saliency detection to integrate small-scale layers, thereby conserving the edge structure within the fusion outcome. Both qualitative and quantitative experimental results affirm the superiority of the proposed method over 11 state-of-the-art image fusion methods. Our method excels in preserving more texture and achieving higher contrast, which proves advantageous for monitoring tasks.

Welding defects extraction method by fusing saliency information of mid-level and underlying level images

X-ray non-destructive testing (NDT) technology is extensively utilized in the welding industry for the detection of weld defects. This paper proposes a novel defect segmentation algorithm to address the challenges of X-ray defect detection images, including low contrast, blurred edges, significant noise, and pronounced background variations. Traditional detection methods often struggle to extract low-contrast defects from weld images, so this approach integrates both underlying and mid-level image information to enhance accuracy. The process begins with a visual saliency model that generates a rough saliency map from underlying details. Next, a Pulse Coupled Neural Network (PCNN) is used to compute the saliency map at the mid-level. Finally, these two saliency maps are combined using a pixel-minimum method to produce the final image saliency map. Experimental results show that this method is highly accurate, broadly applicable, and capable of rapid defect extraction within the welding area.

Gm-APD Lidar Distance Image Recovery Based on Intensity Image Target Edge Guidance

Abstract Gm-APD LiDAR in the detection of long-distance targets, the laser by the atmospheric effects of attenuation is serious, so that the target echo sparse, easy to cause the target distance like edge missing, details of fuzzy and other problems, the traditional method can’t make targeted treatment of the above problems. Therefore, this paper proposes a Gm-APD LIDAR distance image recovery method based on intensity image target edge guidance. The method introduces one-dimensional intensity information as an aid, combines the inverse distance weighting function to guide the reconstruction of the target edge contour information of the distance image, and then applies morphological image processing to correct the target distance value in a partition, and recovers a distance image with sharp edges and clear details. Experiments are carried out on the distance targets, and the results show that the method improves the target restoration by 19.21% and the peak signal-to-noise ratio by 3.12% compared with the concave morphology restoration algorithm, which verifies the advantages and effectiveness of the method proposed in this paper.

Pulp grade monitoring using binocular image through multi-scale feature cross-attention fusion network and saliency map constraint

Pulp grade is an important indicator for performance monitoring in froth flotation process. Previous studies have shown that the pulp grade can be predicted more accurately by using the stereo vision information of froth images. However, due to the low intra-class variation among froth images under the identical working conditions and the similarity in shape and texture among bubbles, it is challenging for current binocular image-based methods for grade prediction. Therefore, to accurately predict the key performance indicators in flotation process, a prediction model based on binocular image fusion is proposed in this paper. First, a calculation method of froth image saliency map is proposed, and the saliency map is used as a priori knowledge to guide the prediction model to learn the characteristics of the region of interest in the image. This measure aims to solve the problem of difficulty in grade prediction caused by low intra-class differences in froth images. Then, a multi-scale feature cross-attention fusion network is introduced, wherein the multi-scale features of the left and right views serve as attention mechanism gating signals to extract common feature from binocular image. After that, a pulp grade prediction model is developed based on Video Transformer Network. The results on actual industrial flotation datasets show that, compared with other pulp grade prediction methods, our proposed approach reduces the mean absolute error and root mean square error by 22.84% and 23.55%, respectively.

Multimodal dual perception fusion framework for multimodal affective analysis

The misuse of social platforms and the difficulty in regulating post contents have culminated in a surge of negative sentiments, sarcasms, and the rampant spread of fake news. In response, Multimodal sentiment analysis, sarcasm detection and fake news detection based on image and text have attracted considerable attention recently. Due to that these areas share semantic and sentiment features and confront related fusion challenges in deciphering complex human expressions across different modalities, integrating these multimodal classification tasks that share commonalities across different scenarios into a unified framework is expected to simplify research in sentiment analysis, and enhance the effectiveness of classification tasks involving both semantic and sentiment modeling. Therefore, we consider integral components of a broader spectrum of research known as multimodal affective analysis towards semantics and sentiment, and propose a novel multimodal dual perception fusion framework (MDPF). Specifically, MDPF contains three core procedures: (1) Generating bootstrapping language-image Knowledge to enrich origin modality space, and utilizing cross-modal contrastive learning for aligning text and image modalities to understand underlying semantics and interactions. (2) Designing dynamic connective mechanism to adaptively match image-text pairs and jointly employing gaussian-weighted distribution to intensify semantic sequences. (3) Constructing a cross-modal graph to preserve the structured information of both image and text data and share information between modalities, while introducing sentiment knowledge to refine the edge weights of the graph to capture cross-modal sentiment interaction. We evaluate MDPF on three publicly available datasets across three tasks, and the empirical results demonstrate the superiority of our proposed model.

PST-Diff: Achieving High-Consistency Stain Transfer by Diffusion Models With Pathological and Structural Constraints.

Histopathological examinations heavily rely on hematoxylin and eosin (HE) and immunohistochemistry (IHC) staining. IHC staining can offer more accurate diagnostic details but it brings significant financial and time costs. Furthermore, either re-staining HE-stained slides or using adjacent slides for IHC may compromise the accuracy of pathological diagnosis due to information loss. To address these challenges, we develop PST-Diff, a method for generating virtual IHC images from HE images based on diffusion models, which allows pathologists to simultaneously view multiple staining results from the same tissue slide. To maintain the pathological consistency of the stain transfer, we propose the asymmetric attention mechanism (AAM) and latent transfer (LT) module in PST-Diff. Specifically, the AAM can retain more local pathological information of the source domain images, while ensuring the model's flexibility in generating virtual stained images that highly confirm to the target domain. Subsequently, the LT module transfers the implicit representations across different domains, effectively alleviating the bias introduced by direct connection and further enhancing the pathological consistency of PST-Diff. Furthermore, to maintain the structural consistency of the stain transfer, the conditional frequency guidance (CFG) module is proposed to precisely control image generation and preserve structural details according to the frequency recovery process. To conclude, the pathological and structural consistency constraints provide PST-Diff with effectiveness and superior generalization in generating stable and functionally pathological IHC images with the best evaluation score. In general, PST-Diff offers prospective application in clinical virtual staining and pathological image analysis.

HSMix: Hard and soft mixing data augmentation for medical image segmentation

Due to the high cost of annotation or the rarity of some diseases, medical image segmentation is often limited by data scarcity and the resulting overfitting problem. Self-supervised learning and semi-supervised learning can mitigate the data scarcity challenge to some extent. However, both of these paradigms are complex and require either hand-crafted pretexts or well-defined pseudo-labels. In contrast, data augmentation represents a relatively simple and straightforward approach to addressing data scarcity issues. It has led to significant improvements in image recognition tasks. However, the effectiveness of local image editing augmentation techniques in the context of segmentation has been less explored. Additionally, traditional data augmentation methods for local image editing augmentation methods generally utilize square regions, which cause a loss of contour information.We propose HSMix, a novel approach to local image editing data augmentation involving hard and soft mixing for medical semantic segmentation. In our approach, a hard-augmented image is created by combining homogeneous regions (superpixels) from two source images. A soft mixing method further adjusts the brightness of these composed regions with brightness mixing based on locally aggregated pixel-wise saliency coefficients. The ground-truth segmentation masks of the two source images undergo the same mixing operations to generate the associated masks for the augmented images.Our method fully exploits both the prior contour and saliency information, thus preserving local semantic information in the augmented images while enriching the augmentation space with more diversity. Our method is a plug-and-play solution that is model agnostic and applicable to a range of medical imaging modalities. Extensive experimental evidence has demonstrated its effectiveness in a variety of medical segmentation tasks. The source code is available in https://github.com/DanielaPlusPlus/HSMix.

Study on canopy extraction method for narrowband spectral images based on superpixel color gradation skewness distribution features

BackgroundCrop phenotype extraction devices based on multiband narrowband spectral images can effectively detect the physiological and biochemical parameters of crops, which plays a positive role in guiding the development of precision agriculture. Although the narrowband spectral image canopy extraction method is a fundamental algorithm for the development of crop phenotype extraction devices, developing a highly real-time and embedded integrated narrowband spectral image canopy extraction method remains challenging owing to the small difference between the narrowband spectral image canopy and background.MethodsThis study identified and validated the skewed distribution of leaf color gradation in narrowband spectral images. By introducing kurtosis and skewness feature parameters, a canopy extraction method based on a superpixel skewed color gradation distribution was proposed for narrowband spectral images. In addition, different types of parameter combinations were input to construct two classifier models, and the contribution of the skewed distribution feature parameters to the proposed canopy extraction method was evaluated to confirm the effectiveness of introducing skewed leaf color skewed distribution features.ResultsLeaf color gradient skewness verification was conducted on 4200 superpixels of different sizes, and 4190 superpixels conformed to the skewness distribution. The intersection over union (IoU) between the soil background and canopy of the expanded leaf color skewed distribution feature parameters was 90.41%, whereas that of the traditional Otsu segmentation algorithm was 77.95%. The canopy extraction method used in this study performed significantly better than the traditional threshold segmentation method, using the same training set, Y1 (without skewed parameters) and Y2 (with skewed parameters) Bayesian classifier models were constructed. After evaluating the segmentation effect of introducing skewed parameters, the average classification accuracies Acc_Y1 of the Y1 model and Acc_Y2 of the Y2 model were 72.02% and 91.76%, respectively, under the same test conditions. This indicates that introducing leaf color gradient skewed parameters can significantly improve the accuracy of Bayesian classifiers for narrowband spectral images of the canopy and soil background.ConclusionsThe introduction of kurtosis and skewness as leaf color skewness feature parameters can expand the expression of leaf color information in narrowband spectral images. The narrowband spectral image canopy extraction method based on superpixel color skewness distribution features can effectively segment the canopy and soil background in narrowband spectral images, thereby providing a new solution for crop canopy phenotype feature extraction.

Design and Implementation of Image Acquisition and Signal Transmission Functions based on FPGA

Abstract In recent years, video network monitoring systems have evolved rapidly, from the first generation of digital-analogue systems to the fully IP-based systems. Based on the FPGA core, this design focuses on image acquisition and information transmission. The FPGA expansion board contains several peripheral modules, including the OV7670 camera captures images, DRAM stores data, and FIFO handles cross-clock domain signal transmission. While the reference clock is 40ns, in this system, the clock period is 20ns, making two clock cycles equal to one reference clock cycle. Then, two counters are used, one is the reference clock and another is the line synchronization (cnt1) to count 800 clock cycles per line. The end of a line is indicated by end_cnt0, and FIFO data is stored in SDRAM. Configuring the VGA display driver allows image data to be transmitted and displayed on the VGA screen. The system features a simple structure, high resource utilization, and strong reliability.

Open Domain Target Detection Based on Image-text Alignment Large Model under Railway Smart Construction Site

Abstract Open domain target detection plays a crucial role in ensuring people’s life safety in Railway Smart Construction Site. This paper proposes a target detection method that is not limited by the number of labels, which is different from traditional models that are usually trained and tested on a fixed set of object categories (or a set of labels). Firstly, the pre-training GLIP model is utilized to train the collected image-text pairing data to generate grounding boxes. Secondly, the pseudo-labeled data formed by associating these boxes with the corresponding text descriptions were combined with the original image-text data to form an enhanced dataset. The final model performs inference based on the prompt and incoming image data to determine the localization information. The GLIP model effectively aligns image and text information and enhances its ability to recognize and handle problems by observing and understanding new situations.

Multi-round decentralized dataset distillation with federated learning for Low Earth Orbit satellite communication

Satellite communication and Low Earth Orbit (LEO) satellites are important components of the 6G network, widely used for Earth observation tasks due to their low cost and short return period, making them a key technology for 6G network connectivity. Due to limitations in satellite system technology and downlink bandwidth, it is not feasible to download all high-resolution image information to ground stations. Even in existing federated learning (FL) methods, sharing well-trained parts of the model can still bottleneck with increasing model size. To address these challenges, we propose a new federated learning framework (FL-M3D) for LEO satellite communication that employs multi-round decentralized dataset distillation techniques. It allows satellites to independently extract local datasets and transmit them to ground stations instead of exchanging model parameters. Communication costs depend only on the size of the synthesized dataset and do not increase with larger models. However, the heterogeneity of satellite datasets can lead to sample ambiguity and decreased model convergence speed. Therefore, we propose distilling the datasets to mitigate the negative effects of data heterogeneity. Through experiments using real-world image datasets, FL-M3D reduces communication volume in simulated satellite networks by approximately 49.84% and achieves improved model performance.

Image sterilization via overwriting

Sterilization refers to the method of removing hidden information from images, effectively eradicating malicious data embedded within digital media. Existing methods are hampered by dual limitations: insufficient resilience to cutting-edge information hidden methods, coupled with the challenge of image restoration in quality. In response to these challenges, this paper proposes a novel method composed of overwriting attack and diffusion optimization stages. Employing the concept of pixel destruction, the overwriting method conducts a more thorough attack on hidden information while preserving sufficient information for image reconstruction and further diffusion optimization. This strategy is not bound to any particular information hiding method, making it effective against a wide range of steganography and watermarking techniques by systematically disrupting every pixel in the image. Diffusion optimization makes full use of the generative capabilities of diffusion models trained on large datasets, enabling higher quality image enhancements for a variety of scene images.Our method stands out in comparison to state-of-the-art studies, as experimental outcomes demonstrate its capability to maintain superior image integrity while efficiently eliminating hidden information across various steganography and watermarking techniques. Notably, when confronted with deep robust watermarking schemes, our approach exhibits a significant improvement over preceding methods. For instance, the removal rate improves significantly from 39.42% to 87.78% with higher quality restoration when compared to SC-Net. Our method delivers superior outcomes in the task of image sterilization, while presents a new challenge to the robustness of image information hiding methods and contributes to network security.

DWTN: deep wavelet transform network for lightweight single image deraining

On rainy days the uncertainty of the shape and distribution of rain streaks can cause the images captured by RGB image-based measurement tools to be blurred and distorted. Thanks to the wavelet transform ability to provide spatial and frequency domain information about an image and its multidirectional and multiscale nature, it is widely used in traditional image enhancement methods. In image deraining the distribution of rain streaks is not only related to spatial domain features but is also closely related to frequency domain spatial features. However, deep learning-based rain removal models mainly rely on the spatial features of the image, and RGB data can hardly distinguish rain marks from image details, which leads to the loss of crucial image information during rain removal. We have developed a lightweight single-image rain removal model called the deep wavelet transform network (DWTN) to address this limitation. This method separates image details from rain images and can more effectively remove rain marks. The proposed DWTN has three significant contributions. First, DWTN uses the feature components after the wavelet transform as the input to the model and assigns a separate frequency-aware enhancement block (FAEB) to each element. These blocks focus on specific frequency features that benefit the rain removal task. Second, we introduce a frequency feature fusion block (FFFB) that fuses different wavelet components to reduce noise and enhance the image background through a channel attention mechanism while attenuating rain streaks. Finally, we design a spatial feature enhancement block (SFEB), which uses a spatial attention mechanism to calibrate the spatial position of features to improve the rain removal performance. We evaluate the performance of DWTN using PSNR and SSIM on four synthetic datasets and NIQE and BRISQUE on two real datasets. The results of the evaluation of the six datasets and at least four performance metrics show that the proposed DWTN is superior to existing methods.

IIS-FVIQA: Finger Vein Image Quality Assessment with intra-class and inter-class similarity

In recent years, Finger Vein Image Quality Assessment (FVIQA) has been recognized as an effective solution to the problem of erroneous recognition resulting from low image quality due to false and missing information in finger vein images, and has become an important part of finger vein recognition systems. Compared to traditional FVIQA methods that rely on domain knowledge, newer methods that reject low-quality images have been favored for their independence from human interference. However, these methods only consider intra-class similarity information and ignore valuable information from inter-class distribution, which is also an important factor in evaluating the performance of recognition systems. In this work, we propose a novel FVIQA approach, named IIS-FVIQA, which concurrently takes into account the intra-class similarity density and inter-class similarity distribution distance within recognition systems. Specifically, our method generates quality scores for finger vein images by combining the information entropy of intra-class similarity distribution and Wasserstein distance of inter-class distribution. Then, we train a regression network for quality prediction using training images and corresponding quality scores. When a new image enters the recognition system, the trained regression network directly predicts the quality score of the image, making it easier for the system to select the corresponding operation based on the quality score of the image. Extensive experiments conducted on benchmark datasets demonstrate that the IIS-FVIQA method proposed in this paper consistently achieves top performance across multiple public datasets. After filtering out 10% of low-quality images predicted by the quality regression network, the recognition system’s performance improves by 43.96% (SDUMLA), 32.23% (MMCBNU_6000), and 21.20% (FV-USM), respectively. Furthermore, the method exhibits strong generalizability across different recognition algorithms (e.g., LBP, MC, and Inception V3) and datasets (e.g., SDUMLA, MMCBNU_6000, and FV-USM).

Multi-Source Data-Driven Extraction of Urban Residential Space: A Case Study of the Guangdong–Hong Kong–Macao Greater Bay Area Urban Agglomeration

The accurate extraction of urban residential space (URS) is of great significance for recognizing the spatial structure of urban function, understanding the complex urban operating system, and scientific allocation and management of urban resources. The traditional URS identification process is generally conducted through statistical analysis or a manual field survey. Currently, there are also superpixel segmentation and wavelet transform (WT) processes to extract urban spatial information, but these methods have shortcomings in extraction efficiency and accuracy. The superpixel wavelet fusion (SWF) method proposed in this paper is a convenient method to extract URS by integrating multi-source data such as Point of Interest (POI) data, Nighttime Light (NTL) data, LandScan (LDS) data, and High-resolution Image (HRI) data. This method fully considers the distribution law of image information in HRI and imparts the spatial information of URS into the WT so as to obtain the recognition results of URS based on multi-source data fusion under the perception of spatial structure. The steps of this study are as follows: Firstly, the SLIC algorithm is used to segment HRI in the Guangdong–Hong Kong–Macao Greater Bay Area (GBA) urban agglomeration. Then, the discrete cosine wavelet transform (DCWT) is applied to POI–NTL, POI–LDS, and POI–NTL–LDS data sets, and the SWF is carried out based on different superpixel scale perspectives. Finally, the OSTU adaptive threshold algorithm is used to extract URS. The results show that the extraction accuracy of the NLT–POI data set is 81.52%, that of the LDS–POI data set is 77.70%, and that of the NLT–LDS–POI data set is 90.40%. The method proposed in this paper not only improves the accuracy of the extraction of URS, but also has good practical value for the optimal layout of residential space and regional planning of urban agglomerations.

Research on a Metal Surface Defect Detection Algorithm Based on DSL-YOLO

In industrial manufacturing, metal surface defect detection often suffers from low detection accuracy, high leakage rates, and false detection rates. To address these issues, this paper proposes a novel model named DSL-YOLO for metal surface defect detection. First, we introduce the C2f_DWRB structure by integrating the DWRB module with C2f, enhancing the model’s ability to detect small and occluded targets and effectively extract sparse spatial features. Second, we design the SADown module to improve feature extraction in challenging tasks involving blurred images or very small objects. Finally, to further enhance the model’s capacity to extract multi-scale features and capture critical image information (such as edges, textures, and shapes) without significantly increasing memory usage and computational cost, we propose the LASPPF structure. Experimental results demonstrate that the improved model achieves significant performance gains on both the GC10-DET and NEU-DET datasets, with a mAP@0.5 increase of 4.2% and 2.6%, respectively. The improvements in detection accuracy highlight the model’s ability to address common challenges while maintaining efficiency and feasibility in metal surface defect detection, providing a valuable solution for industrial applications.