Diverse Images Research Articles

PIPET: A Pipeline to Generate PET Phantom Datasets for Reconstruction Based on Convolutional Neural Network Training

There has been a strong interest in using neural networks to solve several tasks in PET medical imaging. One of the main problems faced when using neural networks is the quality, quantity, and availability of data to train the algorithms. In order to address this issue, we have developed a pipeline that enables the generation of voxelized synthetic PET phantoms, simulates the acquisition of a PET scan, and reconstructs the image from the simulated data. In order to achieve these results, several pieces of software are used in the different steps of the pipeline. This pipeline solves the problem of generating diverse PET datasets and images of high quality for different types of phantoms and configurations. The data obtained from this pipeline can be used to train convolutional neural networks for PET reconstruction.

Side-Scan Sonar Image Generation Under Zero and Few Samples for Underwater Target Detection

The acquisition of side-scan sonar (SSS) images is complex, expensive, and time-consuming, making it difficult and sometimes impossible to obtain rich image data. Therefore, we propose a novel image generation algorithm to solve the problem of insufficient training datasets for SSS-based target detection. For zero-sample detection, we proposed a two-step style transfer approach. The ray tracing method was first used to obtain an optically rendered image of the target. Subsequently, UA-CycleGAN, which combines U-net, soft attention, and HSV loss, was proposed for generating high-quality SSS images. A one-stage image-generation approach was proposed for few-sample detection. The proposed ADA-StyleGAN3 incorporates an adaptive discriminator augmentation strategy into StyleGAN3 to solve the overfitting problem of the generative adversarial network caused by insufficient training data. ADA-StyleGAN3 generated high-quality and diverse SSS images. In simulation experiments, the proposed image-generation algorithm was evaluated subjectively and objectively. We also compared the proposed algorithm with other classical methods to demonstrate its advantages. In addition, we applied the generated images to a downstream target detection task, and the detection results further demonstrated the effectiveness of the image generation algorithm. Finally, the generalizability of the proposed algorithm was verified using a public dataset.

Enhancing Medical Image Classification with Unified Model Agnostic Computation and Explainable AI

Background: Advances in medical image classification have recently benefited from general augmentation techniques. However, these methods often fall short in performance and interpretability. Objective: This paper applies the Unified Model Agnostic Computation (UMAC) framework specifically to the medical domain to demonstrate its utility in this critical area. Methods: UMAC is a model-agnostic methodology designed to develop machine learning approaches that integrate seamlessly with various paradigms, including self-supervised, semi-supervised, and supervised learning. By unifying and standardizing computational models and algorithms, UMAC ensures adaptability across different data types and computational environments while incorporating state-of-the-art methodologies. In this study, we integrate UMAC as a plug-and-play module within convolutional neural networks (CNNs) and Transformer architectures, enabling the generation of high-quality representations even with minimal data. Results: Our experiments across nine diverse 2D medical image datasets show that UMAC consistently outperforms traditional data augmentation methods, achieving a 1.89% improvement in classification accuracy. Conclusion: Additionally, by incorporating explainable AI (XAI) techniques, we enhance model transparency and reliability in decision-making. This study highlights UMAC’s potential as a powerful tool for improving both the performance and interpretability of medical image classification models.

WCAY object detection of fractures for X-ray images of multiple sites

The WCAY (weighted channel attention YOLO) model, which is meticulously crafted to identify fracture features across diverse X-ray image sites, is presented herein. This model integrates novel core operators and an innovative attention mechanism to enhance its efficacy. Initially, leveraging the benefits of dynamic snake convolution (DSConv), which is adept at capturing elongated tubular structural features, we introduce the DSC-C2f module to augment the model’s fracture detection performance by replacing a portion of C2f. Subsequently, we integrate the newly proposed weighted channel attention (WCA) mechanism into the architecture to bolster feature fusion and improve fracture detection across various sites. Comparative experiments were conducted, to evaluate the performances of several attention mechanisms. These enhancement strategies were validated through experimentation on public X-ray image datasets (FracAtlas and GRAZPEDWRI-DX). Multiple experimental comparisons substantiated the model’s efficacy, demonstrating its superior accuracy and real-time detection capabilities. According to the experimental findings, on the FracAtlas dataset, our WCAY model exhibits a notable 8.8% improvement in mean average precision (mAP) over the original model. On the GRAZPEDWRI-DX dataset, the mAP reaches 64.4%, with a detection accuracy of 93.9% for the “fracture” category alone. The proposed model represents a substantial improvement over the original algorithm compared to other state-of-the-art object detection models. The code is publicly available at https://github.com/cccp421/Fracture-Detection-WCAY .

Real-time joint recognition of weather and ground surface conditions by a multi-task deep network

Climate change and the occurrence of intense and unexpected weather events highlighted the need for real-time weather warning systems, especially in smart roads and isolated scenarios like rural areas. In this work, we propose to jointly recognize the weather and the ground surface conditions using existing video surveillance systems. Previous works separately tackled these two tasks even if they are correlated to each other. We propose a convolutional neural network with shared weights in the lower layers and two separate classification branches on top to exploit the correlation between the tasks and, at the same time, learn diverse high-level features for each task. Moreover, the network architecture implements attention mechanisms allowing the classification branches to focus on diverse image regions. The method is versatile and allows us to train the network on partially labeled data. The experimental analysis on real data demonstrate the effectiveness of the proposed method on both tasks, confirmed by the accuracy comparison with existing methods for the recognition of weather and ground surface conditions. The multi-task solution improves the inference speed (50 frames per second) and reduces the required memory (less than 1 GB) with respect to a system with two different single-task approaches; these results confirm that the proposed solution is ready for video surveillance applications to support smart cities.

Determinants of shared and idiosyncratic contributions to judgments of faces.

Recent work has shown that the idiosyncrasies of the observer can contribute more to the variance of social judgments of faces than the features of the faces. However, it is unclear what conditions determine the relative contributions of shared and idiosyncratic variance. Here, we examine two conditions: type of judgment and diversity of face stimuli. First, we show that for simpler, directly observable judgments that are consistent across observers (e.g., masculinity) shared exceeds idiosyncratic variance, whereas for more complex and less directly observable judgments (e.g., trustworthiness), idiosyncratic exceeds shared variance. Second, we show that judgments of more diverse face images increase the amount of shared variance. Finally, using machine-learning methods, we examine how stimulus (e.g., incidental emotion resemblance, skin luminosity) and observer variables (e.g., race, age) contribute to shared and idiosyncratic variance of judgments. Overall, our results indicate that an observer's age is the most consistent and best predictor of idiosyncratic variance contributions to face judgments measured in the current research. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

QualityNet: A multi-stream fusion framework with spatial and channel attention for blind image quality assessment

This study introduces a novel Blind Image Quality Assessment (BIQA) approach leveraging a multi-stream spatial and channel attention model. Our method addresses challenges posed by diverse image content and distortions by integrating feature maps from two distinct backbones. Through spatial and channel attention mechanisms, our algorithm prioritizes regions of interest, enhancing its ability to capture crucial image details. Extensive evaluations on four benchmark datasets demonstrate superior performance compared to existing methods, closely aligning with human perceptual assessment. Our approach exhibits exceptional generalization capabilities on both authentic and synthetic distortion databases. Moreover, it demonstrates a distinctive focus on perceptual foreground information, enhancing its practical applicability. Thorough quantitative analyses underscore the algorithm’s superior performance, establishing its dominance over existing methods.

Enhanced Edge Detection through Binary Particle Swarm Optimization and L0 Guided Filtering

Detecting edges holds significant importance in image processing, serving as a fundamental step in numerous computer vision applications. This paper presents an innovative method for performing edge detection by combining Binary Particle Swarm Optimization (BPSO) with L0 Guided Filtering. The proposed method aims to address the challenge of accurately detecting edges in noisy and complex images by leveraging the benefits of both BPSO and L0 guided filtering. The process begins with the initialization of the BPSO algorithm, where binary particles traverse the solution space to optimize parameters critical for edge detection. These optimized parameters are subsequently employed in the L0 guided filtering framework, a sophisticated edge preserving filter known for its ability to maintain fine details while effectively reducing noise. The synergy of BPSO and L0 guided filtering demonstrates improved adaptability to diverse image characteristics, enhancing the overall robustness of edge detection. The binary nature of BPSO allows for efficient exploration of the solution space, facilitating faster convergence to optimal parameters. Concurrently, the L0 guided filtering ensures edge preserving smoothing, contributing to the suppression of unwanted artifacts. Experimental evaluations on benchmark datasets showcase the effectiveness of the proposed method compared to traditional edge detection techniques. The results indicate superior edge localization and reduced sensitivity to noise, highlighting the potential of the BPSO Based Edge Detection under L0 Guided Filtering in real world applications. The presented approach offers a valuable contribution to the advancement of edge detection methodologies, demonstrating its potential for enhancing the performance of computer vision systems in various domains.

A novel cross fusion model with fine-grained detail reconstruction for remote sensing image pan-sharpening

ABSTRACT Pan-sharpening aims to obtain high resolution multispectral (HRMS) images by integrating the information in the panchromatic and multispectral images. Existing pan-sharpening methods have demonstrated impressive sharpening performance. However, these methods inherently overlook the complementary characteristics and interaction between diverse source images, resulting in sharpened outcomes accompanied by distortion. To solve the above problems, we construct a novel cross fusion model with fine-grained detail reconstruction from the perspective of frequency-domain. The motivation of the model is twofold: (1) to reconstruct spatial detail representations from diverse source images, laying the foundation for the generation of fine details in the subsequent fused images; and (2) to enhance the interaction between diverse source features during the fusion process in order to attain high-fidelity fusion outcomes. Based on the theoretical model, we develop a frequency-spectral dual domain cross fusion network (CF2N) utilizing the deep learning technique. Consequently, the CF2N consist of two main stages, namely frequency-domain dominated detail reconstruction (FD2R) and frequency-spectral cross fusion (FSCF). Specifically, a more reasonable reconstruction of fine frequency details in HRMS can be achieved by performing adaptive weighted fusion of frequency details in the FD2R stage. Furthermore, the FSCF module, which seamlessly integrates frequency- and spectral-domain details in a highly interactive cross fusion manner. As a result, the CF2N possesses the capability to attain high frequency-spectral fidelity results with excellent interpretability. Extensive experiments show the superior performance of ours over state of the art, while maintaining high efficiency. All implementations of this work will be published at our website.

3D modelling-based left atrium geometry analysis to predict atrial fibrillation in patients with cryptogenic stroke

Abstract Introduction Atrial fibrillation (AF) is a high prevalence condition associated with high morbidity and mortality. Its early diagnosis allows to initiate several prophylactic treatments that have proved to improve its prognosis. 3D modeling techniques allow to design accurate representations of cardiac cavities from diverse source images. Purpose We evaluated the association between new geometrical measurements obtained by 3D echocardiographic models of the left atrium (LA) and the presence of underlying paroxysmal AF in patients with cryptogenic stroke (CS). Methods 3D LA volumes from a sample of consecutive patients aged ≥60 years old and admitted to our Stroke Unit with the diagnosis of CS from April 2019 to November 2020 were analyzed. 15 days ECG monitoring was performed after discharge to detect underlying paroxysmal AF. All 3D echo volumes were analyzed by a 3D modeling open-source application. From this analysis several LA geometry parameters were obtained: sphericity, elongation, flatness, major axis length, minor axis length, maximum 2D diameter (max2Ddiam) and surface-volume ratio (S/V). Functional atrial parameters (3D left atrial ejection fraction, LAEF) were also determined. Results 36 patients were analyzed. Mean age was 78 ± 7 years old. 19% were diabetic, 58% had high blood pressure and 14% of patients had history of ischemic cardiomyopathy. Mean body-mass index was 26. These baseline characteristics were equally distributed between groups. Baseline proBNP was significantly higher in patients developing atrial fibrillation (949 vs 275 pg/mL; p= 0,014.). Maximum 2D diameter (max2Ddiam) and Surface-volume ratio (S/V) were the best predictors of AF (AUC of ROC curves: 0.69; 0.49 – 0.88 and 0.63; 0.42 – 0.84, respectively). An optimal cut-off point of 39 mm was selected for max2Ddiam and of 0.20 for S/V. These values were included in a multivariate model adjusted by baseline proBNP. A Max2Ddiam <39 mm and a S/V ≥0.20 were both significantly associated with the presence of paroxysmal AF (OR 6.96; p= 0.048 and OR 12.22; p = 0.043, respectively). Conclusions New geometrical atrial parameters can be obtained from echo 3D models and might be able to predict AF in patients with CS. Further analysis and larger samples are required to better understand these findings.TableFigure

Research on the Character Traits of Female Protagonists in Games: An Example from the Otome Games

Abstract: With the development of the game industry, the role of gender in game design has increasingly come to the fore, with particular attention being paid to the way female players are represented in games. In the domain of gaming, the otome genre, which is characterized by its distinct narrative and characterization, has garnered a significant female player base. However, for a considerable period of time, the portrayal of female characters in games has frequently been influenced by traditional gender concepts, thus resulting in the emergence of certain stereotypes and inequalities. As such, the examination of the character traits of female protagonists in otome games not only facilitates a more profound comprehension of the influence of game culture on gender perceptions, but provides insights and inspiration for the creation of games with more egalitarian and diverse gender images. In this paper, the characteristics of female protagonists in otome games developed by some Japanese game developers are analyzed, and their impact on gender equality is explored. By comparing the characteristics of the female characters across different types of games, the paper elucidates that, despite their frequent portrayal as independent, strong, and assertive, these characters are still shaped by traditional gender norms to varying degrees. Accordingly, the analysis presents recommendations for advancing more egalitarian and diverse gender images within the gaming industry.

Recurrence prediction of gastric cancer based on multi-resolution feature fusion and context information

Pathological images of gastric cancer serve as the gold standard for diagnosing this malignancy. However, the recurrence prediction task often encounters challenges such as insignificant morphological features of the lesions, insufficient fusion of multi-resolution features, and inability to leverage contextual information effectively. To address these issues, a three-stage recurrence prediction method based on pathological images of gastric cancer is proposed. In the first stage, the self-supervised learning framework SimCLR was adopted to train low-resolution patch images, aiming to diminish the interdependence among diverse tissue images and yield decoupled enhanced features. In the second stage, the obtained low-resolution enhanced features were fused with the corresponding high-resolution unenhanced features to achieve feature complementation across multiple resolutions. In the third stage, to address the position encoding difficulty caused by the large difference in the number of patch images, we performed position encoding based on multi-scale local neighborhoods and employed self-attention mechanism to obtain features with contextual information. The resulting contextual features were further combined with the local features extracted by the convolutional neural network. The evaluation results on clinically collected data showed that, compared with the best performance of traditional methods, the proposed network provided the best accuracy and area under curve (AUC), which were improved by 7.63% and 4.51%, respectively. These results have effectively validated the usefulness of this method in predicting gastric cancer recurrence.

DMFGAN: a multifeature data augmentation method for grape leaf disease identification.

The use of deep learning techniques to identify grape leaf diseases relies on large, high-quality datasets. However, a large number of images occupy more computing resources and are prone to pattern collapse during training. In this paper, a depth-separable multifeature generative adversarial network (DMFGAN) was proposed to enhance grape leaf disease data. First, a multifeature extraction block (MFEB) based on the four-channel feature fusion strategy is designed to improve the quality of the generated image and avoid the problem of poor feature learning ability of the adversarial generation network caused by the single-channel feature extraction method. Second, a depth-based D-discriminator is designed to improve the discriminator capability and reduce the number of model parameters. Third, SeLU activation function was substituted for DCGAN activation function to overcome the problem that DCGAN activation function was not enough to fit grape leaf disease image data. Finally, an MFLoss function with a gradient penalty term is proposed to reduce the mode collapse during the training of generative adversarial networks. By comparing the visual indicators and evaluation indicators of the images generated by different models, and using the recognition network to verify the enhanced grape disease data, the results show that the method is effective in enhancing grape leaf disease data. Under the same experimental conditions, DMFGAN generates higher quality and more diverse images with fewer parameters than other generative adversarial networks. The mode breakdown times of generative adversarial networks in training process are reduced, which is more effective in practical application.

Enhanced semantic-positional feature fusion network via diverse pre-trained encoders for remote sensing image water-body segmentation

ABSTRACT In the era of increasingly advanced Earth Observation (EO) technologies, extracting pertinent information (such as water-bodies) from the Earth’s surface has become a crucial task. Deep Learning, especially via pre-trained models, currently offers a highly promising approach for the semantic segmentation of Remote Sensing Imagery (RSI). However, effectively adapting these pre-trained models to RSI tasks remains challenging. Typically, these models undergo fine-tuning for specialized tasks, involving modifications to their parameters or structure of the original architecture, which may impact their inherent generalization capabilities. Furthermore, robust pre-trained models on nature images are not specifically designed for RSI, presenting challenges in their direct application to RSI tasks. To alleviate these problems, our study introduces a light-weight Enhanced Semantic-positional Feature Fusion Network (ESFFNet), leveraging diverse pre-trained image encoders alongside extensive EO data. The proposed method begins by leveraging pre-trained encoders, specifically Vision Transformer (ViT)-based and Convolutional Neural Network (CNN)-based models, to extract deep semantic and precise positional features respectively, without additional training. Following this, we introduce the Enhanced Semantic-positional Feature Fusion Module (ESFFM). This module adeptly merges semantic features derived from the ViT-based encoder with spatial features extracted from the CNN-based encoder. Such integration is realized via multi-scale feature fusion, local and long-distance feature integration, and dense connectivity strategies, leading to a robust feature representation. Finally, the Primary Segmentation-guided Fine Extraction Module (PSFEM) further bolsters the precision of remote sensing image segmentation. Collectively, these two modules constitute our light-weight decoder, with a parameter size of less than 4 M. Our approach is evaluated on two distinct water-body datasets, indicating superiority over other leading segmentation techniques. In addition, our method also demonstrates exemplary efficacy in diverse remote sensing segmentation tasks, such as building extraction and land cover classification. The source codes will be available at https://github.com/zhilyzhang/ESFFNet.

Privacy enhancing and generalizable deep learning with synthetic data for mediastinal neoplasm diagnosis

The success of deep learning (DL) relies heavily on training data from which DL models encapsulate information. Consequently, the development and deployment of DL models expose data to potential privacy breaches, which are particularly critical in data-sensitive contexts like medicine. We propose a new technique named DiffGuard that generates realistic and diverse synthetic medical images with annotations, even indistinguishable for experts, to replace real data for DL model training, which cuts off their direct connection and enhances privacy safety. We demonstrate that DiffGuard enhances privacy safety with much less data leakage and better resistance against privacy attacks on data and model. It also improves the accuracy and generalizability of DL models for segmentation and classification of mediastinal neoplasms in multi-center evaluation. We expect that our solution would enlighten the road to privacy-preserving DL for precision medicine, promote data and model sharing, and inspire more innovation on artificial-intelligence-generated-content technologies for medicine.

Multi-domain improves classification in out-of-distribution and data-limited scenarios for medical image analysis

Current machine learning methods for medical image analysis primarily focus on developing models tailored for their specific tasks, utilizing data within their target domain. These specialized models tend to be data-hungry and often exhibit limitations in generalizing to out-of-distribution samples. In this work, we show that employing models that incorporate multiple domains instead of specialized ones significantly alleviates the limitations observed in specialized models. We refer to this approach as multi-domain model and compare its performance to that of specialized models. For this, we introduce the incorporation of diverse medical image domains, including different imaging modalities like X-ray, MRI, CT, and ultrasound images, as well as various viewpoints such as axial, coronal, and sagittal views. Our findings underscore the superior generalization capabilities of multi-domain models, particularly in scenarios characterized by limited data availability and out-of-distribution, frequently encountered in healthcare applications. The integration of diverse data allows multi-domain models to utilize information across domains, enhancing the overall outcomes substantially. To illustrate, for organ recognition, multi-domain model can enhance accuracy by up to 8% compared to conventional specialized models.

Nutritional composition analysis in food images: an innovative Swin Transformer approach.

Accurate recognition of nutritional components in food is crucial for dietary management and health monitoring. Current methods often rely on traditional chemical analysis techniques, which are time-consuming, require destructive sampling, and are not suitable for large-scale or real-time applications. Therefore, there is a pressing need for efficient, non-destructive, and accurate methods to identify and quantify nutrients in food. In this study, we propose a novel deep learning model that integrates EfficientNet, Swin Transformer, and Feature Pyramid Network (FPN) to enhance the accuracy and efficiency of food nutrient recognition. Our model combines the strengths of EfficientNet for feature extraction, Swin Transformer for capturing long-range dependencies, and FPN for multi-scale feature fusion. Experimental results demonstrate that our model significantly outperforms existing methods. On the Nutrition5k dataset, it achieves a Top-1 accuracy of 79.50% and a Mean Absolute Percentage Error (MAPE) for calorie prediction of 14.72%. On the ChinaMartFood109 dataset, the model achieves a Top-1 accuracy of 80.25% and a calorie MAPE of 15.21%. These results highlight the model's robustness and adaptability across diverse food images, providing a reliable and efficient tool for rapid, non-destructive nutrient detection. This advancement supports better dietary management and enhances the understanding of food nutrition, potentially leading to more effective health monitoring applications.

Vision-and-language navigation based on history-aware cross-modal feature fusion in indoor environment

Vision-and-language navigation (VLN) is a challenging task that requires an agent to navigate an indoor environment using natural language instructions. Traditional VLN employs cross-modal feature fusion, where visual and textual information are combined to guide the agent’s navigation. However, incomplete use of perceptual information, scarcity of domain-specific training data, and diverse image and language inputs result in suboptimal performance. Herein, we propose a cross-modal feature fusion VLN history-aware information, that leverages an agent’s past experiences to make more informed navigation decisions. The regretful model and self-monitoring models are added, and the advantage actor critic(A2C) reinforcement learning algorithm is employed to improve the navigation success rate, reduce action redundancy, and shorten navigation paths. Subsequently, a data augmentation method based on speaker data is introduced to improve the model generalizability. We evaluate the proposed algorithm on the room-to-room (R2R) and room-for-room (R4R) benchmarks, and the experimental results demonstrate that, by comparison, the proposed algorithm outperforms state-of-the-art methods.

PFB-Diff: Progressive Feature Blending diffusion for text-driven image editing

Diffusion models have demonstrated their ability to generate diverse and high-quality images, sparking considerable interest in their potential for real image editing applications. However, existing diffusion-based approaches for local image editing often suffer from undesired artifacts due to the latent-level blending of the noised target images and diffusion latent variables, which lack the necessary semantics for maintaining image consistency. To address these issues, we propose PFB-Diff, a Progressive Feature Blending method for Diffusion-based image editing. Unlike previous methods, PFB-Diff seamlessly integrates text-guided generated content into the target image through multi-level feature blending. The rich semantics encoded in deep features and the progressive blending scheme from high to low levels ensure semantic coherence and high quality in edited images. Additionally, we introduce an attention masking mechanism in the cross-attention layers to confine the impact of specific words to desired regions, further improving the performance of background editing and multi-object replacement. PFB-Diff can effectively address various editing tasks, including object/background replacement and object attribute editing. Our method demonstrates its superior performance in terms of editing accuracy and image quality without the need for fine-tuning or training. Our implementation is available at https://github.com/CMACH508/PFB-Diff.

Improved Breast Cancer Detection using Modified ResNet50-Based on Gradient-Weighted Class Activation Mapping

Breast cancer is a prevalent and serious disease that affects many women around the world every year. Detecting breast cancer early is crucial for improving survival rates and treating the illness effectively. Researchers are exploring various methods, including neural networks and machine learning, to assist in detecting the disease. However, due to limited data availability, leveraging pre-trained models trained on diverse image datasets has become a common practice. This article introduces a novel approach to identifying breast cancer that involves the utilization of a deep learning model utilizing the ResNet50 framework, coupled with heat mapping and gradient-weighted class activation mapping (Grad-CAM). The suggested method was primarily assessed using the FDDM dataset of Subtracted Contrast Enhanced Spectral Mammography (CDD-CESM) images. The outcomes from this model were then contrasted with those of five other well-known models: VGG16, VGG19, MobileNetV2, EfficientNet-B7, and standard ResNet50. The newly proposed model yielded an accuracy of 0.8920, which was better than the other models. Additionally, Grad-CAM showed nearly flawless feature extraction in a breast cancer classification assignment. In the discussion section, the suggested method was utilized with the MIAS dataset to ensure thoroughness and scalability, and to allow for comparison with prior research. The results demonstrated the effectiveness of the suggested approach, with an accuracy of 0.9830 achieved on the MIAS dataset, surpassing previous works. This study significantly enhances the improvement of breast cancer detection through the integration of deep learning, the ResNet50 architecture, and visualization methods including heat maps and Grad-CAM.