Source Images Research Articles

High-Resolution Imaging of Targets Using Broadband Acoustic Orbital Angular Momentum Waves and Modal-Domain Focusing Beamforming

Applying the acoustic orbital angular momentum (AOAM) wave for underwater imaging can yield richer differential target echo information, a consequence of its spiral wavefront phase and multiple mutually orthogonal modes. In broadband AOAM wave imaging, the resolution of conventional beamforming is very low. Although sub-band processing can improve resolution, it cannot handle coherent signal sources. To further enhance the resolution of broadband AOAM wave underwater imaging and address the imaging issue of coherent signals in practice, this paper proposed a modal-domain focusing beamforming method. This paper initially established the echo signal model of broadband AOAM waves based on a uniform circular array. This was followed by the derivation of the beam output signal model. Finally, a new modal-domain focusing transformation matrix was constructed. Numerical results show that the proposed method reduces the background level of the beam pattern to −86dB in simple coherent target source imaging, compared with −40dB for sub-band methods and −70dB for plane wave focusing processing. Furthermore, under different noise conditions, the proposed method achieves high-resolution imaging of complex structures and good imaging of details.

Diagnosis accuracy of machine learning for idiopathic pulmonary fibrosis: a systematic review and meta-analysis

BackgroundThe diagnosis of idiopathic pulmonary fibrosis (IPF) is complex, which requires lung biopsy, if necessary, and multidisciplinary discussions with specialists. Clinical diagnosis of the two ailments is particularly challenging due to the impact of interobserver variability. Several studies have endeavored to utilize image-based machine learning to diagnose IPF and its subtype of usual interstitial pneumonia (UIP). However, the diagnostic accuracy of this approach lacks evidence-based support.ObjectiveWe conducted a systematic review and meta-analysis to explore the diagnostic efficiency of image-based machine learning (ML) for IPF.Data sources and methodsWe comprehensively searched PubMed, Cochrane, Embase, and Web of Science databases up to August 24, 2024. During the meta-analysis, we carried out subgroup analyses by imaging source (computed radiography/computed tomography) and modeling type (deep learning/other) to evaluate its diagnostic performance for IPF.ResultsThe meta-analysis findings indicated that in the diagnosis of IPF, the C-index, sensitivity, and specificity of ML were 0.93 (95% CI 0.89–0.97), 0.79 (95% CI 0.73–0.83), and 0.84 (95% CI 0.79–0.88), respectively. The sensitivity of radiologists/clinicians in diagnosing IPF was 0.69 (95% CI 0.56–0.79), with a specificity of 0.93 (95% CI 0.74–0.98). For UIP diagnosis, the C-index of ML was 0.91 (95% CI 0.87–0.94), with a sensitivity of 0.92 (95% CI 0.80–0.97) and a specificity of 0.92 (95%CI 0.82–0.97). In contrast, the sensitivity of radiologists/clinicians in diagnosing UIP was 0.69 (95% CI 0.50–0.84), with a specificity of 0.90 (95% CI 0.82–0.94).ConclusionsImage-based machine learning techniques demonstrate robust data processing and recognition capabilities, providing strong support for accurate diagnosis of idiopathic pulmonary fibrosis and usual interstitial pneumonia. Future multicenter large-scale studies are warranted to develop more intelligent evaluation tools to further enhance clinical diagnostic efficiency.Trial registration This study protocol was registered with PROSPERO (CRD42022383162).

Stable-Hair: Real-World Hair Transfer via Diffusion Model

Current hair transfer methods struggle to handle diverse and intricate hairstyles, limiting their applicability in real-world scenarios. In this paper, we propose a novel diffusion-based hair transfer framework, named Stable-Hair, which robustly transfers a wide range of real-world hairstyles to user-provided faces for virtual hair try-on. To achieve this goal, our Stable-Hair framework is designed as a two-stage pipeline. In the first stage, we train a Bald Converter alongside stable diffusion to remove hair from the user-provided face images, resulting in bald images. In the second stage, we specifically designed a Hair Extractor and a Latent IdentityNet to transfer the target hairstyle with highly detailed and high-fidelity to the bald image. The Hair Extractor is trained to encode reference images with the desired hairstyles, while the Latent IdentityNet ensures consistency in identity and background. To minimize color deviations between source images and transfer results, we introduce a novel Latent ControlNet architecture, which functions as both the Bald Converter and Latent IdentityNet. After training on our curated triplet dataset, our method accurately transfers highly detailed and high-fidelity hairstyles to the source images. Extensive experiments demonstrate that our approach achieves state-of-the-art performance compared to existing hair transfer methods.

CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation

Despite the rapid development of Chinese vision-language models (VLMs), most existing Chinese vision-language (VL) datasets are constructed on Western-centric images from existing English VL datasets. The cultural bias in the images makes these datasets unsuitable for evaluating VLMs in Chinese culture. To remedy this issue, we present a new Chinese Vision-Language Understanding Evaluation (CVLUE) benchmark dataset, where the selection of object categories and images is entirely driven by Chinese native speakers, ensuring that the source images are representative of Chinese culture. The benchmark contains four distinct VL tasks ranging from image-text retrieval to visual question answering, visual grounding and visual dialogue. We present a detailed statistical analysis of CVLUE and provide a baseline performance analysis with several open-source multilingual VLMs on CVLUE and its English counterparts to reveal their performance gap between English and Chinese. Our in-depth category-level analysis reveals a lack of Chinese cultural knowledge in existing VLMs. We also find that fine-tuning on Chinese culture-related VL datasets effectively enhances VLMs' understanding of Chinese culture.

Can Generative Models Improve Self-Supervised Representation Learning?

The rapid advancement in self-supervised representation learning has highlighted its potential to leverage unlabeled data for learning rich visual representations. However, the existing techniques, particularly those employing different augmentations of the same image, often rely on a limited set of simple transformations that cannot fully capture variations in the real world. This constrains the diversity and quality of samples, which leads to sub-optimal representations. In this paper, we introduce a framework that enriches the self-supervised learning (SSL) paradigm by utilizing generative models to produce semantically consistent image augmentations. By directly conditioning generative models on a source image, our method enables the generation of diverse augmentations while maintaining the semantics of the source image, thus offering a richer set of data for SSL. Our extensive experimental results on various joint-embedding SSL techniques demonstrate that our framework significantly enhances the quality of learned visual representations by up to 10% Top-1 accuracy in downstream tasks. This research demonstrates that incorporating generative models into the joint-embedding SSL workflow opens new avenues for exploring the potential of synthetic data. This development paves the way for more robust and versatile representation learning techniques.

Multi-Focus Image Fusion via Explicit Defocus Blur Modelling

Multi-focus image fusion (MFIF) enhances depth of field in photography by generating an all-in-focus image from multiple images captured at different focal lengths. While deep learning has shown promise in MFIF, most existing methods overlooked the physical properties of defocus blurring in their network design, limiting their interoperability and generalization. This paper introduces a novel framework that integrates explicit defocus blur modelling into the MFIF process, improving both interpretability and performance. Using an atom-based spatially-varying parameterized defocus blurring model, our approach calculates pixel-wise defocus descriptors and initial focused images from multi-focus source images in a scale-recurrent manner to estimate soft decision maps. Fusion is then performed using masks derived from these decision maps, with special treatment for pixels likely defocused in all source images or near boundaries of defocused/focused regions. The model is trained with a fusion loss and a cross-scale defocus estimation loss. Extensive experiments on benchmark datasets demonstrated the effectiveness of our approach.

StyO: Stylize Your Face in Only One-Shot

This paper focuses on face stylization with a single artistic target. Existing works for this task often fail to retain the source content while achieving geometry variation. Here, we present a novel StyO model, i.e., Stylize the face in only One-shot, to solve the above problem. In particular, StyO exploits a disentanglement and recombination strategy. It first disentangles the content and style of source and target images into identifiers, which are then recombined in a cross manner to derive the stylized face image. In this way, StyO decomposes complex images into independent and specific attributes, and simplifies one-shot face stylization as the combination of different attributes from input images, thus producing results better matching face geometry of target image and content of source one. StyO is implemented with latent diffusion models (LDM) and composed of two key modules: 1) Identifier Disentanglement Learner (IDL) for disentanglement phase. It represents identifiers as contrastive text prompts, i.e. positive and negative descriptions. And it introduces a novel triple reconstruction loss to fine-tune the pre-trained LDM for encoding style and content into corresponding identifiers; 2) Fine-graind Content Controller (FCC) for recombination phase. It recombines disentangled identifiers from IDL to form an augmented text prompt for generating stylized faces. In addition, FCC also constrains the cross-attention maps of latent and text features to preserve source face details in results. The extensive evaluation shows that StyO produces high-quality images on numerous paintings of various styles and outperforms the current state-of-the-art.

Image-to-video Adaptation with Outlier Modeling and Robust Self-learning

The image-to-video adaptation task seeks to effectively harness both labeled images and unlabeled videos for achieving effective video recognition. The modality gap of the image and video modalities and the domain discrepancy across the two domains are the two essential challenges in this task. Existing methods reduce the domain discrepancy via close-set domain adaptation techniques, resulting in inaccurate domain alignment as there exist outlier target frames. To tackle this issue, we extend the vanilla classifier with outlier classes, where each outlier class responsible for capturing outlier frames for a specific class via batch nuclear norm maximization loss. We further propose a new loss by treating the source images apart from class c as instances from outlier class specific for c. As for the modality gap, existing methods usually utilize the pseudo labels obtained from an image-level adapted model to learn a video-level model. Rare efforts are dedicated to handling the noise in pseudo labels. We proposed a new metric based on label propagation consistency to select samples for training a better video-level model. Experiments on 3 benchmarks validating the effectiveness of our method.

FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving

The garment structure serves as a crucial medium for expressing the designer's creative vision and showcasing the distinctive character of clothing items. Effective editing of garment structure in fashion images allows for an advanced preview of the design, accelerating the process of garment customization to meet individualized requirements. Although large-scale diffusion models have demonstrated impressive image generation and editing capabilities, no efforts have been made to exploit their potential in part-level editing of images. Unlike previous research, we define a clothing structure editing (CSE) task aimed at accurately editing the local structure of human-centered clothing images through simple instruction-based prompts while maintaining the consistency of clothing appearance. Specifically, this paper develops a new controllable triple-flow framework for structure editing named FashionTailor. An additional network called ClothingNet is proposed to extract the clothing details to address the rigid constraints of the original garment structure. Then, we propose a semantic-refined module to extract the semantic understanding of the source image and adaptively focus on the part to be edited. We also design a cross-blend attention mechanism to integrate fine-grained clothing features to guarantee precise alignment between appearance and target structure features. In addition, a garment structure dataset called StructureFashion has been collated, wherein each item of clothing is represented by multiple photos with diverse structure characteristics, containing over six million pairs. Finally, our method supports editing the structure of multiple parts on a garment simultaneously. Extensive experiments validate the effectiveness of our method for editing part-level human images in StructureFashion dataset and real-scenarios.

Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression

In this paper, we study how to synthesize a dynamic reference from an external dictionary to perform conditional coding of the input image in the latent domain and how to learn the conditional latent synthesis and coding modules in an end-to-end manner. Our approach begins by constructing a universal image feature dictionary using a multi-stage approach involving modified spatial pyramid pooling, dimension reduction, and multi-scale feature clustering. For each input image, we learn to synthesize a conditioning latent by selecting and synthesizing relevant features from the dictionary, which significantly enhances the model's capability in capturing and exploring image source correlation. This conditional latent synthesis involves a correlation-based feature matching and alignment strategy, comprising a Conditional Latent Matching (CLM) module and a Conditional Latent Synthesis (CLS) module. The synthesized latent is then used to guide the encoding process, allowing for more efficient compression by exploiting the correlation between the input image and the reference dictionary. According to our theoretical analysis, the proposed conditional latent coding (CLC) method is robust to perturbations in the external dictionary samples and the selected conditioning latent, with an error bound that scales logarithmically with the dictionary size, ensuring stability even with large and diverse dictionaries. Experimental results on benchmark datasets show that our new method improves the coding performance by a large margin (up to 1.2 dB) with a very small overhead of approximately 0.5% bits per pixel.

Multi-view collaborative ensemble classification for EEG signals based on 3D second-order difference plot and CSP.

EEG signal analysis methods based on electrical source imaging (ESI) technique have significantly improved classification accuracy and response time. However, for the refined and informative source signals, the current studies have not fully considered their dynamic variability in feature extraction and lacked an effective integration of their dynamic variability and spatial characteristics. Additionally, the adaptability and complementarity of classifiers have not been considered comprehensively. These two aspects lead to the issue of insufficient decoding of source signals, which still limits the application of brain-computer interface (BCI). To address these challenges, this paper proposes a multi-view collaborative ensemble classification method for EEG signals based on three-dimensional second-order difference plot (3D SODP) and common spatial pattern (CSP). First, EEG signals are mapped to the source domain using the ESI technique, and then the source signals in the region of interest (ROI) are obtained. Next, features from three viewpoints of the source signals are extracted, including 3D SODP features, spatial features, and the weighted fusion of both. Finally, the extracted multi-view features are integrated with subject-specific sub-classifier combination, and a voting mechanism is used to determine the final classification. The results show that the proposed method achieves classification accuracy of 81.3% and 82.6% respectively in two sessions of the OpenBMI dataset, which is nearly 5% higher than the state-of-the-art method, and maintains the analysis response time required for online BCI. This paper employs multi-view feature extraction to fully capture the characteristics of the source signals and enhances feature utilization through collaborative ensemble classification. The results demonstrate high accuracy and robust performance, providing a novel approach for online BCI.

In vivo magnetic resonance imaging of the interstitial pressure gradients (pgMRI) using a pulsatile poroelastic computational model.

Fluid movement in the interstitial space of the brain affects the clearance of waste products, which is an important factor in the pathophysiology of dementia. Estimating interstitial fluid (ISF) flow is critical to understanding these processes; yet, it has proven difficult to measure non-invasively. The pulsatile component of ISF flow may be particularly important for clearance, e.g. by facilitating fluid mixing. Directly measuring ISF flows is challenging due to the slow velocities and small volume fractions involved; however, pulsatile flows present a unique opportunity as their driving forces can be estimated from observations of pulsatile tissue motion. In this work, we present pressure gradient magnetic resonance imaging (pgMRI), which assimilates retrospectively gated pulsatile tissue deformations measured with a displacement encoding with stimulated echoes MRI sequence into a patient-specific poroelastic computational model by estimating the distribution of fluid sources. The new method is demonstrated to recover a spherical fluid source accurately from synthetic data with simulated noise of up to 20%, and to produce not previously reported in vivo brain fluid source images along with companion images of the three-dimensional stresses and pressure gradients which drive ISF movement. Repeated exams of four healthy volunteers demonstrated variability below 10% for pgMRI parameters in most cases.

Radiomics in the differential diagnosis of glioblastoma under the primary neurooncoimaging conditions

Introduction. According to the 2021 WHO Classification of Tumors of the Central Nervous System (CNS) and the 2023 Clinical Practice Guidelines on the Drug Management of Primary CNS Cancers, the first step of molecular genetic testing to identify the morphological type and malignancy of adult-type diffuse gliomas is the detection of isocitrate dehydrogenase (IDH) mutation status. However, tumor tissue biopsy as the conventional diagnostic standard has a number of limitations that can potentially be mitigated by applying the principles of radiomics to the interpretation of magnetic resonance (MR) images. The aim of our study is to develop a radiomics model for IDH mutation status prediction, which can be applied to primary diagnostic imaging in patients with suspected adult-type diffuse gliomas. Materials and methods. We conducted a retrospective comparative statistical analysis of radiomic features extracted from 46 conventional brain MR images of the patients with adult-type diffuse gliomas and identified IDH mutation status using the Random Forest algorithm of machine learning in combination with various preprocessing methods of the source imaging data and a semi-automated LevelTracing tool used for segmentation of the regions of interest (ROI). Results. The most effective combination of tools for preprocessing, segmentation, and classification was found to be ScaleIntensity, LevelTracing, and Random Forest, respectively. Using this combination, we verified the reliability of six radiomic predictors identified at the previous study stage. These features were all associated with IDH mutation status, and most of them capture texture heterogeneity in the ROIs at the voxel level. We were also able to improve the prognostic performance of our classification model up to AUC = 0.845 ± 0.089 (p 0.05). Conclusion. Based on a small, technically heterogeneous sample of routine MR imaging data, we developed a multiparametric model of IDH mutation status prediction in the patients with adult-type diffuse gliomas. Our conclusion is that relatively uniform preprocessing techniques based on uniform voxel intensity changes, which allow to preserve the structural detail, are feasible in clinical practice. The identified radiomic, likely voxel-based, features reflect the severity of perifocal vasogenic edema and the measure of intratumor morphological heterogeneity. We plan to assess the reproducibility of the study results using similar medical imaging data from open sources and to develop a color mapping technique for the ROIs to facilitate visual interpretation of quantitative radiomic data.

Multi-focus image fusion based on pulse coupled neural network and WSEML in DTCWT domain

The goal of multi-focus image fusion is to merge near-focus and far-focus images of the same scene to obtain an all-focus image that accurately and comprehensively represents the focus information of the entire scene. The current multi-focus fusion algorithms lead to issues such as the loss of details and edges, as well as local blurring in the resulting images. To solve these problems, a novel multi-focus image fusion method based on pulse coupled neural network (PCNN) and weighted sum of eight-neighborhood-based modified Laplacian (WSEML) in dual-tree complex wavelet transform (DTCWT) domain is proposed in this paper. The source images are decomposed by DTCWT into low- and high-frequency components, respectively; then the average gradient (AG) motivate PCNN-based fusion rule is used to process the low-frequency components, and the WSEML-based fusion rule is used to process the high-frequency components; we conducted simulation experiments on the public Lytro dataset, demonstrating the superiority of the algorithm we proposed.

A deep learning framework leveraging spatiotemporal feature fusion for electrophysiological source imaging.

A deep learning framework leveraging spatiotemporal feature fusion for electrophysiological source imaging.

Two-scale decomposition and deep learning fusion for visible and infrared images

The paper focuses on the fusion of visible and infrared images to generate composite images that preserve both the thermal radiation information from the infrared spectrum and the detailed texture from the visible spectrum. The proposed approach combines traditional methods, such as two-scale decomposition, with deep learning techniques, specifically employing an autoencoder architecture. The source images are subjected to two-scale decomposition, which extracts high-frequency detail and low-frequency base information. Additionally, an algorithmic unravelling technique establishes a logical connection between deep neural networks and traditional signal processing algorithms. The model consists of two encoders for decomposition and a decoder after the unravelling operation. During testing, a fusion layer merges the decomposed feature maps, and the decoder generates the fused image. Evaluation metrics including entropy, average gradient, spatial frequency and standard deviation are employed to subjectively assess fusion quality. The proposed approach demonstrates promise for effectively combining visible and infrared imagery for various applications.

PhotoChem Interplays: Lighting the Way for Drug Delivery and Diagnosis.

PhotoChem Interplays: Lighting the Way for Drug Delivery and Diagnosis.

Magnetic source imaging registration based on self-supervised learning and multi-view differentiable rendering

Magnetic source imaging registration based on self-supervised learning and multi-view differentiable rendering

Deep learning-based EEG source imaging is robust under varying electrode configurations

Deep learning-based EEG source imaging is robust under varying electrode configurations

Drone Detection in Restricted Areas Using Deep Learning

As the use of drones becomes more widespread, the corresponding rise in drone-related intrusions poses a growing threat to public safety and privacy. Traditional anti-drone systems typically rely on radio-frequency sensors for drone tracking. This paper investigates the fusion of deep learning-based detection algorithms with surveillance cameras within the framework of radio-frequency anti-drone systems. The primary aim is to assess the efficacy of contemporary models and training methodologies in achieving precise and real-time drone detection. One of the central challenges addressed in this paper is the detection of small drones at extended distances, coupled with the demand for real-time performance. To overcome the scarcity of small drone datasets, the research team constructed a real-world dataset for comprehensive evaluations. Various iterations of the YOLO (You Only Look Once) models were compared using this dataset, with specific modifications implemented to enhance small object detection. Additionally, the image sources are diversified for training, incorporating bird images to mitigate false positives and enhance model robustness. Among the YOLO models tested, YOLOv5 exhibited superior precision, recall, and F1 score. The work delves into the impact of additional detection layers on precision, recall, and F1 score, revealing trade-offs between these metrics. The inclusion of bird images in the background training process demonstrated improvements in accuracy and recall, underlining the importance of diverse training data. An intriguing finding was observed when excluding extremely small drones and birds from the analysis, resulting in heightened precision but diminished recall. This highlights the delicate balance required in optimizing detection algorithms for different scenarios. The paper also acknowledges the need for further investigation into the generalizability of the proposed approach across various drone types.