Multi-domain Image Research Articles

An efficient approach for detecting downsyndrome fetus images using deep learning method

Down Syndrome (DS) is the genetical disorder which can be screened by the ultrasound fetus images. This research work proposes an automated ultrasound fetus image classification system using DL algorithm. This proposed system receives the source ultrasound fetus image and the spatial domain pixels in this image have been converted into multi modal domain pixels using Gabor wavelet. Further, Local Binary Pattern (LBP) has been computed from the multi domain image and these values have been fed into the proposed DL architecture to classify the source fetus image into either normal fetus or abnormal fetus. The morphological algorithm have been used on the abnormal fetus images in order to locate the Nasal Bone (NB) and this proposed DS detection method has been tested on two independent fetus ultrasound datasets Mendeley and Fetal Medicine Foundation (FMF). The proposed NT region segmentation system attains 98.44% NBSe, 98.63% NBSp and 98.62% NBAcc, for the set of 10 fetus images from the Mendeley dataset in this work. The proposed NB region segmentation system attains 98.67% NBSe, 98.66% NBSp and 98.66% NBAcc, for the set of 10 fetus images from the FMF dataset in this work.

Efficient adaptive learning rate for convolutional neural network based on quadratic interpolation egret swarm optimization algorithm

Convolutional neural network (CNN) has recently become popular for addressing multi-domain image classification. However, most existing methods frequently suffer from poor performance, especially in performance and convergence for various datasets. Herein, we have proposed an algorithm for multi-domain image classification by introducing a novel adaptive learning rate rule to the conventional CNN. Specifically, we adopt the CNN to extract rich feature representations. Given that the hyperparameters of the learning rate have a positive effect on the prediction error, the Egret Swarm Optimization Algorithm (ESOA) is introduced to update the learning rate, which can jump out of local extrema during exploration. Therefore, combined with quadratic interpolation, the objective function can be approximated by a polynomial, thereby improving its prediction accuracy. To verify the robustness of the proposed algorithm, we conducted comprehensive experiments in five domain public datasets to fulfil the task of image classification. Meanwhile, the highest accuracy rate of 97.15 % was obtained on the test set. The performances of our method on 24 benchmark functions (CEC2017 and CEC2022) are compared with Particle Swarm Optimization (PSO), Genetic Algorithm(GA), Whale Optimization Algorithm(WOA), Catch Fish Optimization Algorithm(CFOA), GOOSE Algorithm(GO) and ESOA. In two benchmark sets, the performance metric values of our algorithm rank no. 1, especially in all unimodal functions in contrast with other baseline algorithms.

OTMorph: Unsupervised Multi-domain Abdominal Medical Image Registration Using Neural Optimal Transport.

Deformable image registration is one of the essential processes in analyzing medical images. In particular, when diagnosing abdominal diseases such as hepatic cancer and lymphoma, multi-domain images scanned from different modalities or different imaging protocols are often used. However, they are not aligned due to scanning times, patient breathing, movement, etc. Although recent learning-based approaches can provide deformations in real-time with high performance, multi-domain abdominal image registration using deep learning is still challenging since the images in different domains have different characteristics such as image contrast and intensity ranges. To address this, this paper proposes a novel unsupervised multi-domain image registration framework using neural optimal transport, dubbed OTMorph. When moving and fixed volumes are given as input, a transport module of our proposed model learns the optimal transport plan to map data distributions from the moving to the fixed volumes and estimates a domain-transported volume. Subsequently, a registration module taking the transported volume can effectively estimate the deformation field, leading to deformation performance improvement. Experimental results on multi-domain image registration using multi-modality and multi-parametric abdominal medical images demonstrate that the proposed method provides superior deformable registration via the domain-transported image that alleviates the domain gap between the input images. Also, we attain the improvement even on out-of-distribution data, which indicates the superior generalizability of our model for the registration of various medical images. Our source code is available at https://github.com/boahK/OTMorph.

A style-aware network based on multi-task learning for multi-domain image normalization

A style-aware network based on multi-task learning for multi-domain image normalization

Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning

Federated learning (FL) enables multiple clients to collaboratively train a global model without disclosing their data. Previous researches often require training the complete model parameters. However, the emergence of powerful pre-trained models makes it possible to achieve higher performance with fewer learnable parameters in FL. In this paper, we propose a federated adaptive prompt tuning algorithm, FedAPT, for multi-domain collaborative image classification with powerful foundation models, like CLIP. Compared with direct federated prompt tuning, our core idea is to adaptively unlock specific domain knowledge for each test sample in order to provide them with personalized prompts. To implement this idea, we design an adaptive prompt tuning module, which consists of a meta prompt, an adaptive network, and some keys. The server randomly generates a set of keys and assigns a unique key to each client. Then all clients cooperatively train the global adaptive network and meta prompt with the local datasets and the frozen keys. Ultimately, the global aggregation model can assign a personalized prompt to CLIP based on the domain features of each test sample. We perform extensive experiments on two multi-domain image classification datasets across two different settings -- supervised and unsupervised. The results show that FedAPT can achieve better performance with less than 10% of the number of parameters of the fully trained model, and the global model can perform well in diverse client domains simultaneously.

Acoustic vibration multi-domain images vision transformer (AVMDI-ViT) to the detection of moldy apple core: Using a novel device based on micro-LDV and resonance speaker

Moldy-core is a common internal disease in apples, and apples infected with this disease cannot be directly identified according to their external characteristics. In this study, a novel acoustic vibration device based on micro-LDV, resonance speaker and microphone was employed to detect moldy-core in apples, the acoustic vibration signals of healthy apples and apples with different degrees of moldy-core were converted into acoustic vibration multi-domain images (AVMDI), which consisted of time-frequency images generated through continuous wavelet transform (CWT), as well as time-domain and frequency-domain images generated through Gramian Angular Field (GAF). Subsequently the combination of AVMDI and Vision Transformer (ViT) was applied to the identification of internal defects in fruits. The outcomes evince that the classification efficacy of the model, amalgamating sound and vibration signals, surpasses that of models reliant on solitary sound or vibration signals. The AVMDI-ViT model achieved an overall classification accuracy of 97.96 %. Specifically, it achieved 100 % accuracy in identifying healthy apples, 94.74 % accuracy in identifying mild moldy-core apples (≤ 7 %), 97.50 % accuracy in identifying moderate moldy-core apples (> 7 % and ≤ 15 %), and 100 % accuracy in identifying severe moldy-core apples (> 15 %). The proposed method demonstrates a high level of accuracy in the identification of moldy-core apples, while also offering advantages in terms of simplicity, speed, cost-effectiveness and non-contact detection.

QGFormer: Queries-guided transformer for flexible medical image synthesis with domain missing

Domain missing poses a common challenge in medical clinical practice, limiting diagnostic accuracy compared to the complete multi-domain images that provide complementary information. We propose QGFormer to address this issue by flexibly imputing missing domains from any available source domain using a single model, which is challenging due to (1) the inherent limitation of CNNs to capture long-range dependencies, (2) the difficulty in modeling the inter- and intra-domain dependencies of multi-domain images, and (3) inefficiencies in fusing domain-specific features associated with missing domains. To tackle these challenges, we introduce two spatial-domanial attentions (SDAs), which establish intra-domain (spatial dimension) and inter-domain (domain dimension) dependencies independently or jointly. QGFormer, constructed based on SDAs, comprises three components: Encoder, Decoder and Fusion. The Encoder and Decoder form the backbone, modeling contextual dependencies to create a hierarchical representation of features. The QGFormer Fusion then adaptively aggregates these representations to synthesize specific missing domains from coarse to fine, guided by learnable domain queries. This process is interpretable because the attention scores in Fusion indicate how much attention the target domains pay to different inputs and regions. In addition, the scalable architecture enables QGFormer to segment tumors with domain missing by replacing domain queries with segment queries. Extensive experiments demonstrate that our approach achieves consistent improvements in multi-domain imputation, cross-domain image translation and multitask of synthesis and segmentation.

DeepmdQCT: A multitask network with domain invariant features and comprehensive attention mechanism for quantitative computer tomography diagnosis of osteoporosis

In the medical field, the application of machine learning technology in the automatic diagnosis and monitoring of osteoporosis often faces challenges related to domain adaptation in drug therapy research. The existing neural networks used for the diagnosis of osteoporosis may experience a decrease in model performance when applied to new data domains due to changes in radiation dose and equipment. To address this issue, in this study, we propose a new method for multi domain diagnostic and quantitative computed tomography (QCT) images, called DeepmdQCT. This method adopts a domain invariant feature strategy and integrates a comprehensive attention mechanism to guide the fusion of global and local features, effectively improving the diagnostic performance of multi domain CT images. We conducted experimental evaluations on a self-created OQCT dataset, and the results showed that for dose domain images, the average accuracy reached 91%, while for device domain images, the accuracy reached 90.5%. our method successfully estimated bone density values, with a fit of 0.95 to the gold standard. Our method not only achieved high accuracy in CT images in the dose and equipment fields, but also successfully estimated key bone density values, which is crucial for evaluating the effectiveness of osteoporosis drug treatment. In addition, we validated the effectiveness of our architecture in feature extraction using three publicly available datasets. We also encourage the application of the DeepmdQCT method to a wider range of medical image analysis fields to improve the performance of multi-domain images.

Generative adversarial networks for rubber stamp extraction and removal

Due to the variety of documents and people's different stamping habits, it is difficult for current models to achieve satisfactory results in the process of extracting and removing rubber stamps. Most current methods cannot extract or remove rubber stamps without destroying the background text information. To solve the above problems, we propose a neural network that uses a double-cycle structure, named stamp extraction and removal generative adversarial network (SERGAN). In detail, we improve SERGAN's cycle-consistency by simultaneously recombining the generated rubber stamp image and unstamped image into a stamped image, which enables SERGAN to have multi-domain image conversion capability. To prove the model's generalization ability, we compare it with current mainstream methods on the public watermark dataset CLWD, and the experimental results show the excellence of our model. In addition, there is no publicly available dataset for rubber stamp extraction and removal experiments. To address the scarcity of rubber stamp datasets, we construct a large-scale rubber stamp dataset named LRSD.

Multi-Domain Rapid Enhancement Networks for Underwater Images

Images captured during marine engineering operations suffer from color distortion and low contrast. Underwater image enhancement helps to alleviate these problems. Many deep learning models can infer multi-source data, where images with different perspectives exist from multiple sources. To this end, we propose a multichannel deep convolutional neural network (MDCNN) linked to a VGG that can target multi-source (multi-domain) underwater image enhancement. The designed MDCNN feeds data from different domains into separate channels and implements parameters by linking VGGs, which improves the domain adaptation of the model. In addition, to optimize performance, multi-domain image perception loss functions, multilabel soft edge loss for specific image enhancement tasks, pixel-level loss, and external monitoring loss for edge sharpness preprocessing are proposed. These loss functions are set to effectively enhance the structural and textural similarity of underwater images. A series of qualitative and quantitative experiments demonstrate that our model is superior to the state-of-the-art Shallow UWnet in terms of UIQM, and the performance evaluation conducted on different datasets increased by 0.11 on average.

Unsupervised image-to-image translation in multi-parametric MRI of bladder cancer

Detection of muscular invasive bladder cancer (MIBC) is critical for surgical selection of bladder cancer (BCa) patients. Currently, multi-parameter magnetic resonance imaging (mp-MRI) is the predominant approach for identifying MIBC. However, mp-MRI is still insufficient due to the presence of noise and artifacts. Our research aims to synthesize images from the existing sequences of mp-MRI to substitute missing or low signal-to-noise ratio sequences through image-to-image (I2I) translation. Using mp-MRI images of 255 BCa patients collected in our department, we here propose a one-to-many unsupervised I2I translation network with region-wise semantic enhancement to synthesize virtual samples. We introduce an improved adaptive instance normalization module to support the generator for synthesizing multi-domain images. In addition, a branch for region-wise semantic segmentation helps the generator to enhance the quality of image translation for a specific region. A semantically consistent loss is applied to maintaining the consistency between the synthesized and the input images via region-wise semantic segmentation. Experiments on the BraTS and BCa datasets indicate that our I2I translation approach outperforms several state of the art methods. Additionally, we perform clinical feasibility tests using the synthesis images. The clinicians reach a consensus between the Vesical Imaging Reporting and Data System (VI-RADS) scoring results from the synthesized and the real mp-MRI images. In addition, after the BCa training set has been expanded using the proposed generation model, the accuracy of the BCa muscular invasion classification is improved from 77.78% to 85.19%.

Multi-domain CT translation by a routable translation network

Objective. To unify the style of computed tomography (CT) images from multiple sources, we propose a novel multi-domain image translation network to convert CT images from different scan parameters and manufacturers by simply changing a routing vector. Approach. Unlike the existing multi-domain translation techniques, our method is based on a shared encoder and a routable decoder architecture to maximize the expressivity and conditioning power of the network. Main results. Experimental results show that the proposed CT image conversion can minimize the variation of image characteristics caused by imaging parameters, reconstruction algorithms, and hardware designs. Quantitative results and clinical evaluation from radiologists also show that our method can provide accurate translation results. Significance. Quantitative evaluation of CT images from multi-site or longitudinal studies has been a difficult problem due to the image variation depending on CT scan parameters and manufacturers. The proposed method can be utilized to address this for the quantitative analysis of multi-domain CT images.

A novel framework for image-to-image translation and image compression

Data-driven paradigms using machine learning are becoming ubiquitous in image processing and communications. In particular, image-to-image (I2I) translation is a generic and widely used approach to image processing problems, such as image synthesis, style transfer, and image restoration. At the same time, neural image compression has emerged as a data-driven alternative to traditional coding approaches in visual communications. In this paper, we study the combination of these two paradigms into a joint I2I compression and translation framework, focusing on multi-domain image synthesis. We first propose distributed I2I translation by integrating quantization and entropy coding into an I2I translation framework (i.e. I2Icodec). In practice, the image compression functionality (i.e. autoencoding) is also desirable, requiring to deploy alongside I2Icodec a regular image codec. Thus, we further propose a unified framework that allows both translation and autoencoding capabilities in a single codec. Adaptive residual blocks conditioned on the translation/compression mode provide flexible adaptation to the desired functionality. The experiments show promising results in both I2I translation and image compression using a single model.

CDFRegNet: A cross-domain fusion registration network for CT-to-CBCT image registration

Background and ObjectiveComputer tomography (CT) to cone-beam computed tomography (CBCT) image registration plays an important role in radiotherapy treatment placement, dose verification, and anatomic changes monitoring during radiotherapy. However, fast and accurate CT-to-CBCT image registration is still very challenging due to the intensity differences, the poor image quality of CBCT images, and inconsistent structure information. MethodsTo address these problems, a novel unsupervised network named cross-domain fusion registration network (CDFRegNet) is proposed. First, a novel edge-guided attention module (EGAM) is designed, aiming at capturing edge information based on the gradient prior images and guiding the network to model the spatial correspondence between two image domains. Moreover, a novel cross-domain attention module (CDAM) is proposed to improve the network's ability to guide the network to effectively map and fuse the domain-specific features. ResultsExtensive experiments on a real clinical dataset were carried out, and the experimental results verify that the proposed CDFRegNet can register CT to CBCT images effectively and obtain the best performance, while compared with other representative methods, with a mean DSC of 80.01±7.16%, a mean TRE of 2.27±0.62 mm, and a mean MHD of 1.50±0.32 mm. The ablation experiments also proved that our EGAM and CDAM can further improve the accuracy of the registration network and they can generalize well to other registration networks. ConclusionThis paper proposed a novel CT-to-CBCT registration method based on EGAM and CDAM, which has the potential to improve the accuracy of multi-domain image registration.

Domain-Invariant Similarity Activation Map Contrastive Learning for Retrieval-Based Long-Term Visual Localization

Visual localization is a crucial component in the application of mobile robot and autonomous driving. Image retrieval is an efficient and effective technique in image-based localization methods. Due to the drastic variability of environmental conditions, e.g. illumination, seasonal and weather changes, retrieval-based visual localization is severely affected and becomes a challenging problem. In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner. The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models without and with Grad-SAM loss. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. The strong generalization ability of our approach is verified on RobotCar dataset using models pre-trained on urban part of CMU-Seasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision, especially under the challenging environments with illumination variance, vegetation and night-time images. The code and pretrained models are available on https://github.com/HanjiangHu/DISAM.

The Multi-domain Union Clutter Suppression Algorithm Based on Robust Principal Component Analysis

In through-the-wall imaging, the clutter cannot be eliminated completely through traditional algorithms, and seriously affects the subsequent target detection and recognition. To solve the problem, based on robust principal component analysis theory, a joint low-rank and sparse model is established in echo and image domain respectively. The models are solved by smoothing fast alternating linearization method. Then, the target images are dealt with exponentially weighted multiply multi-domain image fusion to obtain the final image. The simulation results indicate that the algorithm has great speed and accuracy with effective improvement on imaging quality of targets.

A Domain Gap Aware Generative Adversarial Network for Multi-Domain Image Translation.

Recent image-to-image translation models have shown great success in mapping local textures between two domains. Existing approaches rely on a cycle-consistency constraint that supervises the generators to learn an inverse mapping. However, learning the inverse mapping introduces extra trainable parameters and it is unable to learn the inverse mapping for some domains. As a result, they are ineffective in the scenarios where (i) multiple visual image domains are involved; (ii) both structure and texture transformations are required; and (iii) semantic consistency is preserved. To solve these challenges, the paper proposes a unified model to translate images across multiple domains with significant domain gaps. Unlike previous models that constrain the generators with the ubiquitous cycle-consistency constraint to achieve the content similarity, the proposed model employs a perceptual self-regularization constraint. With a single unified generator, the model can maintain consistency over the global shapes as well as the local texture information across multiple domains. Extensive qualitative and quantitative evaluations demonstrate the effectiveness and superior performance over state-of-the-art models. It is more effective in representing shape deformation in challenging mappings with significant dataset variation across multiple domains.

SAR-to-Optical Image Translating Through Generate-Validate Adversarial Networks

Synthetic aperture radar (SAR) has the advantages of high resolution in all-weather and all-day. However, SAR images are hard to be understood, due to their unique imaging mechanism. The SAR to optical image translation can assist in interpreting and has become a topic of growing interest in the field of remote sensing. In this letter, a SAR to optical image translation network is proposed, called generate-validate adversarial networks (GVANs). More specifically, there are two Pix2Pix networks form the cyclic structure. The validate module is employed to increase the training process and improve the edge retention ability. In order to improve multidomain images adaptability, the embedded layer is proposed. Additionally, the dilation convolution layer is employed in the generator, which is more suitable for the characteristics of SAR images. The proposed method has experimented on the SEN1-2 dataset. The result demonstrates the superiority of the proposed method over state-of-the-art methods.

Improving the Generalization Ability of Deep Neural Networks for Cross-Domain Visual Recognition

Feature learning with deep neural networks (DNNs) has made remarkable progress in recent years. However, its data-driven nature makes the collection of labeled training data expensive or impossible when the testing domain changes. Here, we propose a method of transferable feature learning and instance-level adaptation to improve the generalization ability of DNNs so as to mitigate the domain shift challenge for cross-domain visual recognition. When less labeled information is available, our proposed method shows attractive results in the new target domain and outperforms the typical fine-tuning method. Two DNNs are chosen as the representatives working with our proposed method, to do a comprehensive study about the generalization ability on the tasks of image-to-image transfer, image-to-video transfer, multidomain image classification, and weakly supervised detection. The experimental results show that our proposed method is superior to other existing works in the literature. In addition, a large scale of cross-domain database is merged from three different domains, providing a quantitative platform to evaluate different approaches in the field of cross-domain object detection.

Unsupervised multi-domain image translation with domain representation learning

Recent years have witnessed tremendous improvements in multi-domain image-to-image translation. However, previous methods require multi-generator models or one-generator model with labeled datasets, which will increase the training cost and limit their further applications. In this paper, we propose an unsupervised network that consists of one pair of generator and discriminator as well as n encoders to achieve multi-domain image translation. Our work aims to learn the mappings of n domains simultaneously and automatically using images without any attribute labels. Besides, a representation loss is proposed to extract a proper representation vector for each domain, which would improve the performance of multi-domain image translation. Experiments have shown that our proposed method can perform better than the state-of-the-art multi-domain methods.