Heterogeneous Images Research Articles

Early autism diagnosis based on path signature and Siamese unsupervised feature compressor.

Autism spectrum disorder has been emerging as a growing public health threat. Early diagnosis of autism spectrum disorder is crucial for timely, effective intervention and treatment. However, conventional diagnosis methods based on communications and behavioral patterns are unreliable for children younger than 2 years of age. Given evidences of neurodevelopmental abnormalities in autism spectrum disorder infants, we resort to a novel deep learning-based method to extract key features from the inherently scarce, class-imbalanced, and heterogeneous structural MR images for early autism diagnosis. Specifically, we propose a Siamese verification framework to extend the scarce data, and an unsupervised compressor to alleviate data imbalance by extracting key features. We also proposed weight constraints to cope with sample heterogeneity by giving different samples different voting weights during validation, and used Path Signature to unravel meaningful developmental features from the two-time point data longitudinally. We further extracted machine learning focused brain regions for autism diagnosis. Extensive experiments have shown that our method performed well under practical scenarios, transcending existing machine learning methods and providing anatomical insights for autism early diagnosis.

Fourier Finite Difference Migration Based Heterogeneous Photoacoustic Imaging in Irregular Layered Media

Fourier Finite Difference Migration Based Heterogeneous Photoacoustic Imaging in Irregular Layered Media

Gradient Weakly Sensitive Multi-Source Sensor Image Registration Method

Aiming at the nonlinear radiometric differences between multi-source sensor images and coherent spot noise and other factors that lead to alignment difficulties, the registration method of gradient weakly sensitive multi-source sensor images is proposed, which does not need to extract the image gradient in the whole process and has rotational invariance. In the feature point detection stage, the maximum moment map is obtained by using the phase consistency transform to replace the gradient edge map for chunked Harris feature point detection, thus increasing the number of repeated feature points in the heterogeneous image. To have rotational invariance of the subsequent descriptors, a method to determine the main phase angle is proposed. The phase angle of the region near the feature point is counted, and the parabolic interpolation method is used to estimate the more accurate main phase angle under the determined interval. In the feature description stage, the Log-Gabor convolution sequence is used to construct the index map with the maximum phase amplitude, the heterogeneous image is converted to an isomorphic image, and the isomorphic image of the region around the feature point is rotated by using the main phase angle, which is in turn used to construct the feature vector with the feature point as the center by the quadratic interpolation method. In the feature matching stage, feature matching is performed by using the sum of squares of Euclidean distances as a similarity metric. Finally, after qualitative and quantitative experiments of six groups of five pairs of different multi-source sensor image alignment correct matching rates, root mean square errors, and the number of correctly matched points statistics, this algorithm is verified to have the advantage of robust accuracy compared with the current algorithms.

A domain adaptation detector for heterogenous SAR image object detection

A domain adaptation detector for heterogenous SAR image object detection

Multi-Energy Guided Image Translation with Stochastic Differential Equations for Near-Infrared Facial Expression Recognition

Illumination variation has been a long-term challenge in real-world facial expression recognition (FER). Under uncontrolled or non-visible light conditions, near-infrared (NIR) can provide a simple and alternative solution to obtain high-quality images and supplement the geometric and texture details that are missing in the visible (VIS) domain. Due to the lack of large-scale NIR facial expression datasets, directly extending VIS FER methods to the NIR spectrum may be ineffective. Additionally, previous heterogeneous image synthesis methods are restricted by low controllability without prior task knowledge. To tackle these issues, we present the first approach, called for NIR-FER Stochastic Differential Equations (NFER-SDE), that transforms face expression appearance between heterogeneous modalities to the overfitting problem on small-scale NIR data. NFER-SDE can take the whole VIS source image as input and, together with domain-specific knowledge, guide the preservation of modality-invariant information in the high-frequency content of the image. Extensive experiments and ablation studies show that NFER-SDE significantly improves the performance of NIR FER and achieves state-of-the-art results on the only two available NIR FER datasets, Oulu-CASIA and Large-HFE.

AI-Based Measurement of Lumbar Spinal Stenosis on MRI: External Evaluation of a Fully Automated Model.

The aim of this study was to clinically validate a fully automated AI model for magnetic resonance imaging (MRI)-based quantifications of lumbar spinal canal stenosis. This retrospective study included lumbar spine MRI of 100 consecutive clinical patients (56 ± 17 years; 43 females, 57 males) performed on clinical 1.5 (51 examinations) and 3 T MRI scanners (49 examinations) with heterogeneous clinical imaging protocols. The AI model performed segmentations of the thecal sac on axial T2-weighted sequences. Based on these segmentations, the anteroposterior (AP) and mediolateral (ML) distance, and the area of the thecal sac were measured in a fully automated manner. For comparison, 2 fellowship-trained musculoskeletal radiologists performed the same segmentations and measurements independently. Statistics included 1-sample t tests, the intraclass correlation coefficient (ICC), Bland-Altman plots, and Dice coefficients. A P value of <0.05 was considered statistically significant. The average measurements of the AI model, reader 1, and reader 2 were 194 ± 72 mm 2 , 181 ± 71 mm 2 , and 179 ± 70 mm 2 for thecal sac area, 13 ± 3.3 mm, 12.6 ± 3.3 mm, and 12.6 ± 3.2 mm for AP distance, and 19.5 ± 3.9 mm, 20 ± 4.3 mm, and 19.4 ± 4 mm for ML distance, respectively. Significant differences existed for all pairwise comparisons, besides reader 1 versus AI model for the ML distance and reader 1 versus reader 2 for the AP distance ( P = 0.1 and P = 0.21, respectively). The pairwise mean absolute errors among reader 1, reader 2, and the AI model ranged from 0.59 mm and 0.75 mm for the AP distance, from 1.16 mm to 1.37 mm for the ML distance, and from 7.9 mm 2 to 15.54 mm 2 for the thecal sac area. Pairwise ICCs among reader 1, reader 2, and the AI model ranged from 0.91 and 0.94 for the AP distance and from 0.86 to 0.9 for the ML distance without significant differences. For the thecal sac area, the pairwise ICC between both readers and the AI model of 0.97 each was slightly, but significantly lower than the ICC between reader 1 and reader 2 of 0.99. Similarly, the Dice coefficient and Hausdorff distance between both readers and the AI model were significantly lower than the values between reader 1 and reader 2, overall ranging from 0.93 to 0.95 for the Dice coefficients and 1.1 to 1.44 for the Hausdorff distances. The investigated AI model is reliable for assessing the AP and the ML thecal sac diameters with human level accuracies. The small differences for measurement and segmentation of the thecal sac area between the AI model and the radiologists are likely within a clinically acceptable range.

Salt-and-pepper Noise Reduction for Medical Images Based on Image Fusion.

During the collection process, the prostate capsule is prone to introduce salt and pepper noise due to gastrointestinal peristalsis, which will affect the precision of subsequent object detection. A cascade optimization scheme for image denoising based on image fusion was proposed to improve the peak signal-to-noise ratio(PSNR) and contour protection performance of heterogeneous medical images after image denoising. Anisotropic diffusion fusion (ADF) was used to decompose the images denoised by adaptive median filter, non-local adaptive median filter and artificial neural network to generate the base layer and detail layer, which were fused by weighted average and Karhunen-Loeve Transform respectively. Finally, the image was reconstructed by linear superposition. Compared with the traditional denoising method, the image denoised by this method has a higher PSNR while maintaining the image edge contour. Using the denoised dataset for object detection, the detection precision of the obtained model is higher.

The role of susceptibility-weighted imaging & contrast-enhanced MRI in the diagnosis of primary CNS vasculitis: a large case series.

Primary CNS Vasculitis (PCNSV) is a rare, diverse, and polymorphic CNS blood vessel inflammatory condition. Due to its rarity, clinical variability, heterogeneous imaging results, and lack of definitive laboratory markers, PCNSV diagnosis is challenging. This retrospective cohort analysis identified patients with histological diagnosis of PCNSV. Demographic data, clinical presentation, neuroimaging studies, and histopathologic findings were recorded. We enrolled 56 patients with a positive biopsy of CNS vasculitis. Most patients had cerebral hemisphere or brainstem symptoms. Most brain MRI lesions were bilateral, diffuse discrete to confluent white matter lesions. Frontal lobe lesions predominated, followed by inferior cerebellar lesions. Susceptibility-weighted imaging (SWI) hemorrhages in 96.4% (54/56) of patients, either solitary microhemorrhages or a combination of micro and macrohemorrhages. Contrast-enhanced T1-WIs revealed parenchymal enhancement in 96.3% (52/54 patients). The most prevalent pattern of enhancement observed was dot-linear (87%), followed by nodular (61.1%), perivascular (25.9%), and patchy (16.7%). Venulitis was found in 19 of 20 individuals in cerebral DSA. Hemorrhages in SWI and dot-linear enhancement pattern should be incorporated as MINOR diagnostic criteria to diagnose PCNSV accurately within an appropriate clinical context. Microhemorrhages in SWI and venulitis in DSA, should be regarded as a potential marker for PCNSV.

Edge-Bound Change Detection in Multisource Remote Sensing Images

Detecting changes in multisource heterogeneous images is a great challenge for unsupervised change detection methods. Image-translation-based methods, which transform two images to be homogeneous for comparison, have become a mainstream approach. However, most of them primarily rely on information from unchanged regions, resulting in networks that cannot fully capture the connection between two heterogeneous representations. Moreover, the lack of a priori information and sufficient training data makes the training vulnerable to the interference of changed pixels. In this paper, we propose an edge-oriented generative adversarial network (EO-GAN) for change detection that indirectly translates images using edge information, which serves as a core and stable link between heterogeneous representations. The EO-GAN is composed of an edge extraction network and a reconstructive network. During the training process, we ensure that the edges extracted from heterogeneous images are as similar as possible through supplemented data based on superpixel segmentation. Experimental results on both heterogeneous and homogeneous datasets demonstrate the effectiveness of our proposed method.

Enhanced Graph Structure Representation for Unsupervised Heterogeneous Change Detection

Heterogeneous change detection (CD) is widely applied in various fields such as urban planning, environmental monitoring, and disaster management. It enhances the accuracy and comprehensiveness of surface change monitoring by integrating multi-sensor remote sensing data. Scholars have proposed many graph-based methods to address the issue of incomparable heterogeneous images caused by imaging differences. However, these methods often overlook the influence of changes in vertex status on the graph structure, which limits their ability to represent image structural features. To tackle this problem, this paper presents an unsupervised heterogeneous CD method based on enhanced graph structure representation (EGSR). This method enhances the representation capacity of the graph structure for image structural features by measuring the unchanged probabilities of vertices, thereby making it easier to detect changes in heterogeneous images. Firstly, we construct the graph structure using image superpixels and measure the structural graph differences of heterogeneous images in the same image domain. Then, we calculate the unchanged probability of each vertex in the structural graph and reconstruct the graph structure using this probability. To accurately represent the graph structure, we adopt an iterative framework for enhancing the representation of the graph structure. Finally, at the end of the iteration, the final change map (CM) is obtained by binary segmentation of the graph vertices based on their unchanged probabilities. The effectiveness of this method is validated through experiments on four sets of heterogeneous image datasets and two sets of homogeneous image datasets.

Image Regression With Structure Cycle Consistency for Heterogeneous Change Detection.

Change detection (CD) between heterogeneous images is an increasingly interesting topic in remote sensing. The different imaging mechanisms lead to the failure of homogeneous CD methods on heterogeneous images. To address this challenge, we propose a structure cycle consistency-based image regression method, which consists of two components: the exploration of structure representation and the structure-based regression. We first construct a similarity relationship-based graph to capture the structure information of image; here, a k -selection strategy and an adaptive-weighted distance metric are employed to connect each node with its truly similar neighbors. Then, we conduct the structure-based regression with this adaptively learned graph. More specifically, we transform one image to the domain of the other image via the structure cycle consistency, which yields three types of constraints: forward transformation term, cycle transformation term, and sparse regularization term. Noteworthy, it is not a traditional pixel value-based image regression, but an image structure regression, i.e., it requires the transformed image to have the same structure as the original image. Finally, change extraction can be achieved accurately by directly comparing the transformed and original images. Experiments conducted on different real datasets show the excellent performance of the proposed method. The source code of the proposed method will be made available at https://github.com/yulisun/AGSCC.

Advanced analysis of disintegrating pharmaceutical compacts using deep learning-based segmentation of time-resolved micro-tomography images

The mechanism governing pharmaceutical tablet disintegration is far from fully understood. Despite the importance of controlling a formulation's disintegration process to maximize the active pharmaceutical ingredient's bioavailability and ensure predictable and consistent release profiles, the current understanding of the process is based on indirect or superficial measurements. Formulation science could, therefore, additionally deepen the understanding of the fundamental physical principles governing disintegration based on direct observations of the process. We aim to help bridge the gap by generating a series of time-resolved X-ray micro-computed tomography (μCT) images capturing volumetric images of a broad range of mini-tablet formulations undergoing disintegration. Automated image segmentation was a prerequisite to overcoming the challenges of analyzing multiple time series of heterogeneous tomographic images at high magnification. We devised and trained a convolutional neural network (CNN) based on the U-Net architecture for autonomous, rapid, and consistent image segmentation. We created our own μCT data reconstruction pipeline and parameterized it to deliver image quality optimal for our CNN-based segmentation. Our approach enabled us to visualize the internal microstructures of the tablets during disintegration and to extract parameters of disintegration kinetics from the time-resolved data. We determine by factor analysis the influence of the different formulation components on the disintegration process in terms of both qualitative and quantitative experimental responses. We relate our findings to known formulation component properties and established experimental results. Our direct imaging approach, enabled by deep learning-based image processing, delivers new insights into the disintegration mechanism of pharmaceutical tablets.

An approach to dynamic visualization of heterogeneous geospatial vector images

One of the well-known problems of geoinformation software libraries in the visualization of geospatial data is the low efficiency and limited frame-by-frame change of a group of spatial layers with time reference. Among the most significant challenges of visualization is that the synchronous switching of a group of spatial layers is impossible to perform, which leads to less accurate estimates of the temporal anisotropy of the corresponding spatial data. The problem is further exacerbated by the heterogeneity of spatial information, which is expressed in sampling steps, formats used by geospatial primitives. The said problem significantly complicates the analysis of spatiotemporal information in many research and application areas. An illustrative example may be found in the problem of analyzing the spatiotemporal anisotropy of geophysical information. Here, for the visualization to be implemented, the dynamic evaluation of the retrospective data in a given time interval needs to performed. The paper proposes an approach that enables heterogeneous vector geospatial data to be integrated before subsequent processing, analysis and visualization. The effectiveness of the developed approach is confirmed by the example of a web application for visualizing a geospatial image as an array of spatial polylines of arbitrary data, as well as in the problem of analysis of spatiotemporal variations of geophysical data.

Using a generative adversarial network to generate synthetic MRI images for multi-class automatic segmentation of brain tumors.

Challenging tasks such as lesion segmentation, classification, and analysis for the assessment of disease progression can be automatically achieved using deep learning (DL)-based algorithms. DL techniques such as 3D convolutional neural networks are trained using heterogeneous volumetric imaging data such as MRI, CT, and PET, among others. However, DL-based methods are usually only applicable in the presence of the desired number of inputs. In the absence of one of the required inputs, the method cannot be used. By implementing a generative adversarial network (GAN), we aim to apply multi-label automatic segmentation of brain tumors to synthetic images when not all inputs are present. The implemented GAN is based on the Pix2Pix architecture and has been extended to a 3D framework named Pix2PixNIfTI. For this study, 1,251 patients of the BraTS2021 dataset comprising sequences such as T1w, T2w, T1CE, and FLAIR images equipped with respective multi-label segmentation were used. This dataset was used for training the Pix2PixNIfTI model for generating synthetic MRI images of all the image contrasts. The segmentation model, namely DeepMedic, was trained in a five-fold cross-validation manner for brain tumor segmentation and tested using the original inputs as the gold standard. The inference of trained segmentation models was later applied to synthetic images replacing missing input, in combination with other original images to identify the efficacy of generated images in achieving multi-class segmentation. For the multi-class segmentation using synthetic data or lesser inputs, the dice scores were observed to be significantly reduced but remained similar in range for the whole tumor when compared with evaluated original image segmentation (e.g. mean dice of synthetic T2w prediction NC, 0.74 ± 0.30; ED, 0.81 ± 0.15; CET, 0.84 ± 0.21; WT, 0.90 ± 0.08). A standard paired t-tests with multiple comparison correction were performed to assess the difference between all regions (p < 0.05). The study concludes that the use of Pix2PixNIfTI allows us to segment brain tumors when one input image is missing.

CellSegUNet: an improved deep segmentation model for the cell segmentation based on UNet++ and residual UNet models

Cell nucleus segmentation is an important method that is widely used in the diagnosis and treatment of many diseases, as well as counting and identifying the cell nucleus. The main challenges when using this method are heterogeneous image intensities in the image, overlapping of cell nuclei, and noise. In order to overcome these difficulties, a hybrid segmentation model with attention block, CellSegUNet, is proposed, inspired by the advantageous points of UNet++ and Residual UNet models. With the proposed attention mechanism, semantic gaps that may occur are prevented by evaluating both horizontal and vertical features together. The serial and parallel connection of the convolutional blocks in the residual modules in the CellSegUNet model prevents data loss. Thus, features with stronger representation ability were obtained. The output layer, which is, especially proposed for the CellSegUNet model, calculated the differences between the data in each layer and the data in the input layer. The output value obtained from the layer level where the lowest value comes from constitutes the output of the whole system. At the same depth level, CellSegUNet versus UNet++ and ResUNet models were compared on Data Science Bowl (DSB), Sartorius Cell Instance Segmentation (SCIS), and Blood Cell Segmentation (BCS) datasets. With the CellSegUNet model, accuracy, dice, and jaccard metrics were obtained as 0.980, 0.970, 0.959 for the DSB dataset, 0.931, 0.957, 0.829 for the SCIS dataset and 0.976, 0.971, 0.927 for the BCS dataset, respectively. As a result, it is predicted that the proposed model can provide solutions to different segmentation problems.

COMIC: An unsupervised change detection method for heterogeneous remote sensing images based on copula mixtures and Cycle-Consistent Adversarial Networks

In this paper, we consider the problem of change detection (CD) with two heterogeneous remote sensing (RS) images. For this problem, an unsupervised change detection method has been proposed recently based on the image translation technique of Cycle-Consistent Adversarial Networks (CycleGANs), where one image is translated from its original modality to the modality of the other image so that the difference map can be obtained by performing arithmetical subtraction. However, the difference map derived from subtraction is susceptible to image translation errors, in which case the changed area and the unchanged area are less distinguishable. To overcome the above shortcoming, we propose a new unsupervised copula mixture and CycleGAN-based CD method (COMIC), which combines the advantages of copula mixtures on statistical modeling and the advantages of CycleGANs on data mining. In COMIC, the pre-event image is first translated from its original modality to the post-event image modality. After that, by constructing a copula mixture, the joint distribution of the features from the heterogeneous images can be learnt according to quantitive analysis of the dependence structure based on the translated image and the original pre-event image, which are of the same modality and contain totally the same objects. Then, we model the CD problem as a binary hypothesis testing problem and derive its test statistics based on the constructed copula mixture. Finally, the difference map can be obtained from the test statistics and the binary change map (BCM) is generated by K-means clustering. We perform experiments on real RS datasets, which demonstrate the superiority of COMIC over the state-of-the-art methods.

Hereditary Thoracic Aortic Diseases.

Advances in both imaging techniques and genetics have led to the recognition of a wide variety of aortic anomalies that can be grouped under the term 'hereditary thoracic aortic diseases'. The present review aims to summarize this very heterogeneous population's clinical, genetic, and imaging characteristics and to discuss the implications of the diagnosis for clinical counselling (on sports activity or pregnancy), medical therapies and surgical management.

TDSCCNet: twin-depthwise separable convolution connect network for change detection with heterogeneous images

The task of change detection (CD) in optical and SAR images is an ever-evolving and demanding subject within the realm of remote sensing (RS). It holds great significance to identify the target areas by using complementary information between the two. Due to the distinct imaging mechanisms employed by optical and SAR sensors, effectively and accurately identifying changing regions can be challenging. To this end, a novel heterogeneous RS images CD network (Twin-Depthwise Separable Convolution Connect Network, TDSCCNet) is proposed in this paper. Image domain transformation is a front-end task, while the back-end employs a single-branch bilayer depthwise separable convolution-connected encoder-decoder to accomplish CD work. Specifically, first, the cycle-consistent adversarial network (CycleGAN) serves to integrate the optical and SAR visual domains, and a consistent feature expression is obtained. Second, the single-branch encoder structure of bilayer depthwise separable convolution is employed to realize change feature extraction. Finally, the multiscale connected decoder reconstructed by change map is utilized to reconstruct the original images and solve the local discontinuities and holes in the binary change map. Multiscale loss is designated for optimizing the global and local effects to alleviate the class imbalance problem. It was tested on four representative datasets from Gloucester, Shuguang Village, Italy and WV-3 datasets with an overall accuracy of 97.36%, 97.01%, 97.62%, and 98.01% respectively. By comparing the existing methods, experimental results confirmed the effectiveness of the proposed method.

Multilabel SegSRGAN-A framework for parcellation and morphometry of preterm brain in MRI.

Magnetic resonance imaging (MRI) is a powerful tool for observing and assessing the properties of brain tissue and structures. In particular, in the context of neonatal care, MR images can be used to analyze neurodevelopmental problems that may arise in premature newborns. However, the intrinsic properties of newborn MR images, combined with the high variability of MR acquisition in a clinical setting, result in complex and heterogeneous images. Segmentation methods dedicated to the processing of clinical data are essential for obtaining relevant biomarkers. In this context, the design of quality control protocols for the associated segmentation is a cornerstone for guaranteeing the accuracy and usefulness of these inferred biomarkers. In recent work, we have proposed a new method, SegSRGAN, designed for super-resolution reconstruction and segmentation of specific brain structures. In this article, we first propose an extension of SegSRGAN from binary segmentation to multi-label segmentation, leading then to a partitioning of an MR image into several labels, each corresponding to a specific brain tissue/area. Secondly, we propose a segmentation quality control protocol designed to assess the performance of the proposed method with regard to this specific parcellation task in neonatal MR imaging. In particular, we combine scores derived from expert analysis, morphometric measurements and topological properties of the structures studied. This segmentation quality control can enable clinicians to select reliable segmentations for clinical analysis, starting with correlations between perinatal risk factors, regional volumes and specific dimensions of cognitive development. Based on this protocol, we are investigating the strengths and weaknesses of SegSRGAN and its potential suitability for clinical research in the context of morphometric analysis of brain structure in preterm infants, and to potentially design new biomarkers of neurodevelopment. The proposed study focuses on MR images from the EPIRMEX dataset, collected as part of a national cohort study. In particular, this work represents a first step towards the design of 3-dimensional neonatal brain morphometry based on segmentation. The (free and open-source) code of multilabel SegSRGAN is publicly available at the following URL: https://doi.org/10.5281/zenodo.12659424.

Semi-Supervised Learning With Heterogeneous Distribution Consistency for Visible Infrared Person Re-Identification.

Visible infrared person re-identification (VI-ReID) exposes considerable challenges because of the modality gaps between the person images captured by daytime visible cameras and nighttime infrared cameras. Several fully-supervised VI-ReID methods have improved the performance with extensive labeled heterogeneous images. However, the identity of the person is difficult to obtain in real-world situations, especially at night. Limited known identities and large modality discrepancies impede the effectiveness of the model to a great extent. In this paper, we propose a novel Semi-Supervised Learning framework with Heterogeneous Distribution Consistency (HDC-SSL) for VI-ReID. Specifically, through investigating the confidence distribution of heterogeneous images, we introduce a Gaussian Mixture Model-based Pseudo Labeling (GMM-PL) method, which adaptively adjusts different thresholds for each modality to label the identity. Moreover, to facilitate the representation learning of unutilized data whose prediction is lower than the threshold, Modality Consistency Regularization (MCR) is proposed to ensure the prediction consistency of the cross-modality pedestrian images and handle the modality variance. Extensive experiments with different label settings on two VI-ReID datasets demonstrate the effectiveness of our method. Particularly, HDC-SSL achieves competitive performance with state-of-the-art fully-supervised VI-ReID methods on RegDB dataset with only 1 visible label and 1 infrared label per class.