Multimodal Imaging Data Research Articles

Identification of mild cognitive impairment using multimodal 3D imaging data and graph convolutional networks

Objective.Mild cognitive impairment (MCI) is a precursor stage of dementia characterized by mild cognitive decline in one or more cognitive domains, without meeting the criteria for dementia. MCI is considered a prodromal form of Alzheimer's disease (AD). Early identification of MCI is crucial for both intervention and prevention of AD. To accurately identify MCI, a novel multimodal 3D imaging data integration graph convolutional network (GCN) model is designed in this paper.Approach.The proposed model utilizes 3D-VGGNet to extract three-dimensional features from multimodal imaging data (such as structural magnetic resonance imaging and fluorodeoxyglucose positron emission tomography), which are then fused into feature vectors as the node features of a population graph. Non-imaging features of participants are combined with the multimodal imaging data to construct a population sparse graph. Additionally, in order to optimize the connectivity of the graph, we employed the pairwise attribute estimation （PAE） method to compute the edge weights based on non-imaging data, thereby enhancing the effectiveness of the graph structure. Subsequently, a population-based GCN integrates the structural and functional features of different modal images into the features of each participant for MCI classification.Main results.Experiments on the AD Neuroimaging Initiative demonstrated accuracies of 98.57%, 96.03%, and 96.83% for the normal controls (NC)-early MCI (EMCI), NC-late MCI (LMCI), and EMCI-LMCI classification tasks, respectively. The AUC, specificity, sensitivity, and F1-score are also superior to state-of-the-art models, demonstrating the effectiveness of the proposed model. Furthermore, the proposed model is applied to the ABIDE dataset for autism diagnosis, achieving an accuracy of 91.43% and outperforming the state-of-the-art models, indicating excellent generalization capabilities of the proposed model.Significance.This study demonstratesthe proposed model's ability to integrate multimodal imaging data and its excellent ability to recognize MCI. This will help achieve early warning for AD and intelligent diagnosis of other brain neurodegenerative diseases.

Characterizing the role of the microbiota-gut-brain axis in cerebral small vessel disease: An integrative multi‑omics study

BackgroundPrior efforts have revealed changes in gut microbiome, circulating metabolome, and multimodal neuroimaging features in cerebral small vessel disease (CSVD). However, there is a paucity of research integrating the multi-omic information to characterize the role of the microbiota-gut-brain axis in CSVD. MethodsWe collected gut microbiome, fecal and blood metabolome, multimodal magnetic resonance imaging data from 37 CSVD patients with white matter hyperintensities and 46 healthy controls. Between-group comparison was performed to identify the differential gut microbial taxa, followed by performance of multi-stage microbiome-metabolome-neuroimaging-neuropsychology correlation analyses in CSVD patients. ResultsOur data showed both depleted and enriched gut microbes in CSVD patients. Among the differential microbes, Haemophilus and Akkermansia were associated with a range of metabolites enriched for Aminoacyl-tRNA biosynthesis pathway. Furthermore, the affected metabolites were associated with neuroimaging measures involving gray matter morphology, spontaneous intrinsic brain activity, white matter integrity, and global structural network topology, which were in turn related to cognition and emotion in CSVD patients. ConclusionOur findings provide an integrative framework to understand the pathophysiological mechanisms underlying the interplay between gut microbiota dysbiosis and CSVD, highlighting the potential of targeting the microbiota-gut-brain axis as a therapeutic strategy in CSVD patients.

Predictive Diagnostic Model for Early Osteoporosis Detection Using Deep Learning and Multimodal Imaging Data: A Systematic Review and Meta-Analysis

Osteoporosis is a common condition that weakens bones, making them more prone to fractures. Early detection is crucial for preventing fractures and improving patients’ quality of life. However, traditional methods, such as Dual-Energy X-ray Absorptiometry (DXA), often struggle to accurately predict fracture risk and may overlook minor changes in bone structure. This study focuses on developing a predictive model for early osteoporosis detection using deep learning algorithms combined with various imaging techniques, including MRI, CT, and High-Resolution Peripheral Quantitative Computed Tomography (HR-pQCT). A systematic review and meta-analysis of studies published between 2014 and 2024 were conducted, examining the use of deep learning models applied to multimodal imaging data. The meta-analysis highlighted differences in the accuracy and effectiveness of various models, and their performance was measured in terms of accuracy, sensitivity, and specificity, following PRISMA guidelines. The results showed that deep learning models outperformed traditional methods in early osteoporosis detection. The use of multiple imaging techniques provided a more detailed assessment of bone health, allowing the models to identify complex patterns that are difficult for human interpretation. These models demonstrated high accuracy and significant potential for improving clinical decision-making. By integrating deep learning with multimodal imaging, this approach offers a promising solution for enhancing the early detection of osteoporosis. The models tested in this study proved to be highly effective, yielding more accurate fracture risk predictions and enabling earlier interventions. This could lead to better patient outcomes and reduced healthcare costs.

Comparison of different group-level templates in gradient-based multimodal connectivity analysis

Abstract The study of large-scale brain connectivity is increasingly adopting unsupervised approaches that derive low-dimensional spatial representations from high-dimensional connectomes, referred to as gradient analysis. When translating this approach to study interindividual variations in connectivity, one technical issue pertains to the selection of an appropriate group-level template to which individual gradients are aligned. Here, we compared different group-level template construction strategies using functional and structural connectome data from neurotypical controls and individuals with autism spectrum disorder (ASD) to identify between-group differences. We studied multimodal magnetic resonance imaging data obtained from the Autism Brain Imaging Data Exchange (ABIDE) Initiative II and the Human Connectome Project (HCP). We designed six template construction strategies that varied in whether (1) they included typical controls in addition to ASD; or (2) they mapped from one dataset onto another. We found that aligning a combined subject template of the ASD and control subjects from the ABIDE Initiative onto the HCP template exhibited the most pronounced effect size. This strategy showed robust identification of ASD-related brain regions for both functional and structural gradients across different study settings. Replicating the findings on focal epilepsy demonstrated the generalizability of our approach. Our findings will contribute to improving gradient-based connectivity research.

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

We propose a complete framework for Bayesian image reconstruction and uncertainty quantification based on a Gaussian process prior (GPP) to overcome limitations of maximum likelihood expectation maximization (ML-EM) image reconstruction algorithm. The prior distribution is constructed with a zero-mean Gaussian process (GP) with a choice of a covariance function, and a link function is used to map the Gaussian process to an image. Unlike many other maximum a posteriori approaches, our method offers highly interpretable hyperparamters that are selected automatically with the empirical Bayes method. Furthermore, the GP covariance function can be modified to incorporate a priori structural priors, enabling multi-modality imaging or contextual data fusion. Lastly, we illustrate that our approach lends itself to Bayesian uncertainty quantification techniques, such as the preconditioned Crank–Nicolson method and the Laplace approximation. The proposed framework is general and can be employed in most radiation image reconstruction problems, and we demonstrate it with simulated free-moving single detector radiation source imaging scenarios. We compare the reconstruction results from GPP and ML-EM, and show that the proposed method can significantly improve the image quality over ML-EM, all the while providing greater understanding of the source distribution via the uncertainty quantification capability. Furthermore, significant improvement of the image quality by incorporating a structural prior is illustrated.

Prognostic value of multimodality imaging in the contemporary management of cardiac sarcoidosis

BackgroundEchocardiography, cardiac magnetic resonance and cardiac 18fluorodeoxyglucose positron emission tomography (FDG-PET) imaging play key roles in the diagnosis and management of cardiac sarcoidosis (CS), but the relative value of each...

Linking transcriptome and morphology in bone cells at cellular resolution with generative AI

Abstract Recent advancements in deep learning (DL) have revolutionized the capability of artificial intelligence (AI) by enabling the analysis of large-scale, complex datasets that are difficult for humans to interpret. However, large amounts of high-quality data are required to train such generative AI models successfully. With the rapid commercialization of single-cell sequencing and spatial transcriptomics platforms, the field is increasingly producing large-scale datasets such as histological images, single-cell molecular data, and spatial transcriptomic data. These molecular and morphological datasets parallel the multimodal text and image data used to train highly successful generative AI models for natural language processing and computer vision. Thus, these emerging data types offer great potential to train generative AI models that uncover intricate biological processes of bone cells at a cellular level. In this Perspective, we summarize the progress and prospects of generative AI applied to these datasets and their potential applications to bone research. In particular, we highlight three AI applications: predicting cell differentiation dynamics, linking molecular and morphological features, and predicting cellular responses to perturbations. To make generative AI models beneficial for bone research, important issues, such as technical biases in bone single-cell datasets, lack of profiling of important bone cell types, and lack of spatial information, need to be addressed. Realizing the potential of generative AI for bone biology will also likely require generating large-scale, high-quality cellular-resolution spatial transcriptomics datasets, improving the sensitivity of current spatial transcriptomics datasets, and thorough experimental validation of model predictions.

Frontoparietal network topology as a neural marker of musical perceptual abilities

Why are some individuals more musical than others? Neither cognitive testing nor classical localizationist neuroscience alone can provide a complete answer. Here, we test how the interplay of brain network organization and cognitive function delivers graded perceptual abilities in a distinctively human capacity. We analyze multimodal magnetic resonance imaging, cognitive, and behavioral data from 200+ participants, focusing on a canonical working memory network encompassing prefrontal and posterior parietal regions. Using graph theory, we examine structural and functional frontoparietal network organization in relation to assessments of musical aptitude and experience. Results reveal a positive correlation between perceptual abilities and the integration efficiency of key frontoparietal regions. The linkage between functional networks and musical abilities is mediated by working memory processes, whereas structural networks influence these abilities through sensory integration. Our work lays the foundation for future investigations into the neurobiological roots of individual differences in musicality.

Baseline multimodal imaging to predict longitudinal clinical decline in atypical Alzheimer's disease

There are recognized neuroimaging regions of interest in typical Alzheimer's disease which have been used to track disease progression and aid prognostication. However, there is a need for validated baseline imaging markers to predict clinical decline in atypical Alzheimer's Disease. We aimed to address this need by producing models from baseline imaging features using penalized regression and evaluating their predictive performance on various clinical measures.Baseline multimodal imaging data, in combination with clinical testing data at two time points from 46 atypical Alzheimer's Disease patients with a diagnosis of logopenic progressive aphasia (N = 24) or posterior cortical atrophy (N = 22), were used to generate our models. An additional 15 patients (logopenic progressive aphasia = 7, posterior cortical atrophy = 8), whose data were not used in our original analysis, were used to test our models. Patients underwent MRI, FDG-PET and Tau-PET imaging and a full neurologic battery at two time points. The Schaefer functional atlas was used to extract network-based and regional gray matter volume or PET SUVR values from baseline imaging. Penalized regression (Elastic Net) was used to create models to predict scores on testing at Time 2 while controlling for baseline performance, education, age, and sex. In addition, we created models using clinical or Meta Region of Interested (ROI) data to serve as comparisons.We found the degree of baseline involvement on neuroimaging was predictive of future performance on cognitive testing while controlling for the above measures on all three imaging modalities. In many cases, model predictability improved with the addition of network-based neuroimaging data to clinical data. We also found our network-based models performed superiorly to the comparison models comprised of only clinical or a Meta ROI score.Creating predictive models from imaging studies at a baseline time point that are agnostic to clinical diagnosis as we have described could prove invaluable in both the clinical and research setting, particularly in the development and implementation of future disease modifying therapies.

Imaging of Sacroiliac Pain: The Current State-of-the-Art.

Pain in the sacroiliac (SI) region is a common clinical manifestation, often caused by diseases involving the SI joints. This is typically due to inflammation or degenerative changes, while infections or cancer are less frequent causes. The SI joint is challenging to image accurately because of its distinct anatomical characteristics. For an accurate diagnosis, conventional radiography often needs to be supplemented with more precise methods such as magnetic resonance imaging (MRI) or computed tomography (CT). Sacroiliitis, a common presenting feature of axial spondyloarthritis (axial SpA), manifests as bone marrow edema, erosions, sclerosis, and joint space narrowing. Septic sacroiliitis and repetitive stress injuries in sports can also cause changes resembling inflammatory sacroiliitis. Other conditions, such as osteitis condensans ilii (OCI), can mimic the radiologic characteristics of sacroiliitis. Inflammatory lesions are diagnosed by concurrent erosions, hyperostosis, and ankylosis. Ligament ossifications or mechanical stress can also result in arthritic disorders. Determining the exact diagnosis can be aided by the distribution of the lesions. Inflammatory lesions can affect any part of the articulation, including the inferior and posterior portions. Mechanical lesions, such as those seen in OCI, often occur in the anterior middle region of the joint. In cases of idiopathic skeletal hyperostosis, ligament ossification is found at the joint borders. This pictorial essay describes common SI joint problems, illustrated with multimodal imaging data. We, also, discuss strategies for selecting the best imaging modalities, along with imaging pitfalls, key points, and approaches for treating patients with suspected inflammatory back pain.

Neural mass modelling for the masses: Democratising access to whole-brain biophysical modelling with FastDMF

Abstract Different whole-brain computational models have been recently developed to investigate hypotheses related to brain mechanisms. Among these, the Dynamic Mean Field (DMF) model is particularly attractive, combining a biophysically realistic model that is scaled up via a mean-field approach and multimodal imaging data. However, an important barrier to widespread usage of the DMF model is that current implementations are computationally expensive, supporting only simulations on brain parcellations that consider less than 100 brain regions. Here, we introduce an efficient and accessible implementation of the DMF model: the FastDMF. By leveraging analytical and numerical advances – including a novel estimation of the feedback inhibition control parameter, and a Bayesian optimization algorithm – the FastDMF circumvents various computational bottlenecks of previous implementations, improving interpretability, performance and memory use. Furthermore, these advances allow the FastDMF to increase the number of simulated regions by one order of magnitude, as confirmed by the good fit to fMRI data parcellated at 90 and 1000 regions. These advances open the way to the widespread use of biophysically grounded whole-brain models for investigating the interplay between anatomy, function, and brain dynamics, and to identify mechanistic explanations of recent results obtained from fine-grained neuroimaging recordings.

Enhanced river suspended sediment concentration identification via multimodal video image, optical flow, and water temperature data fusion

Monitoring suspended sediment concentration (SSC) in rivers is pivotal for water quality management and sustainable river ecosystem development. However, achieving continuous and precise SSC monitoring is fraught with challenges, including low automation, lengthy measurement processes, and high cost. This study proposes an innovative approach for SSC identification in rivers using multimodal data fusion. We developed a robust model by harnessing colour features from video images, motion characteristics from the Lucas–Kanade (LK) optical flow method, and temperature data. By integrating ResNet with a mixed density network (MDN), our method fused the image and optical flow fields, and temperature data to enhance accuracy and reliability. Validated at a hydropower station in the Xinjiang Uygur Autonomous Region, China, the results demonstrated that while the image field alone offers a baseline level of SSC identification, it experiences local errors under specific conditions. The incorporation of optical flow and water temperature information enhanced model robustness, particularly when coupling the image and optical flow fields, yielding a Nash–Sutcliffe efficiency (NSE) of 0.91. Further enhancement was observed with the combined use of all three data types, attaining an NSE of 0.93. This integrated approach offers a more accurate SSC identification solution, enabling non-contact, low-cost measurements, facilitating remote online monitoring, and supporting water resource management and river water–sediment element monitoring.

Ultrasound and diffuse optical tomography-transformer model for assessing pathological complete response to neoadjuvant chemotherapy in breast cancer.

We evaluate the efficiency of integrating ultrasound (US) and diffuse optical tomography (DOT) images for predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer patients. The ultrasound-diffuse optical tomography (USDOT)-Transformer model represents a significant step toward accurate prediction of pCR, which is critical for personalized treatment planning. We aim to develop and assess the performance of the USDOT-Transformer model, which combines US and DOT images with tumor receptor biomarkers to predict the pCR of breast cancer patients under NAC. We developed the USDOT-Transformer model using a dual-input transformer to process co-registered US and DOT images along with tumor receptor biomarkers. Our dataset comprised imaging data from 60 patients at multiple time points during their chemotherapy treatment. We used fivefold cross-validation to assess the model's performance, comparing its results against a single modality of US or DOT. The USDOT-Transformer model demonstrated excellent predictive performance, with a mean area under the receiving characteristic curve of 0.96 (95%CI: 0.93 to 0.99) across the fivefold cross-validation. The integration of US and DOT images significantly enhanced the model's ability to predict pCR, outperforming models that relied on a single imaging modality (0.87 for US and 0.82 for DOT). This performance indicates the potential of advanced deep learning techniques and multimodal imaging data for improving the accuracy (ACC) of pCR prediction. The USDOT-Transformer model offers a promising non-invasive approach for predicting pCR to NAC in breast cancer patients. By leveraging the structural and functional information from US and DOT images, the model offers a faster and more reliable tool for personalized treatment planning. Future work will focus on expanding the dataset and refining the model to further improve its accuracy and generalizability.

Introducing ARTiMiS: A Low-Cost Flow Imaging Microscope for Microalgal Monitoring.

Manual microscopy is the gold standard for phytoplankton monitoring in diverse engineered and natural environments. However, it is both labor-intensive and requires specialized training for accuracy and consistency, and therefore difficult to implement on a routine basis without significant time investment. Automation can reduce this burden by simplifying the measurement to a single indicator (e.g., chlorophyll fluorescence) measurable by a probe, or by processing samples on an automated cytometer for more granular information. The cost of commercially available flow imaging cytometers, however, poses a steep financial barrier to adoption. To overcome these labor and cost barriers, we developed ARTiMiS: the Autonomous Real-Time Microbial 'Scope. The ARTiMiS is a low-cost flow imaging microscopy-based platform with onboard software capable of providing taxonomically resolved quantitation of phytoplankton communities in real-time. ARTiMiS leverages novel multimodal imaging and onboard machine learning-based data processing that is currently optimized for a curated and expandable database of industrially relevant microalgae. We demonstrate its operational limits, performance in identification of laboratory-cultivated microalgae, and potential for continuous monitoring of complex microalgal communities in full-scale industrial cultivation systems.

A Novel Fragmented Approach for Securing Medical Health Records in Multimodal Medical Images

Medical health records hold personal medical information and should only be accessed by authorized medical personnel or concerned patients. The importance of medical health records privacy is increasing as these records are shared in cloud environments. In this paper, we propose an enhanced system for securing patient data (Medical Health Records) embedded in multiple medical images in fragments for secure transmission and public sharing on the cloud or other environments. To protect the patient’s privacy, Medical Records are first encrypted, and then the ciphertext is broken into several fragments based on the number of multimodal medical images of a patient. A key generator randomly selects medical images from the multimodal image data to embed the encrypted patient health record segment using a modified least significant bit embedding process. The proposed technique enables an extra layer of security as even if files fall into the wrong hands and a fragment of the file is decrypted, it will not present any understandable information until all fragments from other medical images are extracted and combined in the correct order. The experiments are performed using multimodal 3255 MRI scans of 21 patients. The robustness of the proposed method was measured using different metrics such as PSNR, MSE, and SSIM. The results show that the proposed system is robust and that image quality is also maintained. To further study the stego image quality, a deep learning-based classification was applied to the images, and the results show that the diagnosis using stego medical images and performance remains unaffected even after embedding the encrypted data.

Multimodal phenotyping of foveal hypoplasia in albinism and albino-like conditions: a pediatric case series with adaptive optics insights

Aim of the present study is to evaluate the relationship between genetic and phenotypic data in a series of patients affected by grade I and II of foveal hypoplasia with stable fixation and good visual acuity using multimodal imaging techniques. All patients underwent complete clinical and instrumental assessment including structural Optical Coherence Tomography (OCT), OCT Angiography and Adaptive Optics (AO) imaging. Central macular thickness (CMT), inner nuclear layer (INL), vessel density in superficial capillary plexus were the main variables evaluated with OCT technology. Cone density, cone spacing, cone regularity, cone dispersion and angular density were the parameters evaluated with AO. Genetic evaluation and trio exome sequencing were performed in all affected individuals. Eight patients (3 males and 5 females) with a mean age of 12.62 years (range 8–18) were enrolled. The mean best corrected visual acuity (BCVA) was 0.18 ± 0.13 logMAR, mean CMT was 291.9 ± 16.6 µm and INL was 26.2 ± 4.6 µm. The absence of a foveal avascular zone (FAZ) was documented by examination of OCT-A in seven patients in the superficial capillary plexus. However, there was a partial FAZ in the deep plexus in patients P5 and P8. Of note, all the patients presented with major retinal vessels clearly crossing the foveal center. All individuals exhibited a grade I or II of foveal hypoplasia. In 5 patients molecular analyses showed an extremely mild form of albinism caused by compound heterozygosity of a TYR pathogenic variant and the hypomorphic p.[Ser192Tyr;Arg402Gln] haplotype. One patient had Waardenburg syndrome type 2A caused by a de novo variant in MITF. Two patients had inconclusive molecular analyses. All the patients displayed abnormalities on OCT-A. Photoreceptor count did not differ from normal subjects according to the current literature, but qualitative analysis of AO imaging showed distinctive features likely related to an abnormal pigment distribution in this subset of individuals. In patients with foveal hypoplasia, genetic and multimodal imaging data, including AO findings, can help understand the physiopathology of the foveal hypoplasia phenotype. This study confirms that cone density and visual function can both be preserved despite the absence of a pit.

Linking functional and structural brain organisation with behaviour in healthy adults.

Multimodal data integration approaches, such as Linked Independent Component Analysis (LICA), increase sensitivity to brain-behaviour relationships and allow us to probe the relationship between modalities. Here we focus on inter-regional functional and structural organisation to determine if organisational patterns persist across modalities and if investigating multi-modality organisations provides increased sensitivity to brain-behaviour associations. We utilised multimodal magnetic resonance imaging (MRI; T1w, resting-state functional [fMRI] and diffusion weighted [DWI]) and behavioural data from the Human Connectome Project (HCP, n=676; 51% female). Unimodal features were extracted to produce individual grey matter density maps, probabilistic tractography connectivity matrices and connectopic maps from the T1w, DWI and fMRI data, respectively. DWI and fMRI analyses were restricted to subcortical regions for computational reasons. LICA was then used to integrate features, generating 100 novel independent components. Associations between these components and demographic/behavioural (n=308) variables were examined. 15 components were significantly associated with various demographic/behavioural measures. 2 components were strongly related to various measures of intoxication, driven by DWI and resemble components previously identified. Another component was driven by striatal functional data and related to working memory. A small number of components showed shared variance between structure and function but none of these displayed any significant behavioural associations. Our working memory findings provide support for the use of fMRI connectopic mapping in future research of working memory. Given the lack of behaviourally relevant shared variance between functional and structural organisation, as indexed here, we question the utility of integrating connectopic maps and tractography data.

Deep Attention Fusion Hashing (DAFH) Model for Medical Image Retrieval.

To address this, we propose a novel deep learning-based hashing model, the Deep Attention Fusion Hashing (DAFH) model, which integrates advanced attention mechanisms with medical imaging data. The DAFH model enhances retrieval performance by integrating multi-modality medical imaging data and employing attention mechanisms to optimize the feature extraction process. Utilizing multimodal medical image data from the Cancer Imaging Archive (TCIA), this study constructed and trained a deep hashing network that achieves high-precision classification of various cancer types. At hash code lengths of 16, 32, and 48 bits, the model respectively attained Mean Average Precision (MAP@10) values of 0.711, 0.754, and 0.762, highlighting the potential and advantage of the DAFH model in medical image retrieval. The DAFH model demonstrates significant improvements in the efficiency and accuracy of medical image retrieval, proving to be a valuable tool in clinical settings.

Multimodal neuroimaging exploration of the mechanisms of sleep quality deterioration after SARS-CoV-2 Omicron infection

BackgroundTo evaluate the neurological alterations induced by Omicron infection, to compare brain changes in chronic insomnia with those in exacerbated chronic insomnia in Omicron patients, and to examine individuals without insomnia alongside those with new-onset insomnia.MethodsIn this study, a total of 135 participants were recruited between January 11 and May 4, 2023, including 26 patients with chronic insomnia without exacerbation, 24 patients with chronic insomnia with exacerbation, 40 patients with no sleep disorder, and 30 patients with new-onset insomnia after infection with Omicron (a total of 120 participants with different sleep statuses after infection), as well as 15 healthy controls who were never infected with Omicron. Neuropsychiatric data, clinical symptoms, and multimodal magnetic resonance imaging data were collected. The gray matter thickness and T1, T2, proton density, and perivascular space values were analyzed. Associations between changes in multimodal magnetic resonance imaging findings and neuropsychiatric data were evaluated with correlation analyses.ResultsCompared with healthy controls, gray matter thickness changes were similar in the patients who have and do not have a history of chronic insomnia groups after infection, including an increase in cortical thickness near the parietal lobe and a reduction in cortical thickness in the frontal, occipital, and medial brain regions. Analyses showed a reduced gray matter thickness in patients with chronic insomnia compared with those with an aggravation of chronic insomnia post-Omicron infection, and a reduction was found in the right medial orbitofrontal region (mean [SD], 2.38 [0.17] vs. 2.67 [0.29] mm; P < 0.001). In the subgroups of Omicron patients experiencing sleep deterioration, patients with a history of chronic insomnia whose insomnia symptoms worsened after infection displayed heightened medial orbitofrontal cortical thickness and increased proton density values in various brain regions. Conversely, patients with good sleep quality who experienced a new onset of insomnia after infection exhibited reduced cortical thickness in pericalcarine regions and decreased proton density values. In new-onset insomnia patients post-Omicron infection, the thickness in the right pericalcarine was negatively correlated with the Self-rating Anxiety Scale (r = − 0.538, P = 0.002, PFDR = 0.004) and Self-rating Depression Scale (r = − 0.406, P = 0.026, PFDR = 0.026) scores.ConclusionsThese findings help us understand the pathophysiological mechanisms involved when Omicron invades the nervous system and induces various forms of insomnia after infection. In the future, we will continue to pay attention to the dynamic changes in the brain related to insomnia caused by Omicron infection.

CMR-net: A cross modality reconstruction network for multi-modality remote sensing classification.

In recent years, the classification and identification of surface materials on earth have emerged as fundamental yet challenging research topics in the fields of geoscience and remote sensing (RS). The classification of multi-modality RS data still poses certain challenges, despite the notable advancements achieved by deep learning technology in RS image classification. In this work, a deep learning architecture based on convolutional neural network (CNN) is proposed for the classification of multimodal RS image data. The network structure introduces a cross modality reconstruction (CMR) module in the multi-modality feature fusion stage, called CMR-Net. In other words, CMR-Net is based on CNN network structure. In the feature fusion stage, a plug-and-play module for cross-modal fusion reconstruction is designed to compactly integrate features extracted from multiple modalities of remote sensing data, enabling effective information exchange and feature integration. In addition, to validate the proposed scheme, extensive experiments were conducted on two multi-modality RS datasets, namely the Houston2013 dataset consisting of hyperspectral (HS) and light detection and ranging (LiDAR) data, as well as the Berlin dataset comprising HS and synthetic aperture radar (SAR) data. The results demonstrate the effectiveness and superiority of our proposed CMR-Net compared to several state-of-the-art methods for multi-modality RS data classification.