Robust Representation Research Articles

High-fidelity and high-speed wavefront shaping by leveraging complex media.

High-precision light manipulation is crucial for delivering information through complex media. However, existing spatial light modulation devices face a fundamental speed-fidelity tradeoff. Digital micromirror devices have emerged as a promising candidate for high-speed wavefront shaping but at the cost of compromised fidelity due to the limited control degrees of freedom. Here, we leverage the sparse-to-random transformation through complex media to overcome the dimensionality limitation of spatial light modulation devices. We demonstrate that pattern compression by sparsity-constrained wavefront optimization allows sparse and robust wavefront representations in complex media, improving the projection fidelity without sacrificing frame rate, hardware complexity, or optimization time. Our method is generalizable to different pattern types and complex media, supporting consistent performance with up to 89% and 126% improvements in projection accuracy and speckle suppression, respectively. The proposed optimization framework could enable high-fidelity high-speed wavefront shaping through different scattering media and platforms without changes to the existing holographic setups, facilitating a wide range of physics and real-world applications.

Contrastive Self-supervised Learning for Neurodegenerative Disorder Classification.

Neurodegenerative diseases such as Alzheimer's disease (AD) or frontotemporal lobar degeneration (FTLD) involve specific loss of brain volume, detectable in vivo using T1-weighted MRI scans. Supervised machine learning approaches classifying neurodegenerative diseases require diagnostic-labels for each sample. However, it can be difficult to obtain expert labels for a large amount of data. Self-supervised learning (SSL) offers an alternative for training machine learning models without data-labels. We investigated if the SSL models can applied to distinguish between different neurodegenerative disorders in an interpretable manner. Our method comprises a feature extractor and a downstream classification head. A deep convolutional neural network trained in a contrastive self-supervised way serves as the feature extractor, learning latent representation, while the classifier head is a single-layer perceptron. We used N=2694 T1-weighted MRI scans from four data cohorts: two ADNI datasets, AIBL and FTLDNI, including cognitively normal controls (CN), cases with prodromal and clinical AD, as well as FTLD cases differentiated into its sub-types. Our results showed that the feature extractor trained in a self-supervised way provides generalizable and robust representations for the downstream classification. For AD vs. CN, our model achieves 82% balanced accuracy on the test subset and 80% on an independent holdout dataset. Similarly, the Behavioral variant of frontotemporal dementia (BV) vs. CN model attains an 88% balanced accuracy on the test subset. The average feature attribution heatmaps obtained by the Integrated Gradient method highlighted hallmark regions, i.e., temporal gray matter atrophy for AD, and insular atrophy for BV. In conclusion, our models perform comparably to state-of-the-art supervised deep learning approaches. This suggests that the SSL methodology can successfully make use of unannotated neuroimaging datasets as training data while remaining robust and interpretable.

Learning Robust Shape Regularization for Generalizable Medical Image Segmentation.

Generalizable medical image segmentation enables models to generalize to unseen target domains under domain shift issues. Recent progress demonstrates that the shape of the segmentation objective, with its high consistency and robustness across domains, can serve as a reliable regularization to aid the model for better cross-domain performance, where existing methods typically seek a shared framework to render segmentation maps and shape prior concurrently. However, due to the inherent texture and style preference of modern deep neural networks, the edge or silhouette of the extracted shape will inevitably be undermined by those domain-specific texture and style interferences of medical images under domain shifts. To address this limitation, we devise a novel framework with a separation between the shape regularization and the segmentation map. Specifically, we first customize a novel whitening transform-based probabilistic shape regularization extractor namely WT-PSE to suppress undesirable domain-specific texture and style interferences, leading to more robust and high-quality shape representations. Second, we deliver a Wasserstein distance-guided knowledge distillation scheme to help the WT-PSE to achieve more flexible shape extraction during the inference phase. Finally, by incorporating domain knowledge of medical images, we propose a novel instance-domain whitening transform method to facilitate a more stable training process with improved performance. Experiments demonstrate the performance of our proposed method on both multi-domain and single-domain generalization.

Evaluating Existing Ocean Glider Sampling Strategies for Submesoscale Dynamics

Abstract Mixing in the upper ocean is important for biological production and the transfer of heat and carbon between the atmosphere and deep ocean, properties commonly targeted by observational campaigns using ocean gliders. We assess the reliability of ocean gliders to obtain a robust statistical representation of submesoscale variability in the ocean mixed layer of the Weddell Sea. A 1/48° regional simulation of the Southern Ocean is sampled with virtual “bow-tie” glider deployments, which are then compared against the reference model output. Sampling biases of lateral buoyancy gradients associated with the arbitrary alignment between glider paths and fronts are formally quantified, and the magnitude of the biases is comparable to observational estimates, with a mean error of 52%. The sampling bias leaves errors in the retrieved distribution of buoyancy gradients largely insensitive to deployment length and the deployment of additional gliders. Notable sensitivity to these choices emerges when the biases are removed by sampling perpendicular to fronts at all times. Detecting seasonal change in the magnitude of buoyancy gradients is sensitive to the glider-orientation sampling bias but the change in variance is not. We evaluate the impact of reducing the number of dives and climbs in an observational campaign and find that small reductions in the number of dive–climb pairs have a limited effect on the results. Lastly, examining the sensitivity of the sampling bias to path orientation indicates that the bias is not dependent on the direction of travel in our deep ocean study site. Significance Statement Recent observational campaigns have focused on using autonomous vehicles to better understand processes responsible for mixing in the surface region of the ocean. There exists uncertainty around how effective these missions are at returning reliable and representative information. This study seeks to quantify the performance of existing strategies in observing mixing processes, and we confirm that strategies are biased to underestimate indicators of mixing. Furthermore, compensating for the bias by increasing the number of resources or changing the manner in which resources are used has limited reward. Our findings are important for decision-making during the planning phase of an observational campaign and display that further innovations are required to account for the sampling bias.

Joint low-rank tensor fusion and cross-modal attention for multimodal physiological signals based emotion recognition

Objective. Physiological signals based emotion recognition is a prominent research domain in the field of human-computer interaction. Previous studies predominantly focused on unimodal data, giving limited attention to the interplay among multiple modalities. Within the scope of multimodal emotion recognition, integrating the information from diverse modalities and leveraging the complementary information are the two essential issues to obtain the robust representations. Approach. Thus, we propose a intermediate fusion strategy for combining low-rank tensor fusion with the cross-modal attention to enhance the fusion of electroencephalogram, electrooculogram, electromyography, and galvanic skin response. Firstly, handcrafted features from distinct modalities are individually fed to corresponding feature extractors to obtain latent features. Subsequently, low-rank tensor is fused to integrate the information by the modality interaction representation. Finally, a cross-modal attention module is employed to explore the potential relationships between the distinct latent features and modality interaction representation, and recalibrate the weights of different modalities. And the resultant representation is adopted for emotion recognition. Main results. Furthermore, to validate the effectiveness of the proposed method, we execute subject-independent experiments within the DEAP dataset. The proposed method has achieved the accuracies of 73.82% and 74.55% for valence and arousal classification. Significance. The results of extensive experiments verify the outstanding performance of the proposed method.

Simulation and Modeling of the Adhesion of Staphylococcus aureus onto Inert Surfaces under Fluid Shear Stress.

Bacterial adhesion to biotic and abiotic surfaces under fluid shear stress plays a major role in the pathogenesis of infections linked to medical implants and tissues. This study employed an automated BioFlux 200 microfluidic system and video microscopy to conduct real-time adhesion assays, examining the influence of shear stress on adhesion kinetics and spatial distribution of Staphylococcus aureus on glass surfaces. The adhesion rate exhibited a non-linear relationship with shear stress, with notable variations at intermediate levels. Empirical adhesion events were simulated with COMSOL Multiphysics® and Python. Overall, COMSOL accurately predicted the experimental trend of higher rates of bacterial adhesion with decreasing shear stress but poorly characterized the plateauing phenomena observed over time. Python provided a robust mathematical representation of the non-linear relationship between cell concentration, shear stress, and time but its polynomial regression approach was not grounded on theoretical physical concepts. These insights, combined with advancements in AI and machine learning, underscore the potential for synergistic computational techniques to enhance our understanding of bacterial adhesion to surfaces, offering a promising avenue for developing novel therapeutic strategies.

MolLM: a unified language model for integrating biomedical text with 2D and 3D molecular representations.

The current paradigm of deep learning models for the joint representation of molecules and text primarily relies on 1D or 2D molecular formats, neglecting significant 3D structural information that offers valuable physical insight. This narrow focus inhibits the models' versatility and adaptability across a wide range of modalities. Conversely, the limited research focusing on explicit 3D representation tends to overlook textual data within the biomedical domain. We present a unified pre-trained language model, MolLM, that concurrently captures 2D and 3D molecular information alongside biomedical text. MolLM consists of a text Transformer encoder and a molecular Transformer encoder, designed to encode both 2D and 3D molecular structures. To support MolLM's self-supervised pre-training, we constructed 160K molecule-text pairings. Employing contrastive learning as a supervisory signal for learning, MolLM demonstrates robust molecular representation capabilities across four downstream tasks, including cross-modal molecule and text matching, property prediction, captioning, and text-prompted molecular editing. Through ablation, we demonstrate that the inclusion of explicit 3D representations improves performance in these downstream tasks. Our code, data, pre-trained model weights, and examples of using our model are all available at https://github.com/gersteinlab/MolLM. In particular, we provide Jupyter Notebooks offering step-by-step guidance on how to use MolLM to extract embeddings for both molecules and text.

Lightweight multimodal Cycle-Attention Transformer towards cancer diagnosis

Cancer diagnosis, prognosis and therapeutic response predictions are based on data extracted from various modalities, such as histology slides, cell graphs and molecular profiles from genomic data. In recent years, with the increasing availability of multimodal biomedical data, an increasing number of multimodal fusion strategies have been proposed to overcome the challenge of data heterogeneity and make full use of intersections and complementarities of different data sources. However, most of them have the problem of parameter redundancy and inefficiency in the process of fusion. Therefore, in this work, we present a Cycle-Attention Transformer (Cy-Atten) framework that learns a more robust joint representation for multimodal cancer survival analysis. The Cy-Atten adopts a cycling mechanism with sparse attention and yields a theoretical cost of O(N) for adding a modality, which is proven to effectively reduce the parameter redundancy. Empirically in the experiments on two cancer datasets from the TCGA database, the parameters in our fusion part are only of the order of 103 considering three modalities. In addition to being lightweight, the Cy-Atten outperforms several state-of-the-art multimodal fusion methods in survival analysis and grade classification of two cancers from the TCGA database. Our work can be reproduced at https://github.com/dlmscl/Cycle-Attention-Transformer.

Automated Segmentation of Brain Gliomas in Multimodal MRI Data

ABSTRACTBrain gliomas, common in adults, pose significant diagnostic challenges. Accurate segmentation from multimodal magnetic resonance imaging (MRI) scans is critical for effective treatment planning. Traditional manual segmentation methods, labor‐intensive and error‐prone, often lead to inconsistent diagnoses. To overcome these limitations, our study presents a sophisticated framework for the automated segmentation of brain gliomas from multimodal MRI images. This framework consists of three integral components: a 3D UNet, a classifier, and a Classifier Weight Transformer (CWT). The 3D UNet, acting as both an encoder and decoder, is instrumental in extracting comprehensive features from MRI scans. The classifier, employing a streamlined 1 × 1 convolutional architecture, performs detailed pixel‐wise classification. The CWT integrates self‐attention mechanisms through three linear layers, a multihead attention module, and layer normalization, dynamically refining the classifier's parameters based on the features extracted by the 3D UNet, thereby improving segmentation accuracy. Our model underwent a two‐stage training process for maximum efficiency: in the first stage, supervised learning was used to pre‐train the encoder and decoder, focusing on robust feature representation. In the second stage, meta‐training was applied to the classifier, with the encoder and decoder remaining unchanged, ensuring precise fine‐tuning based on the initially developed features. Extensive evaluation of datasets such as BraTS2019, BraTS2020, BraTS2021, and a specialized private dataset (ZZU) underscored the robustness and clinical potential of our framework, highlighting its superiority and competitive advantage over several state‐of‐the‐art approaches across various segmentation metrics in training and validation sets.

Orchard bird song recognition based on multi-view multi-level contrastive learning

In the harvest season, orchards are frequently plagued by birds, and thus significant fruit pecking can adversely affect both the fruit quality and yield. Recognizing bird songs is crucial for preventing damage caused by orchard birds because it can provide the basis for subsequent bird repellent efforts. However, the extensive effort required to annotate sound samples poses a significant challenge for supervised deep learning. In this paper, we propose a self-supervised multi-view learning framework based on multi-level contrasting (MV-MLC) for bird song recognition, which utilizes both time and spectrogram views as inputs. This framework leverages MLC to automatically learn representations from unlabeled data and a multi-scale feature extraction (MSFE) backbone network is employed to capture the temporal features of bird songs at different scales. The time-spectrogram consistency task in MLC learning facilitates semantic-level information exchange across multi-views, while the hierarchical contrastive learning task captures granularity-level information, thereby resulting in more robust contextual representations. In addition, embedding the shuffle attention module in MSFE facilitates mining of the spatial and channel dependencies of bird song features to further enhance the representation of features by the multi-scale network. We conducted extensive experiments using our self-built 10-class bird song data set (Orchard-birds) and the publicly available Birdsdata and Powdermill data sets. The experimental results demonstrated that MV-MLC performed better than state-of-the-art self-supervised models. In particular, MV-MLC obtained outstanding performance even with a small proportion of labeled data. The recognition accuracies based on the Orchard-birds and Birdsdata data sets were 99.40% and 92.67%, respectively, with macro F1-scores of 99.40% and 92.61%.

Arithmetic in two languages: Localizing simple multiplication processing in the adult bilingual brain.

Verbally memorized multiplication tables are thought to create language-specific memories. Supporting this idea, bilinguals are typically faster and more accurate in the language in which they learned math (LA+) than in their other language (LA- ) . No study has yet revealed the underlying neurocognitive mechanisms explaining this effect, or the role of problem size in explaining the recruitment of different brain regions in LA+ and LA- . To fill this gap in the literature, 29 Spanish-English early bilingual adults, proficient in both languages, verified simple multiplication problems in each language while functional magnetic resonance imaging (fMRI) was acquired. More specifically, this study aimed to answer two questions: 1) Does LA+ recruit left superior and middle temporal gyri (STG/MTG) to a greater extent than LA- , reflecting more robust verbal representations of multiplication facts in LA+? In contrast, does LA- recruit the inferior frontal gyrus (IFG), reflecting more effortful retrieval, or the intraparietal sulcus (IPS), reflecting reliance on quantity processes? 2) Is there an interaction between language and problem size, where language differences are more pronounced for less practiced, large multiplication problems (e.g., 8 × 9) in comparison to more familiar, small problems (e.g., 2 × 3). Functional localizer tasks were used to identify hypothesis-driven regions of interest in verbal areas associated with verbal representations of arithmetic facts (left STG/MTG) and with the effortful retrieval of these facts (left IFG) and quantity areas engaged when calculation-based strategies are used (bilateral IPS). In planned analyses, no cluster reached significance for the direct comparison of languages (question 1) or for the interaction between language and problem size (question 2). An exploratory analysis found a main effect of problem size, where small problems recruited left STG/MTG and left IFG to a greater extent than large problems, suggesting greater verbal involvement for these problems in both languages. Additionally, large problems recruited right IPS to a greater extent than small problems, suggesting reliance on quantity processes. Our results suggest that proficient early bilingual adults engage similar brain regions in both languages, even for more difficult, large problems.

Water quality and surroundings for systemic management of springs in rural areas

Springs enable the passage of groundwater to flow to the ground surface, forming and sustaining watercourses. Despite the importance of these ecosystems, their functionality and the availability of water in terms of quantity and quality in rural areas have been threatened. The objective of this study was to evaluate the water quality of springs under different care and preservation scenarios in rural areas and assess the need to combine water potability analysis with biological indicators and the effects of the interaction between aquatic and terrestrial environments on the characteristics of surroundings to generate a systemic and robust representation of the impact of human actions on these threatened environments. The study was developed considering the Ijuí River Basin, in the northwestern state of Rio Grande do Sul, Brazil. It involved the evaluation of springs in a native forest area protected by drainage and a preserved vegetation; in a reforested area with concrete protection and surrounding vegetation; and in a livestock area with no vegetation fragments in the surroundings and with an outcrop protected by concrete pipes. Physicochemical and microbiological analyses are essential for determining the potability of spring water. However, these are specific assessments that do not reflect historical environmental conditions. The use of bioindicators enables these analyses and, together with the determination of surroundings' characteristics, allows for evaluating the degree of preservation to ensure spring quality, providing perspectives that combine care and sustainable use.

Few-shot online anomaly detection and segmentation

Detecting anomaly patterns from images is a crucial artificial intelligence technique in industrial applications. Recent research in this domain has emphasized the necessity of a large volume of training data, overlooking the practical scenario where, post-deployment of the model, unlabeled data containing both normal and abnormal samples can be utilized to enhance the performance. Consequently, this paper focuses on addressing the challenging yet practical few-shot online anomaly detection and segmentation (FOADS) task. Under the FOADS framework, models are trained on a few-shot normal dataset, followed by inspection and improvement of their capabilities by leveraging unlabeled streaming data containing both normal and abnormal samples simultaneously. Since the data stream has no ground truth labels, the model needs to filter the anomalous data by detection to avoid contaminating its parameters, which presents a significant challenge when initial samples are insufficient. To tackle this issue, we propose modeling the feature distribution of normal images using a Neural Gas network, which offers the flexibility to adapt the topology structure to identify outliers in the data flow. In order to achieve improved performance with limited training samples, we employ multi-scale feature embedding extracted from a CNN pre-trained on ImageNet to obtain a robust representation. Furthermore, we introduce an algorithm that can incrementally update parameters without the need to store previous samples. Comprehensive experimental results demonstrate that our method can achieve substantial performance under the FOADS setting, while ensuring that the time complexity remains within an acceptable range on MVTec AD and BTAD datasets.

Dual-consistency constraints network for noisy facial expression recognition

Although existing facial expression recognition (FER) methods have achieved great success, their performance degrades significantly under noisy labels caused by low-quality images, ambiguous expressions, and subjective and incorrect labeling. Recent studies have shown that deep neural networks (DNNs) can easily overfit noisy labels, which poses a great challenge to FER task in real-world scenarios. To address this issue, we propose a novel Dual-consistency Constraints Network (DC-Net) to automatically suppress noisy samples during training. Specifically, we first propose a Class Activation Mapping (CAM) Attention Consistency (CAC), which makes the model focus on partially important feature information. As a result, we obtain more robust local feature representations and reduce excessive attention to noisy labels. Then, a Class Feature Consistency (CFC) is designed to encourage the model to focus on the global semantic information of the image. Finally, with the collaboration of the CAC and the CFC, DC-Net can learn robust local and global feature information to prevent the model from learning biased information with noisy labels. We conducted extensive experiments on three field datasets, including RAF-DB, AffectNet, and FERPlus2013. Experimental results show that DC-Net significantly outperforms state-of-the-art noisy labeling methods at different noise rates and generalizes well to other tasks with a large number of classes, such as CIFAR100 and Tiny- ImageNet.

High Glass Transition Temperature Fluorinated Polymers Based on Transfer Learning with Small Experimental Data.

Machine learning can be used to predict the properties of polymers and explore vast chemical spaces. However, the limited number of available experimental datasets hinders the enhancement of the predictive performance of a model. This study proposes a machine learning approach that leverages transfer learning and ensemble modeling to efficiently predict the glass transition temperature (Tg) of fluorinated polymers and guide the design of high Tg copolymers. Initially, the quantum machine 9 (QM9) dataset is employed for model pretraining, thus providing robust molecular representations for the subsequent fine-tuning of a specialized copolymer dataset. Ensemble modeling is used to further enhance prediction robustness and reliability, effectively addressing the problems owing to the limited and unevenly distributed nature of the copolymer dataset. Finally, a fine-tuned ensemble model is used to navigate a vast chemical space comprising 61 monomers and identify promising candidates for high Tg fluorinated polymers. The model predicts 247 entries capable of achieving a Tg over 390 K, of which 14 are experimentally validated. This study demonstrates the potential of machine learning in material design and discovery, highlighting the effectiveness of transfer learning and ensemble modeling strategies for overcoming the challenges posed by small datasets in complex copolymer systems.

Learning Modality Consistency and Difference Information with Multitask Learning for Multimodal Sentiment Analysis

The primary challenge in Multimodal sentiment analysis (MSA) lies in developing robust joint representations that can effectively learn mutual information from diverse modalities. Previous research in this field tends to rely on feature concatenation to obtain joint representations. However, these approaches fail to fully exploit interactive patterns to ensure consistency and differentiation across different modalities. To address this limitation, we propose a novel framework for multimodal sentiment analysis, named CDML (Consistency and Difference using a Multitask Learning network). Specifically, CDML uses an attention mechanism to assign the attention weights of each modality efficiently. Adversarial training is used to obtain consistent information between modalities. Finally, the difference among the modalities is acquired by the multitask learning framework. Experiments on two benchmark MSA datasets, CMU-MOSI and CMU-MOSEI, showcase that our proposed method outperforms the seven existing approaches by at least 1.3% for Acc-2 and 1.7% for F1.

The influence of story character realism and theme on protagonists’ internal states and dialogue in children’s retells

Anthropomorphised animals are a prevalent character type in children’s books. However, stories with fantastical protagonists are associated with poorer social learning than those with human protagonists. We explored whether children’s representations of characters’ internal states and dialogue in story retells were related to story character realism (anthropomorphised animal, human), story theme (sharing, busyness), age, and vocabulary. Three- to seven-year-olds (N = 171) listened to one of four versions of an illustrated storybook that manipulated character realism and theme. Developmental trends were evident: Older children included more internal state references and dialogue in their retells than younger children. Of note, children retelling a prosocial story with human protagonists included more socio-relational language than those retelling a busy-themed story featuring humans; the same advantage was not evident for stories with animal protagonists. These findings imply that realistic protagonists may elicit more robust representations of social ideas in the minds of young children.

Exploring the potential of representation and transfer learning for anatomical neuroimaging: Application to psychiatry

The perspective of personalized medicine for brain disorders requires efficient learning models for anatomical neuroimaging-based prediction of clinical conditions. There is now a consensus on the benefit of deep learning (DL) in addressing many medical imaging tasks, such as image segmentation. However, for single-subject prediction problems, recent studies yielded contradictory results when comparing DL with Standard Machine Learning (SML) on top of classical feature extraction. Most existing comparative studies were limited in predicting phenotypes of little clinical interest, such as sex and age, and using a single dataset. Moreover, they conducted a limited analysis of the employed image pre-processing and feature selection strategies. This paper extensively compares DL and SML prediction capacity on five multi-site problems, including three increasingly complex clinical applications in psychiatry namely schizophrenia, bipolar disorder, and Autism Spectrum Disorder (ASD) diagnosis. To compensate for the relative scarcity of neuroimaging data on these clinical datasets, we also evaluate three pre-training strategies for transfer learning from brain imaging of the general healthy population: self-supervised learning, generative modeling and supervised learning with age. Overall, we find similar performance between randomly initialized DL and SML for the three clinical tasks and a similar scaling trend for sex prediction. This was replicated on an external dataset. We also show highly correlated discriminative brain regions between DL and linear ML models in all problems. Nonetheless, we demonstrate that self-supervised pre-training on large-scale healthy population imaging datasets (N≈10k), along with Deep Ensemble, allows DL to learn robust and transferable representations to smaller-scale clinical datasets (N≤1k). It largely outperforms SML on 2 out of 3 clinical tasks both in internal and external test sets. These findings suggest that the improvement of DL over SML in anatomical neuroimaging mainly comes from its capacity to learn meaningful and useful abstract representations of the brain anatomy, and it sheds light on the potential of transfer learning for personalized medicine in psychiatry

Neural models for semantic analysis of handwritten document images

Semantic analysis of handwritten document images offers a wide range of practical application scenarios. A sequential combination of handwritten text recognition (HTR) and a task-specific natural language processing system offers an intuitive solution in this domain. However, this HTR-based approach suffers from the problem of error propagation. An HTR-free model, which avoids explicit text recognition and solves the task end-to-end, tackles this problem, but often produces poor results. A possible reason for this is that it does not incorporate largely pre-trained semantic word embeddings, which turn out to be one of the most powerful advantages in the textual domain. In this work, we propose an HTR-based and an HTR-free model and compare them on a variety of segmentation-based handwritten document image benchmarks including semantic word spotting, named entity recognition, and question answering. Furthermore, we propose a cross-modal knowledge distillation approach to integrate semantic knowledge from textually pre-trained word embeddings into HTR-free models. In a series of experiments, we investigate optimization strategies for robust semantic word image representation. We show that the incorporation of semantic knowledge is beneficial for HTR-free approaches in achieving state-of-the-art results on a variety of benchmarks.

Performance evaluation of attention-deep hashing based medical image retrieval in brain MRI datasets

BackgroundBrain MRI images pose significant challenges due to their complexity and voluminous data, which often hinder the accuracy of traditional image retrieval methods. In response, this research delves into a novel medical image retrieval approach grounded in attention mechanisms and deep hashing techniques. MethodologyThe study enhances the network's edge perception capability by incorporating a dual mixed attention module (CBAM-MA dual attention) into the convolutional neural network architecture. This addition optimally captures edge saliency and distributional features by introducing an extra attention layer atop CBAM. Furthermore, a novel triple loss function is introduced, amalgamating Euclidean and cosine distances. This fusion exploits the strengths of both distance metrics to comprehensively consider sample geometric attributes, thereby generating more robust and expressive feature representations crucial for preserving categorical details. ResultsExperimental evaluations showcase significant performance advancements in MRI image retrieval tasks using the proposed method. The average hit rate stands at 0.7783, average precision at 0.752246, and average inverse rank at 0.956721. These metrics collectively underscore the method's efficacy in enhancing the quality and expediting the diagnosis of brain diseases. ConclusionThe research underscores the critical role of attention and deep hashing techniques in augmenting the accuracy and efficiency of medical image retrieval, particularly in the context of brain MRI images. The proposed method's notable performance improvements pave the way for enhanced diagnostic capabilities, signaling a significant step forward in improving healthcare outcomes related to brain diseases.