Meaningful Representation Research Articles

Discovering fully semantic representations via centroid- and orientation-aware feature learning

Abstract Learning meaningful representations of images in scientific domains that are robust to variations in centroids and orientations remains an important challenge. Here we introduce centroid- and orientation-aware disentangling autoencoder (CODAE), an encoder–decoder-based neural network that learns meaningful content of objects in a latent space. Specifically, a combination of a translation- and rotation-equivariant encoder, Euler encoding and an image moment loss enables CODAE to extract features invariant to positions and orientations of objects of interest from randomly translated and rotated images. We evaluate this approach on several publicly available scientific datasets, including protein images from life sciences, four-dimensional scanning transmission electron microscopy data from material science and galaxy images from astronomy. The evaluation shows that CODAE learns centroids, orientations and their invariant features and outputs, as well as aligned reconstructions and the exact view reconstructions of the input images with high quality.

Only the (informationally) stronger survive: A probe recognition study with scale-mates and antonyms

Speakers often use scalar words such as warm in a pragmatically strengthened way that results in the conveyed meaning being warm but not hot. In these inferences, known as scalar implicatures, meaning alternatives have been postulated to play a crucial role. Upon encountering warm, the informationally stronger alternative hot has been shown to be active in online sentence processing. Antonyms (cool) have also been shown to be activated in the same way even though they are standardly assumed to not be involved in scalar implicature derivation. In the current study, we focus on the question of whether both strong scalar alternatives and antonyms are represented in the final mental model of the discourse following scalar implicature derivation. We ran two probe recognition experiments, testing strong scalars and antonyms. We found an interference effect for strong scalars, indicating their representation, but not one for antonyms. Thus, we provide evidence that only the strong scalars survive in the eventual representation of the pragmatic meaning of a sentence.

Contrastive-learning of language embedding and biological features for cross modality encoding and effector prediction

Identifying and characterizing virulence proteins secreted by Gram-negative bacteria are fundamental for deciphering microbial pathogenicity as well as aiding the development of therapeutic strategies. Effector predictors utilizing pre-trained protein language models (PLMs) have shown sound performance by leveraging extensive evolutionary and sequential protein features. However, the accuracy and sensitivity of effector prediction remain challenging. Here, we introduce a model named Contrastive-learning of Language Embedding and Biological Features (CLEF) leveraging contrastive learning to integrate PLM representations with supplementary biological features. Biologically information is captured in learned contextualized embeddings to yield meaningful representations. With cross-modality biological features, CLEF outperforms state-of-the-art (SOTA) models in predicting type III, type IV, and type VI secreted effectors (T3SEs/T4SEs/T6SEs) in enteric pathogens. All experimentally verified effectors in Enterohemorrhagic Escherichia coli and 41 of 43 experimentally verified T3SEs of Salmonella Typhimurium are recognized. Moreover, 12 predicted T3SEs and 11 predicted T6SEs are validated by extensive experiments in Edwardsiella piscicida. Furthermore, integrating omics data via CLEF framework enhances protein representations to illustrate effector-effector interactions and determine in vivo colonization-essential genes. Collectively, CLEF provides a blueprint to bridge the gap between in silico PLM’s capacity and experimental biological information to fulfill complicated tasks.

Deep learning and explainable AI for classification of potato leaf diseases

The accurate classification of potato leaf diseases plays a pivotal role in ensuring the health and productivity of crops. This study presents a unified approach for addressing this challenge by leveraging the power of Explainable AI (XAI) and transfer learning within a deep Learning framework. In this research, we propose a transfer learning-based deep learning model that is tailored for potato leaf disease classification. Transfer learning enables the model to benefit from pre-trained neural network architectures and weights, enhancing its ability to learn meaningful representations from limited labeled data. Additionally, Explainable AI techniques are integrated into the model to provide interpretable insights into its decision-making process, contributing to its transparency and usability. We used a publicly available potato leaf disease dataset to train the model. The results obtained are 97% for validation accuracy and 98% for testing accuracy. This study applies gradient-weighted class activation mapping (Grad-CAM) to enhance model interpretability. This interpretability is vital for improving predictive performance, fostering trust, and ensuring seamless integration into agricultural practices.

Classification of poetry text into the emotional state using Deep Learning

Poetry can elicit powerful emotions on the basis of elevated language and literary devices, and placing these emotions becomes problematic owing to the subjectivity of human emotions and the complexity entwined in poetic expression. This research puts forward a deep learning-based approach towards automatic classification of poetry into emotions like happiness, sadness, anger, fear, and surprise. The proposed approach also engages advanced NLP techniques that aid in discovering the semantics and emotional hinterlands of poems. For the purpose of this study, we represent poetic texts in higher-dimensional space using word embeddings such as Word2Vec and GloVe as reliable representations of contextual meanings. Recurrent Neural Networks (RNNs), particularly Long Short-Term Memory (LSTM) networks, have been employed to cater to the sequential nature of poems, thus recognizing complex patterns of emotions. Furthermore, we explore Transformer-based models such as BERT because of their superior power to understand context and nuances of emotion in text. The dataset was a curated collection of poems with annotations for emotions. Then, to ensure robust results, the performance of the models was evaluated in terms of accuracy, precision, recall, and F1-score. Initial results showed promising accuracy for such emotional identification techniques, emphasizing the aptitude of the proposed deep learning techniques to understand and elucidate poetry's emotional content. The offered study adds a new light on the conjunction of computational linguistics and emotional analysis, bettering the understanding of automated literary analysis. Keywords: Poetry, Emotion Analytics, Sentiment Analytics, Creative Writing, Listening AI, Natural Language Processing, Textual Analysis, Expressive AI, Mood Poetry, Emotional Intelligence, Machine Training.

A comparative study of statistical, radiomics, and deep learning feature extraction techniques for medical image classification in optical and radiological modalities.

Feature extraction in ML plays a crucial role in transforming raw data into a more meaningful and interpretable representation. In this study, we thoroughly examined a range of feature extraction techniques and assessed their impact on the binary classification models for medical images, utilizing a diverse and rich set of medical imaging modalities. Using H&E-stained, chest X-ray, and retina OCT images, we applied methods to extract statistical, radiomics, and deep features. These features were then used to develop PCA-LDA models as the employed classifier. We evaluated the models based on two decisive metrics: latency and performance. Latency measured the time taken for feature extraction and prediction, while mean sensitivity (balanced accuracy) characterizes the model performance. Our comparative study revealed that statistical and radiomics features were less effective for medical image classification, as they showed high latency and lower performance scores. In contrast, pre-trained DL networks performed efficiently, with high sensitivity and low latency. For H&E-stained images, the statistical feature extraction took about an hour and achieved 90.8% sensitivity, while ResNet50 reduced processing time fourfold and increased sensitivity to 96.9%. For chest X-rays, radiomics features were time-intensive with 92.2% sensitivity, while ResNet50 improved sensitivity to 96% with faster extraction time. For retina OCT images, radiomics yielded a sensitivity of 91%, while DenseNet121 achieved 98.6% sensitivity in 15min. These findings underscore the superior performance of DL techniques over the statistical and radiomics features, highlighting their potential for real-world applications where accurate and rapid diagnostic decisions are essential.

MMnc: Multi-modal interpretable representation for non-coding RNA classification and class annotation.

As the biological roles and disease implications of non-coding RNAs continue to emerge, the need to thoroughly characterize previously unexplored non-coding RNAs becomes increasingly urgent. These molecules hold potential as biomarkers and therapeutic targets. However, the vast and complex nature of non-coding RNAs data presents a challenge. We introduce MMnc, an interpretable deep learning approach designed to classify non-coding RNAs into functional groups. MMnc leverages multiple data sources-such as the sequence, secondary structure, and expression-using attention-based multi-modal data integration. This ensures learning of meaningful representations while accounting for missing sources in some samples. Our findings demonstrate that MMnc achieves high classification accuracy across diverse non-coding RNA classes. The method's modular architecture allows for the consideration of multiple types of modalities, whereas other tools only consider one or two at most. MMnc is resilient to missing data, ensuring that all available information is effectively utilized. Importantly, the generated attention scores offer interpretable insights into the underlying patterns of the different non-coding RNA classes, potentially driving future non-coding RNA research and applications. Data and source code can be found at EvryRNA.ibisc.univ-evry.fr/EvryRNA/MMnc. Supplementary data are available at Bioinformatics online.

(Semi-)automatic Extraction of Urban Planning Rules in French for Better Management of Land Artificialization

Land artificialization is a significant modern concern, as it is irreversible, diminishes agriculturally suitable land and causes environmental problems. Our project, Hérelles, aims to address this challenge by developing a framework for land artificialization management. In this framework, we associate urban planning rules in text form with clusters extracted from time series of satellite images. To achieve this, it is crucial to understand the planning rules with two key objectives: (1) to verify if the constraints derived from the rules are verifiable on satellite images and (2) to use these constraints to guide the labelling (or semantization) of clusters. The first step in this process involves the automatic extraction of rules from urban planning documents written in the French language. To solve this problem, we propose a method based on the multilabel classification of textual segments and their subsequent summarization. This method includes a special format for representing segments, in which each segment has a title and a subtitle. We then propose a cascade approach to address the hierarchy of class labels. Additionally, we develop several text augmentation techniques for French texts that can improve prediction results. Finally, we reformulate classified segments into concise text portions containing necessary elements for expert rule construction. We adapt an approach based on Meaning Representation (AMR) graphs to generate these portions in the French language and conduct a comparative analysis with ChatGPT. We experimentally demonstrate that the resulting framework correctly classifies each type of segment with more than 90% accuracy. Furthermore, our results indicate that ChatGPT outperforms the AMR-based approach, leading to a discussion of the advantages and limitations of both methods.

Real-time deformable structure tracking in 3D ultrasound sequences using deformable convolutional layers.

Ultrasound imaging can provide 3D images of soft tissue structures in real-time without harmful radiation. Due to its high level of availability and low-cost characteristics, it is becoming more and more interesting for therapy guidance purposes like in radiotherapy. However, for usage in radiotherapy a robust and real-time image analysis method is required to be able to track the target during the treatment session. Soft tissue structures move in high dimensional motion patterns, including deformation especially due to breathing which makes the tracking task challenging. To overcome the deformation complexity, a novel real-time capable tracking approach in 3D ultrasound is proposed in this study. Deformable convolution layers are included in a 2D convolutional autoencoder architecture to learn deformation-invariant representations from ultrasound patches. For this, a novel 3D to 2D ultrasound patch reduction strategy is proposed. The tracking procedure is performed in the representation space of the autoencoder. We therefore implemented a greedy local search tracking algorithm and evaluated it in a preliminary study on the basis of nine expert-labeled landmarks in 18 in-vivo 3D ultrasound liver sequences. Four different autoencoder architectures with different deformable convolution arrangements are compared in the evaluation. The results show that using deformable convolution layers is beneficial compared to conventional convolution layers. A mean tracking error of 1.58 ± 0.87 mm was measured using deformable convolution, which is an improvement by 10.7% compared to conventional convolution. Additionally, evaluating the runtime shows that the tracking algorithm is real-time capable as the mean runtime per 3D ultrasound frame was around 4 ms. It could be shown that using deformable convolution layers is beneficial for learning meaningful representations of deformable structures from ultrasound patches. A tracking error similar to state-of-the-art methods could be achieved but in a runtime up to 100 times faster.

A Nested GLM Framework with Neural Network Encoding and Spatially Constrained Clustering in Non-Life Insurance Ratemaking

The generalized linear model (GLM) is a popular modeling choice for pricing non-life insurance policies. However, high-cardinality categorical insurance data presents significant challenges for these GLM rate-making models. Additionally, insurance regulators often require rating territories, which are clusters of insurance policies’ geographic locations for setting insurance rates, to meet certain standards. For instance, (1) the credibility standard ensures that the number of policies in a territory is large enough to be credible and representative, (2) the contiguity standard requires the locations in each territory to be geographically adjacent to promote a logical and practical spatial grouping, and (3) the cardinality standard specifies an acceptable range for the number of territories in a geographic area. To address these challenges, this article proposes a nested GLM framework for non-life insurance rate-making applications. In this framework, neural network models with categorical embedding layers are constructed to model the residual deviance from simple GLMs, using high-cardinality categorical variables as input. Low-dimensionly features extracted from the neural network model effectively translate categorical variables into meaningful numerical representations, capturing their effects on the initial model’s residuals. The features corresponding to the location-related variable are further converted into a contiguous territory rating variable via spatially constrained clustering models. By incorporating outcomes from these models, the nested GLM not only satisfies regulatory requirements but also enhances the model’s predictive power, while maintaining the interpretability from the (generalized) linear form. The construction of a nested Poisson GLM is presented in this article. Its performance is demonstrated using a real-life Brazil auto insurance data to model claim frequency.

Mapping conformational landscape in protein folding: Benchmarking dimensionality reduction and clustering techniques on the Trp-Cage mini-protein.

Quantitative characterization of protein conformational landscapes is a computationally challenging task due to their high dimensionality and inherent complexity. In this study, we systematically benchmark several widely used dimensionality reduction and clustering methods to analyze the conformational states of the Trp-Cage mini-protein, a model system with well-documented folding dynamics. Dimensionality reduction techniques, including Principal Component Analysis (PCA), Time-lagged Independent Component Analysis (TICA), and Variational Autoencoders (VAE), were employed to project the high-dimensional free energy landscape onto 2D spaces for visualization. Additionally, clustering methods such as K-means, hierarchical clustering, HDBSCAN, and Gaussian Mixture Models (GMM) were used to identify discrete conformational states directly in the high-dimensional space. Our findings reveal that density-based clustering approaches, particularly HDBSCAN, provide physically meaningful representations of free energy minima. While highlighting the strengths and limitations of each method, our study underscores that no single technique is universally optimal for capturing the complex folding pathways, emphasizing the necessity for careful selection and interpretation of computational methods in biomolecular simulations. These insights will contribute to refining the available tools for analyzing protein conformational landscapes, enabling a deeper understanding of folding mechanisms and intermediate states.

Dislocation cartography: Representations and unsupervised classification of dislocation networks with unique fingerprints

Detecting structure in data is the first step to arrive at meaningful representations for systems. This is particularly challenging for evolving dislocation networks evolving as a consequence of plastic deformation of crystalline materials. Our study employs Isomap, a manifold learning technique, to show the intrinsic structure of high-dimensional dislocation density field data of dislocation structures resulting from different compression axes. Our maps provide a systematic framework for quantitatively comparing dislocation structures and offer unique fingerprints based on dislocation density fields. It represents a novel, unbiased approach that contributes to the quantitative classification of dislocation structures, which can be systematically extended using different representations of dislocation systems.

Evaluating the effectiveness of prompt engineering for knowledge graph question answering.

Many different methods for prompting large language models have been developed since the emergence of OpenAI's ChatGPT in November 2022. In this work, we evaluate six different few-shot prompting methods. The first set of experiments evaluates three frameworks that focus on the quantity or type of shots in a prompt: a baseline method with a simple prompt and a small number of shots, random few-shot prompting with 10, 20, and 30 shots, and similarity-based few-shot prompting. The second set of experiments target optimizing the prompt or enhancing shots through Large Language Model (LLM)-generated explanations, using three prompting frameworks: Explain then Translate, Question Decomposition Meaning Representation, and Optimization by Prompting. We evaluate these six prompting methods on the newly created Spider4SPARQL benchmark, as it is the most complex SPARQL-based Knowledge Graph Question Answering (KGQA) benchmark to date. Across the various prompting frameworks used, the commercial model is unable to achieve a score over 51%, indicating that KGQA, especially for complex queries, with multiple hops, set operations and filters remains a challenging task for LLMs. Our experiments find that the most successful prompting framework for KGQA is a simple prompt combined with an ontology and five random shots.

Characterization of Imagination Patterns in Students' Drawings Class 1 SDIT ALIF

Background: Learning art plays an important role in developing students' creative and imaginative abilities. Through art, students not only learn about techniques and aesthetics but also express their ideas, feelings, and imagination. Art education is realized in the form of visual arts, which can be enjoyed through the sense of sight. This enhances local cultural awareness, fosters appreciation for cultural values, and serves as a medium for creativity development and the generation of new ideas. Art also enables self-actualization and the cultivation of creativity. Imagination, often referred to as the ability to visualize, is the capacity to form new images or sensations that are not directly experienced through sight, hearing, or other senses. Among elementary school students, drawing is one of the most dominant dimensions of imagination. Through drawing, students engage in activities such as scribbling, sketching, and coloring objects, which results in meaningful visual representations. Purpose: This article aims to explore the characterization of imaginative themes that emerge in drawings created by students. Design and methods: The method used in this study is the Narrative Method with a qualitative descriptive approach. The Narrative Method, as applied here, follows Polkinghorne's model. Narrative research aims to explore and understand individual experiences through storytelling. Polkinghorne’s model provides an analytical framework consisting of three key components: theme, context (domain), and cognitive structure. Data collection techniques used in this study include interviews and documentation. This research focuses on identifying the characterization of imaginative themes that appear in the drawings of first-grade students at SDIT ALIF. Results: The findings indicate that the principle of proximality is implemented by grouping nearby objects into patterns. This reflects a perceptual tendency wherein humans perceive closely positioned objects as part of a larger group or relationship. The principle of continuity suggests that individuals tend to view lines and patterns as unified or continuous entities. In the context of imagination, this means that people often form mental images of ideas or concepts in a continuous manner. For example, when imagining an object or scene, individuals tend to fill in the missing or unclear parts in their minds to create a completer and more coherent picture. Based on the discussion, the characterization of imaginative patterns in the drawings of first-grade students involves the senses of sight, hearing, touch, and taste.

ICoN: integration using co-attention across biological networks.

Molecular interaction networks are powerful tools for studying cellular functions. Integrating diverse types of networks enhances performance in downstream tasks such as gene module detection and protein function prediction. The challenge lies in extracting meaningful protein feature representations due to varying levels of sparsity and noise across these heterogeneous networks. We propose ICoN, a novel unsupervised graph neural network model that takes multiple protein-protein association networks as inputs and generates a feature representation for each protein that integrates the topological information from all the networks. A key contribution of ICoN is exploiting a mechanism called "co-attention" that enables cross-network communication during training. The model also incorporates a denoising training technique, introducing perturbations to each input network and training the model to reconstruct the original network from its corrupted version. Our experimental results demonstrate that ICoN surpasses individual networks across three downstream tasks: gene module detection, gene coannotation prediction, and protein function prediction. Compared to existing unsupervised network integration models, ICoN exhibits superior performance across the majority of downstream tasks and shows enhanced robustness against noise. This work introduces a promising approach for effectively integrating diverse protein-protein association networks, aiming to achieve a biologically meaningful representation of proteins. The ICoN software is available under the GNU Public License v3 at https://github.com/Murali-group/ICoN.

Infinite Mixture Chaining: An Efficiency-Based Framework for the Dynamic Construction of Word Meaning.

The lexicon is an evolving symbolic system that expresses an unbounded set of emerging meanings with a limited vocabulary. As a result, words often extend to new meanings. Decades of research have suggested that word meaning extension is non-arbitrary, and recent work formalizes this process as cognitive models of semantic chaining whereby emerging meanings link to existing ones that are semantically close. Existing approaches have typically focused on a dichotomous formulation of chaining, couched in the exemplar or prototype theories of categorization. However, these accounts yield either memory-intensive or simplistic representations of meaning, while evidence for them is mixed. We propose a unified probabilistic framework, infinite mixture chaining, that derives different forms of chaining through the lens of cognitive efficiency. This framework subsumes the existing chaining models as a trade-off between representational accuracy and memory complexity, and it contributes a flexible class of models that supports the dynamic construction of word meaning by automatically forming semantic clusters informed by existing and novel usages. We demonstrate the effectiveness of this framework in reconstructing the historical development of the lexicon across multiple word classes and in different languages, and we also show that it correlates with human judgment of semantic change. Our study offers an efficiency-based view on the cognitive mechanisms of word meaning extension in the evolution of the lexicon.

Investigating Mechanisms Driving Differences in the Characteristics of Precipitation in the E3SM Multiscale Modeling Framework With 2D Versus 3D Cloud Resolving Model Configurations

AbstractIn this study, we compare the Energy Exascale Earth Systems Model (E3SM) multiscale modeling framework (MMF) with the cloud resolving model (CRM) configured in two (2dMMF) and three (3dMMF) dimensions. We explore how CRM dimensionality impacts the representation of mean and extreme precipitation characteristics. Our results show that tropical mean precipitation patterns are better represented in 3dMMF compared to observations (Integrated Multi‐satellitE Retrivals for GPM and Global Precipitation Climatology Project One Degree Daily products), while 2dMMF better captures extreme precipitation intensity, with systematic land‐ocean differences in precipitation and cloud‐associated variables. These differences are attributed to the co‐occurrence of CRM throttling (i.e., suppressed convection in due to smaller numbers of CRM columns and domain size) and dilution (i.e., 3‐D cloud circulations with increased entrainment and lower precipitation efficiency) effects. Overall, throttling results in more low‐level humidity in 2dMMF and dilution contributes to more high clouds with less precipitation efficiency in 3dMMF. Since throttling occurs more strongly over the ocean than land, the 3dMMF tends to have less cloud liquid and precipitation over the ocean and more cloud ice and precipitation over land. These results may serve as a guide for choosing the CRM structure to reduce precipitation and cloud‐related biases.

Delineating the effective use of self-supervised learning in single-cell genomics

Self-supervised learning (SSL) has emerged as a powerful method for extracting meaningful representations from vast, unlabelled datasets, transforming computer vision and natural language processing. In single-cell genomics (SCG), representation learning offers insights into the complex biological data, especially with emerging foundation models. However, identifying scenarios in SCG where SSL outperforms traditional learning methods remains a nuanced challenge. Furthermore, selecting the most effective pretext tasks within the SSL framework for SCG is a critical yet unresolved question. Here we address this gap by adapting and benchmarking SSL methods in SCG, including masked autoencoders with multiple masking strategies and contrastive learning methods. Models trained on over 20 million cells were examined across multiple downstream tasks, including cell-type prediction, gene-expression reconstruction, cross-modality prediction and data integration. Our empirical analyses underscore the nuanced role of SSL, namely, in transfer learning scenarios leveraging auxiliary data or analysing unseen datasets. Masked autoencoders excel over contrastive methods in SCG, diverging from computer vision trends. Moreover, our findings reveal the notable capabilities of SSL in zero-shot settings and its potential in cross-modality prediction and data integration. In summary, we study SSL methods in SCG on fully connected networks and benchmark their utility across key representation learning scenarios.

A systematic exploration of collaborative immersive systems for sense-making in STEM

ABSTRACT Scientific sense-making in STEM fields is a complex, yet essential activity, that greatly benefits from collaborations. However, challenges associated with collaboration, such as the geographic separation of experts, access to specialised equipment, and meaningful data representation, often hinder this process. Solutions to collaborative challenges have been extensively explored in CSCW and HCI literature. Among such solutions, immersive systems offer novel data visualisations, interactions, and representations that can support collaborative sense-making in STEM fields. Recognising the increasing interest from HCI researchers on the intersection of collaboration and immersive systems, we conduct a systematic review to answer pertinent questions regarding the research landscape, the design and implementation of collaborative immersive systems for STEM sense-making. We find that current research leans towards synchronous collaborations, AR technology, and sense-making for learning in science domains. We further discuss prevalent trends and considerations observed in our findings, to inform future research directions.

Graph analysis of guilt processing network highlights links with subclinical anxiety and self-blame.

Maladaptive forms of guilt, such as excessive self-blame, are common characteristics of anxiety and depressive disorders. The underlying network consists of multiple associative areas, including the superior anterior temporal lobe (sATL), underlying the conceptual representations of social meaning, and fronto-subcortical areas involved in the affective dimension of guilt. Nevertheless, despite understanding the circuitry's anatomy, network-level changes related to subclinical anxiety and self-blaming behaviour have not been depicted. To fill this gap, we used graph theory analyses on a resting-state functional and diffusion-weighted magnetic resonance imaging dataset of 78 healthy adults (20 females, 20-35 years old). Within the guilt network, we found increased functional contributions of the left sATL for individuals with higher self-blaming, while functional isolation of the left pars opercularis and insula was related to higher trait anxiety. Trait anxiety was also linked to the structural network's mean clustering coefficient, with the circuitry's architecture favouring increased local information processing in individuals with increased anxiety levels, however, only when a highly specific subset of connections was considered. Previous research suggests that aberrant interactions between conceptual (sATL) and affective (fronto-limbic) regions underlie maladaptive guilt, and the current results align and expand on this theory by detailing network changes associated with self-blame and trait anxiety.