Text Representation Research Articles

GGAS2SN: Gated Graph and SmilesToSeq Network for Solubility Prediction.

Aqueous solubility is a critical physicochemical property of drug discovery. Solubility is a key issue in pharmaceutical development because it can limit a drug's absorption capacity. Accurate solubility prediction is crucial for pharmacological, environmental, and drug development studies. This research introduces a novel method for solubility prediction by combining gated graph neural networks (GGNNs) and graph attention neural networks (GATs) with Smiles2Seq encoding. Our methodology involves converting chemical compounds into graph structures with nodes representing atoms and edges indicating chemical bonds. These graphs are then processed by using a specialized graph neural network (GNN) architecture. Incorporating attention mechanisms into GNN allows for capturing subtle structural dependencies, fostering improved solubility predictions. Furthermore, we utilized the Smiles2Seq encoding technique to bridge the semantic gap between molecular structures and their textual representations. Smiles2Seq seamlessly converts chemical notations into numeric sequences, facilitating the efficient transfer of information into our model. We demonstrate the efficacy of our approach through comprehensive experiments on benchmark solubility data sets, showcasing superior predictive performance compared to traditional methods. Our model outperforms existing solubility prediction models and provides interpretable insights into the molecular features driving solubility behavior. This research signifies an important advancement in solubility prediction, offering potent tools for drug discovery, formulation development, and environmental assessments. The fusion of GGNN and Smiles2Seq encoding establishes a robust framework for accurately forecasting solubility across various chemical compounds, fostering innovation in various domains reliant on solubility data.

Exploring Recent NLP Advances for Tamil: Word Vectors and Hybrid Deep Learning Architectures

The advancements of deep learning methods and the availability of large corpora and data sets have led to an exponential increase in the performance of Natural Language Processing (NLP) methods resulting in successful NLP applications for various day-to-day tasks such as Language translation, Voice to text, Grammar checking, Sentiment analysis, etc. These advancements enabled the well-resourced languages to adapt themselves to the digital era while the gap for the low-resource languages widened. This research work explores the suitability of the recent advancements in NLP for Tamil, a low-resource language spoken mainly in South India, Sri Lanka, and Malaysia. From the literature survey, it has been found that there is a lack of comprehensive study on the effect of the recent advancements of NLP for the Tamil text. To fill this gap, this research work analysed the performance of deep learning based text representation and classification approaches namely word embedding, Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN) for Tamil text classification tasks. Different dimensional pretrained word2Vec and FastText word vectors were built for Tamil and their effectiveness on Text classification was evaluated. The study found that the pre-trained 300- dimensional FastText word vector showed better performance than other pre-trained word vectors for Tamil text classification. Further, in this study, four simple hybrid CNN and Bi-GRU models were proposed for Tamil text classification and their performances were evaluated. The study found that hybrid CNN and Bi-GRU models perform better compared to the classical machine learning models, individual CNN and RNN models, and the Multilingual BERT model. These results confirm that the jointly learned embeddings with different deep learning architectures like CNN and RNN can achieve remarkable results for Tamil text classification, thus ensuring that the deep learning approaches can be successful for NLP on Tamil text.

Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

Understanding spoken language is crucial for conversational agents, with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally, these tasks have been addressed individually, but recent studies have highlighted their interconnection, suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However, the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally, traditional models face difficulties in capturing both the semantic and syntactic nuances of language, which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information, part-of-speech (POS) tag embeddings for syntactic features, and contextual embeddings for deeper contextual understanding. Specifically, we utilized word2vec for non-contextual embeddings, one-hot vectors for POS tags, and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network, followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models, achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset, and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems, and paving the way for more accurate and efficient natural language understanding in real-world applications.

Social Event Classification Based on Multimodal Masked Transformer Network

The key to multimodal social event classification is to fully and accurately utilize the features of both image and text modalities. However, most existing methods have the following limitations: (1) they simply concatenate the image features and text features of the event, and (2) there is irrelevant contextual information between different modalities, which leads to mutual interference. Therefore, it is not enough to only consider the relationship between the modalities of multimodal data, but also the irrelevant contextual information (i.e., regions or words) between the modalities. To overcome these limitations, a novel social event classification method based on multimodal masked transformer network (MMTN) is proposed. A better representation of text and image is learned through an image-text encoding network. Then, the obtained image and text representations are input into the multimodal masked transformer network to fuse the multimodal information, and the relationship between the modalities of multimodal information is modeled by calculating the similarity between the multimodal information, masking the irrelevant context between the modalities. Extensive experiments on two benchmark datasets show that the proposed multimodal masked transformer network model achieves state-of-the-art performance.

Multi-Modal Contextualization of Trajectory Data for Advanced Analysis

SummaryRising amounts of generated geospatial data, either trajectory-like tracking data, raster-like imagery, or vector-like mappings as in OpenStreetMap (OSM), grow the need for multi-modal algorithmic analysis. Existing machine-learning-based algorithms contradictly mainly focus on image and textual input representations and cannot deal with other modes of geospatial data. Therefore, we propose a novel method to contextualize vector-like trajectory data with surrounding data to create easy-to-be-analyzed image-like representations. Our approach includes the proposition of a chase-cam-like scanline over space according to the trajectory’s speed and possibly smoothed orientation. Thereby, surrounding pixels in the vicinity of the trajectory points are accumulated along the scanline and are combined into a visual representation of the trajectory. To show the potential effects of our work, we predict traffic regulations for trajectory sections in the vehicle speed dataset based on our proposed trajectory-based sampling of orthophotos in the same region. This proposes a new way of using multi-modal data sources (trajectories and airborne imagery) to extract road metadata.

La caricatura política como signo visual durante el primer trimestre del gobierno del presidente Daniel Noboa.

This study analyzes the political cartoons of Xavier Bonilla (Bonil) and Vilma Vargas (Vilmatraca) in Ecuador during the beginning of Daniel Noboa's government (November 23, 2023 - February 25, 2024), focusing on five critical themes: Inauguration of Power, Popular Consultation, Drug Trafficking, VAT Increase, and Diplomatic Crisis with Russia. It explores the visual and textual representations in the intentionally selected works, analyzing drawing techniques, use of colors, facial expressions, and cultural references. This study shows how Bonil and Vilmatraca use signs and symbols to critique and comment on the political and social situation, demonstrating semiotics as an essential tool for understanding visual communication in politics. The study highlights the role of cartoons in provoking reflection on the Ecuadorian reality.

Leveraging Contrastive Language–Image Pre-Training and Bidirectional Cross-attention for Multimodal Keyword Spotting

In resource-limited keyword spotting scenarios, the scarcity of annotated corpora hinders deep learning’s ability to develop robust models for representing acoustic features. Recent studies focus on contrastive learning, using paired examples for self-supervision, indicating a growing research interest in keyword spotting. Typically, models are trained on segmented spoken words. However, the absence of distinct word boundaries in lengthy audio poses significant challenges. This study presents a novel approach that integrates Contrastive Language–Image Pre-Training with Cross-Attention for Self-Supervised Alignment in the field of Multimodal Keyword Spotting. The proposed method introduces a cross-modal process for matching word pairs, enhancing collaborative effectiveness between audio and text embeddings, achieving word-level congruence across different modalities. Employing a self-supervised learning approach, we use a restricted number of annotated audio–text pairs to discern semantic congruities and divergences, resulting in improved multimodal feature representation. Using this strategy, we input into subsequent keyword detection tasks, adopting a Bidirectional Cross-Attention block for detailed coordination across modalities, bridging the semantic gap between diverse audio and text representations, consequently improving keyword localization precision. Comprehensive experiments conducted on the Aishell-2 and Librispeech datasets demonstrate that the proposed method markedly surpasses existing techniques, especially regarding the comprehensive performance metric F1 for keyword spotting. It attains state-of-the-art outcomes for both pristine and noisy speech, showing respective enhancements of 5.2% and 5.4% on Aishell-2, as well as 5.4% and 5.6% on Librispeech.

AtCAF: Attention-based causality-aware fusion network for multimodal sentiment analysis

Multimodal sentiment analysis (MSA) involves interpreting sentiment using various sensory data modalities. Traditional MSA models often overlook causality between modalities, resulting in spurious correlations and ineffective cross-modal attention. To address these limitations, we propose the Attention-based Causality-Aware Fusion (AtCAF) network from a causal perspective. To capture a causality-aware representation of text, we introduce the Causality-Aware Text Debiasing Module (CATDM) utilizing the front-door adjustment. Furthermore, we employ the Counterfactual Cross-modal Attention (CCoAt) module integrate causal information in modal fusion, thereby enhancing the quality of aggregation by incorporating more causality-aware cues. AtCAF achieves state-of-the-art performance across three datasets, demonstrating significant improvements in both standard and Out-Of-Distribution (OOD) settings. Specifically, AtCAF outperforms existing models with a 1.5% improvement in ACC-2 on the CMU-MOSI dataset, a 0.95% increase in ACC-7 on the CMU-MOSEI dataset under normal conditions, and a 1.47% enhancement under OOD conditions. CATDM improves category cohesion in feature space, while CCoAt accurately classifies ambiguous samples through context filtering. Overall, AtCAF offers a robust solution for social media sentiment analysis, delivering reliable insights by effectively addressing data imbalance. The code is available at https://github.com/TheShy-Dream/AtCAF.

Towards fine-grained adaptive video captioning via Quality-Aware Recurrent Feedback Network

To achieve accurate video captioning, most existing solutions focus on employing auxiliary features to provide complementary generation clues, such as detected target regions, audio speech signals, and in-plane text. However, beyond introducing external information, the potential of using the generated captions has been overlooked. We argue that the intrinsic value lies in the caption text, which can faithfully reflect the video content. To this end, we propose to reuse the captions via a Quality-Aware Recurrent Feedback Network (QARFNet), a model that progressively exploits the supportive power between video features and the generated captions. Specifically, inspired by the backtracking AdaBoost algorithm, we propose to build a recurrent loop structure that recycles the obtained linguistic predictions to refine the input visual features. Referring to the greedy selective nature of the AdaBoost algorithm, we set a quality-aware gate to assess the necessity of forwarding the next loop, lessening the complexity. After the refinement loop, if the confidence score remains below the predefined threshold, we select the output with the highest confidence score to advance the subsequent transformer decoder. To facilitate feedback in each loop, a Multi-level Update Module is constructed before fusing linguistic predictions with video features. This involves extracting nouns and verbs from linguistic predictions to lighten relevant video tokens, achieving self-interactions across recurrent loops. By merging the coarse-grained and gradually refined linguistic prediction features, the video tokens are semantically closer to the desired textual representation. To alleviate the increased calculation caused by the loops, we employ a flat transformer encoder to decrease the complexity. The experimental results on several benchmark datasets confirm the effectiveness of our approach, with superior performance compared to the state-of-the-art.

TS‐Mixer: A lightweight text representation model based on context awareness

AbstractLarge pre‐trained models (PTMs) have shown their powerful ability in multiple natural language processing tasks. However, using them in practical application remains a challenge due to the significant computational cost and memory requirements. In order to achieve the balance of computational cost and accuracy, MLP architecture can be used as an alternative to the self‐attention module, such as pNLP‐Mixer and Hyper‐Mixer. Experiments indicate that, MLP‐based models can attain competitive performance with low cost. They maintain the balance of computation cost and accuracy successfully, yet, this is at the expense of not being able to capture short‐range dependencies. In this paper, a novel MLP‐based model, termed TS‐Mixer, is proposed which can capture local dependencies by shifting operation. Compared with other MLP‐based models, the parameters of TS‐Mixer are decoupled from the sequence length, hence it has a smaller model size in long sequence tasks. In addition, TS‐Mixer has linear computational complexity, therefore it can be used as a lightweight alternative to the self‐attention model. Experiments show that the TS‐Mixer outperforms other MLP‐based models, which achieves higher accuracy with fewer parameters in multiple downstream tasks. Notably, compared with pre‐trained models, TS‐Mixer can reach more than 90% of their accuracy with 1% or even one thousandth of the parameters (0.174 ~ 1.2 M). These results demonstrate that TS‐Mixer can achieve a better balance between the computing resources and accuracy. Code is available at: https://github.com/wyl-privacy-project/TS-Mixer.

The Cognitive Map Mechanism in Digital Reading: An Experimental Study on the Influence of Textual Representation on Cognitive Effects

ABSTRACTThe Cognitive Map Mechanism explains that different textual representations influence cognitive effects by affecting how readers mentally organize and navigate information. The application of this mechanism in digital reading remains underexplored. We conducted a lab experiment(N = 148) manipulating low/medium/high textual representation through non−/micro−/combined‐annotation modes. Our aim was to clarify how the cognitive map based digital reading variations impact reading comprehension, performance calibration, and reading speed. Sufficient results showed that high‐level textual representation provides superior cognitive effects in all dimensions. It is interesting to discover that medium‐level textual representation impaired cognitive effects compared to low‐level representation. Specifically, metacognition negatively moderated the influence of textual representation level on reading speed, suggesting that texts with both micro‐annotations and graphic organizers can provide comprehensive cues and visual correlations to build cognitive map, thereby enhancing cognitive effects. Our study contributes novel insights for future research and practical implementation of cognitive‐oriented strategies in digital context.

Advancing MRI segmentation with CLIP-driven semi-supervised learning and semantic alignment

Precise segmentation and reconstruction of multi-structures within MRI are crucial for clinical applications such as surgical navigation. However, medical image segmentation faces several challenges. Although semi-supervised methods can reduce the annotation workload, they often suffer from limited robustness. To address this issue, this study proposes a novel CLIP-driven semi-supervised model, that includes two branches and a module. In the image branch, copy-paste is used as data augmentation method to enhance consistency learning. In the text branch, patient-level information is encoded via CLIP to drive the image branch. Notably, a novel cross-modal fusion module is designed to enhance the alignment and representation of text and image. Additionally, a semantic spatial alignment module is introduced to register segmentation results from different axial MRIs into a unified space. Three multi-modal datasets (one private and two public) were constructed to demonstrate the model’s performance. Compared to previous state-of-the-art methods, this model shows a significant advantage with both 5% and 10% labeled data. This study constructs a robust semi-supervised medical segmentation model, particularly effective in addressing label inconsistency and abnormal organ deformations. It also tackles the axial non-orthogonality challenges inherent in MRI, providing a consistent view of multi-structures.

Global-aware Fragment Representation Aggregation Network for image-text retrieval

Image-text retrieval is an important kind of cross-modal retrieval method and has recently attracted much attention. Existing image-text retrieval methods often ignore the relative importance of each fragment (region in an image or word in a sentence) on the global semantic of image or text when aggregating features of image or text fragments, resulting in the ineffectiveness of the learned image and text representations. To address this problem, we propose an image-text retrieval method named Global-aware Fragment Representation Aggregation Network (GFRAN). Specifically, it first designs a fine-grained multimodal information interaction module based on the self-attention mechanism to model both the intra-modality and inter-modality relationships between image regions and words. Then, with the guidance of the global image or text feature, it aggregates image or text fragment features conditioned on their attention weights over global feature, to highlight fragments that contribute more to the overall semantics of images and texts. Extensive experiments on two benchmark datasets Flickr30K and MS-COCO demonstrate the superiority of the proposed GFRAN model over several state-of-the-art baselines.

A phenotypic drug discovery approach by latent interaction in deep learning

Contemporary drug discovery paradigms rely heavily on binding assays about the bio-physicochemical processes. However, this dominant approach suffers from overlooked higher-order interactions arising from the intricacies of molecular mechanisms, such as those involving cis -regulatory elements. It introduces potential impairments and restrains the potential development of computational methods. To address this limitation, I developed a deep learning model that leverages an end-to-end approach, relying exclusively on therapeutic information about drugs. By transforming textual representations of drug and virus genetic information into high-dimensional latent representations, this method evades the challenges arising from insufficient information about binding specificities. Its strengths lie in its ability to implicitly consider complexities such as epistasis and chemical–genetic interactions, and to handle the pervasive challenge of data scarcity. Through various modeling skills and data augmentation techniques, the proposed model demonstrates outstanding performance in out-of-sample validations, even in scenarios with unknown complex interactions. Furthermore, the study highlights the importance of chemical diversity for model training. While the method showcases the feasibility of deep learning in data-scarce scenarios, it reveals a promising alternative for drug discovery in situations where knowledge of underlying mechanisms is limited.

Breaking the gap between label correlation and instance similarity via new multi-label contrastive learning

Multi-label text classification (MLTC) is a fundamental yet challenging task in natural language processing. Existing MLTC models mostly learn text representations and label correlations, separately; while the instance-level correlation, which is crucial for the classification is ignored. To rectify this, we propose a new multi-label contrastive learning model, that captures instance-level correlations, for the MLTC task. Specifically, we first learn label representations by using Graph Convolutional Network (GCN) on label co-occurrence graphs. We next learn text representations by taking label correlations into consideration. Through an attention mechanism, instance-level correlation can be established. To better utilize label correlations, we propose a new contrastive learning model, whose learning is guided by a new learning objective, to further refine label representations. We finally implement a k-NN mechanism, that identifies k nearest neighbors of a given text for final prediction. Intensive experimental studies over benchmark multi-label datasets demonstrate the effectiveness of our approach.

Hcnnxgboost: A Hybrid Cnn-Xgboost Approach for Effective Emotion Detection in Textual Data

in recent years, emotional analysis has become a key focus in computational studies, driven by the need to understand societal sentiments. this exploration is motivated by its many promising applications, such as community well-being evaluation, human-computer interaction, suicide prevention, and personalized recommendations. even with progress in other areas like identifying expressions from facial cues and speech, the study of text-based emotion recognition remains a fascinating field of research because machines struggle to interpret context, especially compared to human capabilities. our research addresses this by introducing a multiclass text-based emotion detection system that combines a cnn architecture with xgboost for improved classification in natural language processing. we pre-process publicly available datasets and use glove pre-trained word embeddings for better text representation. a major contribution of our work is enhancing the feature space by combining cnn probabilities with the original text data. the proposed hcnnxgboost model outperforms all other machine learning and deep learning algorithms across the emoint, isear, and crowdflower datasets, achieving f-scores of 90.1%, 87.4%, and 62.2%, respectively. experimental evaluations on benchmark datasets show better f-scores, confirming the effectiveness of our approach. comparisons with other classifiers highlight the enhanced performance and effectiveness of our hybrid cnnxgboost (hcnnxgboost) model, making it one of the best solutions for emotion classification in natural language processing tasks.

Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification

Abstract Hierarchical text classification (HTC) is an important task with broad applications, and few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.

Research of multi-label text classification based on label attention and correlation networks.

Multi-Label Text Classification (MLTC) is a crucial task in natural language processing. Compared to single-label text classification, MLTC is more challenging due to its vast collection of labels which include extracting local semantic information, learning label correlations, and solving label data imbalance problems. This paper proposes a model of Label Attention and Correlation Networks (LACN) to address the challenges of classifying multi-label text and enhance classification performance. The proposed model employs the label attention mechanism for a more discriminative text representation and uses the correlation network based on label distribution to enhance the classification results. Also, a weight factor based on the number of samples and a modulation function based on prediction probability are combined to alleviate the label data imbalance effectively. Extensive experiments are conducted on the widely-used conventional datasets AAPD and RCV1-v2, and extreme datasets EUR-LEX and AmazonCat-13K. The results indicate that the proposed model can be used to deal with extreme multi-label data and achieve optimal or suboptimal results versus state-of-the-art methods. For the AAPD dataset, compared with the suboptimal method, it outperforms the second-best method by 2.05% ∼ 5.07% in precision@k and by 2.10% ∼ 3.24% in NDCG@k for k = 1, 3, 5. The superior outcomes demonstrate the effectiveness of LACN and its competitiveness in dealing with MLTC tasks.

Comparison of Latent Semantic Analysis (LSA) and Doc2Vec Algorithms of Thesis Similarity Detection

This study aims to develop a system for detecting similarities in thesis titles and content to prevent plagiarism and support student originality. The high level of similarity in final projects is a significant concern in academic environments. Two text vectorization methods, Latent Semantic Analysis (LSA) and Doc2Vec, were compared to measure document similarity. Results showed that LSA achieved a very high cosine similarity (99.94%) due to dimensionality reduction that preserved semantic correlations. In contrast, Doc2Vec produced lower similarity scores, with 7.17% for PV-DM and 39.07% for PV-DBOW, indicating richer text representations. This study adopted the CRISP-DM model, which includes Business Understanding, Data Understanding, Data Preparation, Modelling, and Evaluation. The model is expected to strengthen academic integrity and encourage valuable scientific contributions.

Transformer-Based Regression Models for Assessing Reading Passage Complexity: A Deep Learning Approach in Natural Language Processing

Natural Language Processing (NLP) is a vital area in deep learning, widely applied in tasks like text classification, virtual assistants, speech recognition, and autocorrect features in digital devices. It allows machines to understand and generate human language, enhancing user interactions with software. This paper presents a deep learning model using the Transformer architecture for a regression task to predict the complexity of reading passages based on text excerpts. By leveraging the Transformer’s capability to identify complex patterns in text, the model achieves a relative error rate of about 10%. The paper also examines how different architectural choices influence model performance, focusing on one-hot encoding and embeddings. While one-hot encoding provides a simple text representation, embeddings offer a richer, more nuanced understanding of word relationships. The findings highlight the significance of model design and data representation in optimizing NLP tasks, providing insights for future advancements in the field.