Strong Baseline Research Articles

Three-way open intent classification with nearest centroid-based representation

Open intent classification aims to identify the unknown (open) intents and simultaneously classify the known ones under the open-world assumption. However, the existing studies still face two challenges, i.e., coarse-grained representation learning and uncertain decision boundary. On the one hand, previous methods viewed each class as a unified entity during representation learning, which fails to capture the fine-grained intra-class data structure. On the other hand, traditional two-way decision for open classification struggle to classify the uncertain samples distributed at the edge of the decision boundary, increasing the risk of misclassification. To overcome these limitations, we present a three-way open intent classification method that utilizes the nearest centroid to learn representations, named 3WNC-Open. Specifically, we learn a structured representation by extracting fine-grained information from the sub-classes within each class. Then, we design a three-way open classification strategy to handle uncertainty, initially identifying uncertain samples and then processing them using an effective alternative approach. Experiments on challenging datasets demonstrate that 3WNC-Open outperforms strong baselines.

What works in implementing shared medical appointments for patients with diabetes in primary care to enhance reach: a qualitative comparative analysis from the Invested in Diabetes study

BackgroundDiabetes is a serious public health problem affecting 37.3 million Americans. Diabetes shared medical appointments (SMAs) are an effective strategy for providing diabetes self-management support and education in primary care. However, practices delivering SMAs experience implementation challenges. This analysis examined conditions associated with successful practice implementation of diabetes SMAs in the context of participation in a pragmatic trial.MethodsMixed methods study using qualitative and quantitative data collected from interviews, observations, surveys, and practice-reported data, guided by the practical, robust implementation and sustainability model (PRISM). Data were analyzed using qualitative comparative analysis (QCA). Successful implementation was defined as meeting patient recruitment targets (Reach) during the study period. Participants were clinicians and staff members from 22 primary care practices in Colorado and Missouri, USA.ResultsThe first necessary condition identified from the QCA was the presence of additional resources for patients with diabetes in the practice. Within practices that had these additional resources, we found that a sufficiency condition was the presence of an effective key person to make things happen with the SMAs. A second QCA was conducted to determine conditions underlying the presence of the effective key person (often performing functions of an implementation champion), which revealed factors including low or managed employee turnover, a strong baseline practice culture, and previous experience delivering SMAs.ConclusionsIdentification of key factors necessary and sufficient for implementation of new care processes is important to enhance patient access to evidence-based interventions. This study suggests that practice features and resources have important implications for implementation of diabetes SMAs. There may be opportunities to support practices with SMA implementation by enabling the presence of skilled implementation champions.Trial registrationRegistered at clinicaltrials.gov under trial ID NCT03590041, registered on July 18, 2018.

Explicitly Exploiting Implicit User and Item Relations in Graph Convolutional Network (GCN) for Recommendation

Most existing collaborative filtering-based recommender systems rely solely on available user–item interactions for user and item representation learning. Their performance often suffers significantly when interactions are sparse, as limited user and item interactions are insufficient for learning robust representations. To address this issue, recent research has explored additional information between users and items by leveraging the user–item bipartite graph. However, these methods have not fully exploited high-order neighborhood information, primarily using sampled interactions to enrich training data rather than integrating this information directly into representation learning. In this paper, we propose a novel model, EIR-GCN (Embedding Integration with Relational Graph Convolutional Network), which directly incorporates various types of collaborative relations, such as user–user and item–item interactions, into the embedding function for user preference modeling. Specifically, our model employs advanced graph convolutional network (GCN) techniques to integrate user–item, user–user, and item–item relations for comprehensive representation learning. EIR-GCN initially selects the most influential second-order neighbors from the user–item bipartite graph to form user–user and item–item connections. With these enriched connections, a message-passing method is adopted to learn node representations by aggregating messages from directly linked nodes, including first-order item neighbors and selected second-order user neighbors. Extensive experiments on several public datasets demonstrate that EIR-GCN outperforms strong baselines, including recent GCN-based models and those exploiting high-order information. Our results show that EIR-GCN achieves state-of-the-art performance and effectively addresses the sparsity issue, highlighting its robustness and efficacy in recommendation tasks.

Large Language Models for Inorganic Synthesis Predictions.

We evaluate the effectiveness of pretrained and fine-tuned large language models (LLMs) for predicting the synthesizability of inorganic compounds and the selection of precursors needed to perform inorganic synthesis. The predictions of fine-tuned LLMs are comparable to─and sometimes better than─recent bespoke machine learning models for these tasks but require only minimal user expertise, cost, and time to develop. Therefore, this strategy can serve both as an effective and strong baseline for future machine learning studies of various chemical applications and as a practical tool for experimental chemists.

What distinguishes conspiracy from critical narratives? A computational analysis of oppositional discourse

AbstractThe current prevalence of conspiracy theories on the internet is a significant issue, tackled by many computational approaches. However, these approaches fail to recognize the relevance of distinguishing between texts which contain a conspiracy theory and texts which are simply critical and oppose mainstream narratives. Furthermore, little attention is usually paid to the role of inter‐group conflict in oppositional narratives. We contribute by proposing a novel topic‐agnostic annotation scheme that differentiates between conspiracies and critical texts, and that defines span‐level categories of inter‐group conflict. We also contribute with the multilingual XAI‐DisInfodemics corpus (English and Spanish), which contains a high‐quality annotation of Telegram messages related to COVID‐19 (5000 messages per language). We also demonstrate the feasibility of an NLP‐based automatization by performing a range of experiments that yield strong baseline solutions. Finally, we perform an analysis which demonstrates that the promotion of intergroup conflict and the presence of violence and anger are key aspects to distinguish between the two types of oppositional narratives, that is, conspiracy versus critical.

Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm.

Previous work for video captioning aims to objectively describe the video content but the captions lack human interest and attractiveness, limiting its practical application scenarios. The intention of video title generation (video titling) is to produce attractive titles, but there is a lack of benchmarks. This work offers CREATE, the first large-scale Chinese shoRt vidEo retrievAl and Title gEneration dataset, to assist research and applications in video titling, video captioning, and video retrieval in Chinese. CREATE comprises a high-quality labeled 210 K dataset and two web-scale 3 M and 10 M pre-training datasets, covering 51 categories, 50K+ tags, 537K+ manually annotated titles and captions, and 10M+ short videos with original video information. This work presents ACTEr, a unique Attractiveness-Consensus-based Title Evaluation, to objectively evaluate the quality of video title generation. This metric measures the semantic correlation between the candidate (model-generated title) and references (manual-labeled titles) and introduces attractive consensus weights to assess the attractiveness and relevance of the video title. Accordingly, this work proposes a novel multi-modal ALignment WIth Generation model, ALWIG, as one strong baseline to aid future model development. With the help of a tag-driven video-text alignment module and a GPT-based generation module, this model achieves video titling, captioning, and retrieval simultaneously. We believe that the release of the CREATE dataset, ACTEr metric, and ALWIG model will encourage in-depth research on the analysis and creation of Chinese short videos.

Towards Better Quantity Representations for Solving Math Word Problems

Solving a math word problem requires selecting quantities in it and performing appropriate arithmetic operations to obtain the answer. For deep learning-based methods, it is vital to obtain good quantity representations, i.e., to selectively and emphatically aggregate information in the context of quantities. However, existing works have not paid much attention to this aspect. Many works simply encode quantities as ordinary tokens, or use some implicit or rule-based methods to select information in their context. This leads to poor results when dealing with linguistic variations and confounding quantities. This article proposes a novel method to identify question-related distinguishing features of quantities by contrasting their context with the question and the context of other quantities, thereby enhancing the representation of quantities. Our method not only considers the contrastive relationship between quantities but also considers multiple relationships jointly. Besides, we propose two auxiliary tasks to further guide the representation learning of quantities: (1) predicting whether a quantity is used in the question and (2) predicting the relations (operators) between quantities given the question. Experimental results show that our method outperforms previous methods on SVAMP and ASDiv-A under similar settings, even some newly released strong baselines. Supplementary experiments further confirm that our method indeed improves the performance of quantity selection by improving the representation of both quantities and questions.

FETILDA: Evaluation Framework for Effective Representations of Long Financial Documents

In the financial sphere, there is a wealth of accumulated unstructured financial data, such as the textual disclosure documents that companies submit on a regular basis to regulatory agencies, such as the Securities and Exchange Commission. These documents are typically very long and tend to contain valuable soft information about a company’s performance that is not present in quantitative predictors. It is therefore of great interest to learn predictive models from these long textual documents, especially for forecasting numerical key performance indicators. In recent years, there has been great progress in natural language processing via pre-trained language models (LMs) learned from large corpora of textual data. This prompts the important question of whether they can be used effectively to produce representations for long documents, as well as how we can evaluate the effectiveness of representations produced by various LMs. Our work focuses on answering this critical question, namely, the evaluation of the efficacy of various LMs in extracting useful soft information from long textual documents for prediction tasks. In this article, we propose and implement a deep learning evaluation framework that utilizes a sequential chunking approach combined with an attention mechanism. We perform an extensive set of experiments on a collection of 10-K reports submitted annually by U.S. banks, and another dataset of reports submitted by U.S. companies, to investigate thoroughly the performance of different types of language models. Overall, our framework using LMs outperforms strong baseline methods for textual modeling as well as for numerical regression. Our work provides better insights into how utilizing pre-trained domain-specific and fine-tuned long-input LMs for representing long documents can improve the quality of representation of textual data and, therefore, help in improving predictive analyses.

Improving platelet-RNA-based diagnostics: a comparative analysis of machine learning models for cancer detection and multiclass classification.

Liquid biopsy demonstrates excellent potential in patient management by providing a minimally invasive and cost-effective approach to detecting and monitoring cancer, even at its early stages. Due to the complexity of liquid biopsy data, machine-learning techniques are increasingly gaining attention in sample analysis, especially for multidimensional data such as RNA expression profiles. Yet, there is no agreement in the community on which methods are the most effective or how to process the data. To circumvent this, we performed a large-scale study using various machine-learning techniques. First, we took a closer look at existing datasets and filtered out some patients to assert data collection quality. The final data collection included platelet RNA samples acquired from 1397 cancer patients (17 types of cancer) and 354 asymptomatic, presumed healthy, donors. Then, we assessed an array of different machine-learning models and techniques (e.g., feature selection of RNA transcripts) in pan-cancer detection and multiclass classification. Our results show that simple logistic regression performs the best, reaching a 68% cancer detection rate at a 99% specificity level, and multiclass classification accuracy of 79.38% when distinguishing between five cancer types. In summary, by revisiting classical machine-learning models, we have exceeded the previously used method by 5% and 9.65% in cancer detection and multiclass classification, respectively. To ease further research, we open-source our code and data processing pipelines (https://gitlab.com/jopekmaksym/improving-platelet-rna-based-diagnostics), which we hope will serve the community as a strong baseline.

Mbtransfer: Microbiome intervention analysis using transfer functions and mirror statistics.

Time series studies of microbiome interventions provide valuable data about microbial ecosystem structure. Unfortunately, existing models of microbial community dynamics have limited temporal memory and expressivity, relying on Markov or linearity assumptions. To address this, we introduce a new class of models based on transfer functions. These models learn impulse responses, capturing the potentially delayed effects of environmental changes on the microbial community. This allows us to simulate trajectories under hypothetical interventions and select significantly perturbed taxa with False Discovery Rate guarantees. Through simulations, we show that our approach effectively reduces forecasting errors compared to strong baselines and accurately pinpoints taxa of interest. Our case studies highlight the interpretability of the resulting differential response trajectories. An R package, mbtransfer, and notebooks to replicate the simulation and case studies are provided.

Improving biomedical Named Entity Recognition with additional external contexts

Objective:Biomedical Named Entity Recognition (bio NER) is the task of recognizing named entities in biomedical texts. This paper introduces a new model that addresses bio NER by considering additional external contexts. Different from prior methods that mainly use original input sequences for sequence labeling, the model takes into account additional contexts to enhance the representation of entities in the original sequences, since additional contexts can provide enhanced information for the concept explanation of biomedical entities. Methods:To exploit an additional context, given an original input sequence, the model first retrieves the relevant sentences from PubMed and then ranks the retrieved sentences to form the contexts. It next combines the context with the original input sequence to form a new enhanced sequence. The original and new enhanced sequences are fed into PubMedBERT for learning feature representation. To obtain more fine-grained features, the model stacks a BiLSTM layer on top of PubMedBERT. The final named entity label prediction is done by using a CRF layer. The model is jointly trained in an end-to-end manner to take advantage of the additional context for NER of the original sequence. Results:Experimental results on six biomedical datasets show that the proposed model achieves promising performance compared to strong baselines and confirms the contribution of additional contexts for bio NER. Conclusion:The promising results confirm three important points. First, the additional context from PubMed helps to improve the quality of the recognition of biomedical entities. Second, PubMed is more appropriate than the Google search engine for providing relevant information of bio NER. Finally, more relevant sentences from the context are more beneficial than irrelevant ones to provide enhanced information for the original input sequences. The model is flexible to integrate any additional context types for the NER task.

CLSESSP: Contrastive learning of sentence embedding with strong semantic prototypes

By drawing inspiration from the computer vision domain, contrastive learning has emerged as another paradigm for unsupervised sentence embedding. By using contrastive learning, most recent sentence embedding methods have achieved promising results. However, previous methods focused on constructing positive and negative pairs through traditional data augmentation, which was unable to balance the loss of semantics and the additional features brought by aggressive data augmentation. In the present study, we propose a prototype contrastive learning framework based on a distribution divergence minimization loss function and introduce the concept of strong semantic prototypes. Specifically, with the aid of prompts, our method constructs three semantic prototypes for each instance: a basic semantic prototype, a strong semantic prototype and a negative prototype. By combining InfoNCE loss with distribution divergence minimization loss, Contrastive Learning of Sentence Embedding with Strong Semantic Prototypes (CLSESSP) creates another optimization objective that integrates strong augmentation. The experiments conducted on the semantic text similarity (STS) datasets demonstrate that the proposed CLSESSP surpasses the strong baseline by an average of 2.9 points in the BERT-base model. Extensive experimental results on transfer and clustering tasks demonstrate the effectiveness of our proposed model compared to strong baselines. The code is available at https://github.com/KCshen1125/CLSESSP.

A systematic study of DNN based speech enhancement in reverberant and reverberant-noisy environments

Deep learning has led to dramatic performance improvements for the task of speech enhancement, where deep neural networks (DNNs) are trained to recover clean speech from noisy and reverberant mixtures. Most of the existing DNN-based algorithms operate in the frequency domain, as time-domain approaches are believed to be less effective for speech dereverberation. In this study, we employ two DNNs: ARN (attentive recurrent network) and DC-CRN (densely-connected convolutional recurrent network), and systematically investigate the effects of different components on enhancement performance, such as window sizes, loss functions, and feature representations. We conduct evaluation experiments in two main conditions: reverberant-only and reverberant-noisy. Our findings suggest that incorporating larger window sizes is helpful for dereverberation, and adding transform operations (either convolutional or linear) to encode and decode waveform features improves the sparsity of the learned representations, and boosts the performance of time-domain models. Experimental results demonstrate that ARN and DC-CRN with proposed techniques achieve superior performance compared with other strong enhancement baselines.

Neighbors selective Graph Convolutional Network for homophily and heterophily

Graph Convolutional Networks (GCNs) gain remarkable success in graph-related tasks under homophily graph assumption—most connected nodes have the same label. However, this assumption is fragile since heterophily is common in real-world networks, where most linked nodes have different labels. Some methods address heterophily using graph-level weighted combinations of multi-hop neighbor representations. However, they introduce noise and irrelevant information from two aspects: (1) higher-order neighborhoods may include more neighbors with different labels than those with the same label; (2) graph-level weighted combination fails to capture distinct node properties. To address these issues, firstly, a Neighborhood Distribution-induced Similarity (NDS) measure is developed to identify potential similar neighbors for nodes. Secondly, a node-level fusion mechanism Selective-Neighbors Gated Unit (SNGU) is designed to adaptively aggregate potential neighbors, first-hop neighbors, and the node itself by learning distinct weights for nodes. By combining the above two designs, this work proposes a novel Neighbors Selective Graph Convolutional Network (NSGCN), which allows nodes to selectively receive relevant neighbor information for better node representations, effectively modeling homophily and heterophily. Experiments on 10 widely used real-world datasets with varying properties demonstrate the superiority of the NSGCN, surpassing a strong baseline with an absolute increase of 9.83% in classification accuracy. The code is available at https://github.com/GGA23/NSGCN.

Learning to Generalize With Object-Centric Agents in the Open World Survival Game Crafter

Reinforcement learning agents must generalize beyond their training experience. Prior work has focused mostly on identical training and evaluation environments. Starting from the recently introduced Crafter benchmark, a 2D open world survival game, we introduce a new set of environments suitable for evaluating some agent's ability to generalize on previously unseen (numbers of) objects and to adapt quickly (meta-learning). In Crafter, the agents are evaluated by the number of unlocked achievements (such as collecting resources) when trained for 1M steps. We show that current agents struggle to generalize, and introduce novel object-centric agents that improve over strong baselines. We also provide critical insights of general interest for future work on Crafter through several experiments. We show that careful hyper-parameter tuning improves the PPO baseline agent by a large margin and that even feedforward agents can unlock almost all achievements by relying on the inventory display. We achieve new state-of-the-art performance on the original Crafter environment. Additionally, when trained beyond 1M steps, our tuned agents can unlock almost all achievements. We show that the recurrent PPO agents improve over feedforward ones, even with the inventory information removed. We introduce CrafterOOD, a set of 15 new environments that evaluate OOD generalization. On CrafterOOD, we show that the current agents fail to generalize, whereas our novel object-centric agents achieve state-of-the-art OOD generalization while also being interpretable. Our code is public. <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/astanic/crafter-ood</uri>

SPARK: A High-Efficiency Black-Box Domain Adaptation Framework for Source Privacy-Preserving Drowsiness Detection.

Developing an effective and efficient electroencephalography (EEG)-based drowsiness monitoring system is crucial for enhancing road safety and reducing the risk of accidents. For general usage, cross-subject evaluation is indispensable. Despite progress in unsupervised domain adaptation (UDA) and source-free domain adaptation (SFDA) methods, these often rely on the availability of labeled source data or white-box source models, posing potential privacy risks. This study explores a more challenging setting of UDA for EEG-based drowsiness detection, termed black-box domain adaptation (BBDA). In BBDA, adaptation in the target domain relies solely on a black-box source model, without access to the source data or parameters of the source model. Specifically, we propose a framework called Self-distillation and Pseudo-labelling for Ensemble Deep Random Vector Functional Link (edRVFL)-based Black-box Knowledge Adaptation (SPARK). SPARK employs entropy-based selection of high-confidence samples, which are then pseudo-labeled to train a student edRVFL network. Subsequently, ensemble self-distillation is performed to extract knowledge by training the edRVFL using refined labels introduced by ensemble learning. This process further improves the robustness of the student edRVFL network. The features of the edRVFL are beneficial for improving the computational efficiency of the framework, making it more suitable for tasks involving small datasets. The proposed SPARK framework is evaluated on two publicly available driver drowsiness datasets. Experimental results demonstrate its superior performance over strong baselines, while significantly reducing training time. These findings underscore the potential for practical integration of the proposed framework into drowsiness monitoring systems, thereby contributing substantially to the privacy preservation of source subjects.

Compositionally Equivariant Representation Learning.

Deep learning models often need sufficient supervision (i.e., labelled data) in order to be trained effectively. By contrast, humans can swiftly learn to identify important anatomy in medical images like MRI and CT scans, with minimal guidance. This recognition capability easily generalises to new images from different medical facilities and to new tasks in different settings. This rapid and generalisable learning ability is largely due to the compositional structure of image patterns in the human brain, which are not well represented in current medical models. In this paper, we study the utilisation of compositionality in learning more interpretable and generalisable representations for medical image segmentation. Overall, we propose that the underlying generative factors that are used to generate the medical images satisfy compositional equivariance property, where each factor is compositional (e.g., corresponds to human anatomy) and also equivariant to the task. Hence, a good representation that approximates well the ground truth factor has to be compositionally equivariant. By modelling the compositional representations with learnable von-Mises-Fisher (vMF) kernels, we explore how different design and learning biases can be used to enforce the representations to be more compositionally equivariant under un-, weakly-, and semi-supervised settings. Extensive results show that our methods achieve the best performance over several strong baselines on the task of semi-supervised domain-generalised medical image segmentation. Code will be made publicly available upon acceptance at https://github.com/vios-s.

Channel-Aware Decoupling Network for Multiturn Dialog Comprehension.

Training machines to understand natural language and interact with humans is one of the major goals of artificial intelligence. Recent years have witnessed an evolution from matching networks to pretrained language models (PrLMs). In contrast to the plain-text modeling as the focus of the PrLMs, dialog texts involve multiple speakers and reflect special characteristics, such as topic transitions and structure dependencies, between distant utterances. However, the related PrLM models commonly represent dialogs sequentially by processing the pairwise dialog history as a whole. Thus, the hierarchical information on either utterance interrelation or speaker roles coupled in such representations is not well addressed. In this work, we propose compositional learning for holistic interaction across the utterances beyond the sequential contextualization from PrLMs, in order to capture the utterance-aware and speaker-aware representations entailed in a dialog history. We decouple the contextualized word representations by masking mechanisms in transformer-based PrLM, making each word only focus on the words in the current utterance, other utterances, and two speaker roles (i.e., utterances of the sender and utterances of the receiver), respectively. In addition, we employ domain-adaptive training strategies to help the model adapt to the dialog domains. Experimental results show that our method substantially boosts the strong PrLM baselines in four public benchmark datasets, achieving new state-of-the-art performance over previous methods.

Enhancing Cross-Lingual Sarcasm Detection by a Prompt Learning Framework with Data Augmentation and Contrastive Learning

Given their intricate nature and inherent ambiguity, sarcastic texts often mask deeper emotions, making it challenging to discern the genuine feelings behind the words. The proposal of the sarcasm detection task is to assist us with more accurately understanding the true intention of the speaker. Advanced methods, such as deep learning and neural networks, are widely used in the field of sarcasm detection. However, most research mainly focuses on sarcastic texts in English, as other languages lack corpora and annotated datasets. To address the challenge of low-resource languages in sarcasm detection tasks, a zero-shot cross-lingual transfer learning method is proposed in this paper. The proposed approach is based on prompt learning and aims to assist the model with understanding downstream tasks through prompts. Specifically, the model uses prompt templates to construct training data into cloze-style questions and then trains them using a pre-trained cross-lingual language model. Combining data augmentation and contrastive learning can further improve the capacity of the model for cross-lingual transfer learning. To evaluate the performance of the proposed model, we utilize a publicly accessible sarcasm dataset in English as training data in a zero-shot cross-lingual setting. When tested with Chinese as the target language for transfer, our model achieves F1-scores of 72.14% and 76.7% on two test datasets, outperforming the strong baselines by significant margins.

Nurse Educators, Clinical Nurses and Students’ Knowledge, Attitude and Practice towards Nursing Education in Bhutan

Abstract Background: In Bhutan, the Health School started in 1972 providing a 2-year certificate course for community health workers, assistant nurses and assistant technicians. In 1982, the Diploma course for nurses commenced and was assisted by the RAK College of Nursing Faculty, New Delhi, India, in formalising the curriculum and also carried out the evaluation of the first batch of nursing students. Objective: The study assesses the knowledge, attitude and practice (KAP) of nurse educators (NEs), clinical nurses and students undergoing the Diploma in Nursing and Midwifery training towards nursing education in Bhutan. Methods: The methodology was a quantitative study using a self-administered questionnaire. The first section had demographic data, and the second had questions on KAP. The participants were NEs, students and clinical nurses. The study was done in the nursing institute, the national hospital and the district hospitals. The instrument, i.e. questionnaire, had a five-point Likert scale from ‘excellent, good, fair, poor, to very poor’ Results: In the results, students rated the KAP of NEs and clinical nurses as ‘good’. On the contrary, the attitude of students towards ‘gaining knowledge from the clinical area’ and their ‘eagerness to learn nursing’ are rated ‘fair’ Conclusion: The KAP provides a detailed view of the status of nursing education, and the data obtained from the study provide strong baseline information for nursing education in the future.