Zero-shot Learning Research Articles

Path Planning for Robots Combined with Zero-Shot and Hierarchical Reinforcement Learning in Novel Environments

Path planning for robots based on reinforcement learning encounters challenges in integrating semantic information about environments into the training process. In unseen or complex environmental information, agents often perform sub-optimally and require more training time. In response to these challenges, this manuscript pioneers a framework integrating zero-shot learning combined with hierarchical reinforcement learning to enhance agent decision-making in complex environments. Zero-shot learning enables agents to infer correct actions for previously unseen objects or situations based on learned semantic associations. Subsequently, the path planning component utilizes hierarchical reinforcement learning with adaptive replay buffer, directed by the insights gained from zero-shot learning, to make decisions effectively. Two parts are trained separately, so zero-shot learning is available in different and unseen environments. Through simulation experiments, we compare the traditional hierarchical reinforcement learning method with the proposed method. The results prove that this structure can make full use of environmental information to generalize across unseen environments and plan collision-free paths.

Knowledge Guided Transformer Network for Compositional Zero-Shot Learning

Compositional Zero-shot Learning (CZSL) attempts to recognise images of new compositions of states and objects when images of only a subset of state-object compositions are available as training data. An example of CZSL is to recognise images of peeled apple by a model when it is trained using images of peeled orange , ripe apple and ripe orange . There are two major challenges in solving CZSL. First, the visual features of a state vary depending on the context of a state-object composition. For example state like ripe produces distinct visual properties in the compositions ripe orange and ripe banana . Hence, understanding the context dependency of state features is a necessary requirement to solve CZSL. Second, the extent of association between the features of a state and an object varies significantly in different images of same composition. For example, in different images of peeled oranges , the oranges may be peeled to different extents. As a consequence, the visual features of images of the class peeled orange may vary. Hence, there exists a significant amount of intra-class variability among the visual features of different images of a composition. Existing approaches merely look for the existence or absence of features of particular state or object in a composition. Our approach not only looks for the existence of a particular state features or object features but also the extent of association of state features and object features to better tackle the intra-class variability in visual features of compositional images. The proposed architecture is constructed using a novel Knowledge Guided Transformer . The transformer-based framework is utilised for processing larger context dependency between the state and object. Extensive experiments on C-GQA, MIT-States and UT-Zappos50k datasets demonstrate the superiority of the proposed approach in comparison with the state-of-the-art in both open-world and closed-world CZSL settings.

Abstract Su801: Use of Large Language Models to Optimize Clinical Text Analysis for In-Hospital Cardiac Arrest Identification

Introduction/Background: In-hospital cardiac arrest (IHCA) is experienced by approximately 300,000 patients annually in the US. While individual care teams can readily identify IHCA at the bedside, the reporting of these events after the event often lacks consistency and poses a major challenge for hospitals across the US. Accurate and timely reporting of IHCA events is crucial for facilitating QI initiatives, such as CPR quality review, optimizing team performance, and benchmarking IHCA outcomes. Hypothesis: Large language models (LLMs) can be used to identify IHCA events at the patient encounter-level by analyzing targeted note types from the inpatient record with high performance. Methods: Adult (≥18 years old) inpatient encounters at the Hospital of the University of Pennsylvania from 06/2018 to 03/2022 with a reported clinical emergency were identified from a QI database. Discharge summaries and notes associated with the encounters of interest were extracted from the EHR and deidentified using a natural language processing tool. Notes were reviewed by research staff to ascertain true IHCA events. Positive IHCA was defined as the loss of pulses followed by the delivery of CPR. An LLM, GPT4 v. 32K-Chat, was employed on the Penn Medicine Microsoft Azure Databricks environment to compare zero- and few-shot prompting methods to identify IHCA. Model performance was measured using accuracy, precision, recall and F1 scores. Results: Manual chart review determined 96.8% and 3.2% of cases to be IHCA+ and IHCA-, respectively. A balanced pilot dataset of positive and negative cases was created and the accuracy values across strategically sampled note types was checked. Using zero-shot learning with Chain-of-Thought (COT) prompting, the LLM performed with an accuracy of 86%, precision of 77%, recall of 100%, and F1-score of 87%. Using three-shot learning with COT, the LLM performed with an accuracy of 90%, precision of 83%, recall of 100%, and F1-score of 91%. Conclusions: This study demonstrates the potential efficacy of leveraging LLMs to automatically classify IHCA from strategically sampled note types using both zero and few-shot learning approaches. In the future, we will implement temporal sampling of note types and refine model parameters such as output token size and temperature. These findings suggest that such technologies can be effective in real-world clinical settings, providing a scalable solution to improve patient outcomes and quality improvement efforts.

Advanced Imaging Integration: Multi-Modal Raman Light Sheet Microscopy Combined with Zero-Shot Learning for Denoising and Super-Resolution.

This study presents an advanced integration of Multi-modal Raman Light Sheet Microscopy with zero-shot learning-based computational methods to significantly enhance the resolution and analysis of complex three-dimensional biological structures, such as 3D cell cultures and spheroids. The Multi-modal Raman Light Sheet Microscopy system incorporates Rayleigh scattering, Raman scattering, and fluorescence detection, enabling comprehensive, marker-free imaging of cellular architecture. These diverse modalities offer detailed spatial and molecular insights into cellular organization and interactions, critical for applications in biomedical research, drug discovery, and histological studies. To improve image quality without altering or introducing new biological information, we apply Zero-Shot Deconvolution Networks (ZS-DeconvNet), a deep-learning-based method that enhances resolution in an unsupervised manner. ZS-DeconvNet significantly refines image clarity and sharpness across multiple microscopy modalities without requiring large, labeled datasets, or introducing artifacts. By combining the strengths of multi-modal light sheet microscopy and ZS-DeconvNet, we achieve improved visualization of subcellular structures, offering clearer and more detailed representations of existing data. This approach holds significant potential for advancing high-resolution imaging in biomedical research and other related fields.

A Cross-Lingual Media Profiling Model for Detecting Factuality and Political Bias

Media profiling offers valuable insights to enhance the objectivity and reliability of news coverage by providing comprehensive analysis, but the diversity in languages posed significant challenges to our identification of factuality and political bias of non-English sources. The limitation of existing media analysis research is its concentration on a singular high-resource language, and it hardly extends to languages beyond English. To address this, we introduce xMP, a dataset for zero-shot cross-lingual media profiling tasks. xMP’s cross-lingual test set encompasses 34 non-English languages and 18 language families, extending media profiling beyond English resources and allowing us to assess cross-lingual media profiling model performance. Additionally, we propose a method, named R-KAT, to enhance the model’s zero-shot cross-lingual transfer learning capability by building virtual multilingual embedding. Our experiments illustrate that our method improves the transferability of models in cross-lingual media profiling tasks. Additionally, we further discuss the performance of our method for different target languages. Our dataset and code are publicly available.

ValveVision AI - a multimodal language model for qualitative reporting of the aortic valve in echocardiography

Abstract Background The precise assessment of the aortic valve via echocardiography is critical for early detection and management of aortic valve diseases. Until recently, previous studies have examined machine learning models to estimate individual measurements and severity of aortic stenosis (AS) from echocardiographic images. These image processing algorithms, while precise in their narrow focus, fall short in mirroring the holistic and interconnected clinical judgment typical of human echocardiographers in producing a qualitative report. Large language models (LLMs), particularly image-to-text multi-modal LLMs, are a fundamental advance in the field of deep learning with implications for a host of applications in medical imaging. They promise to encapsulate not just discrete data points typical in traditional machine learning, but also the complex contextual interrelations in clinical diagnosis. Methods In this study, a large-scale heterogeneous database of echocardiographic images containing over 90,681 studies with textual descriptors of the aortic valve was used to train a single, image-to-text multimodal LLM called ValveVision AI. The ground truth textual summaries were drafted by level III echocardiographers in a clinical setting between 2015-2020. BLEU and ROUGE score was calculated. The models were retrospectively assessed on a holdout dataset. Reviewing physicians compared the generated summary to the ground truth and binarily agreed or rejected it. Receiver Operating Characteristics (ROC) for distinct pathologies were also assessed (Figure I). Results ValveVision AI performed with a BLEU score of 0.45 and a ROUGE score of 0.49. The performance of the model in reporting on classification of moderate/severe vs none/mild AS in concordance with the validation protocol described above achieved a specificity of 91.98% and a sensitivity of 83.89%, along with more precise qualitative description. Qualitatively, the model exhibited the capability of zero-shot learning in certain instances, however, this result remains an area of exploration. Conclusion This study represents to our knowledge, the first attempt at an image-representation to text-tokenizer deep learning model architecture to mimic the thought and subtlety of echocardiographic qualitative analysis of the aortic valve. The results suggest that this multimodal LLM has sufficient accuracy to create a preliminary textual summary of the aortic valve that, if paired with a point of care ultrasound (POCUS) device in a primary care setting, may facilitate case triage, increase efficiency, and determine a more precise care pathway for patients.

Can a zero-shot learning Large Language Model code complex interview data?

Abstract Background Psychological autopsy is essential to establish theories, explore trends and identify previously unexplored psychosocial risk factors in suicide research. However, qualitative research has been scrutinized for being prone to interpretation bias, problems with accuracy, challenges to reproducibility, and is very time- and cost intensive. The current study aimed to investigate if a Large Language Model can achieve sufficient agreement with a researcher in the deductive coding of interview data obtained in a psychological autopsy study of suicide, to be integrated with qualitative research procedures. Methods Data from 38 interviews was deductively coded by a researcher and a LLAMA 3 based language model. The model performance was evaluated in four increasingly difficult coding tasks, including binary classification and data summarization. Intercoder agreement scores were calculated using Cohen’s Kappa. Results The preliminary results showed that the LLM achieved substantial agreement with the human coders for the binary classification task (.78). The variability in performance was influenced by code definitions. The results of the quality of LLM interpretation and summarization will also be presented. Conclusions State-of-the-art LLM can be easily integrated into the qualitative analyses of psychological autopsy interviews and may improve real time monitoring of suicides. We recommend a human-AI collaborative model, whereby deductive coding by the LLM is complemented by human inductive coding and further interpretation. Key messages • Integrating an LLM with qualitative research procedures is feasible in a collaborative model. • Integrating an LLM with qualitative research procedures allows near real-time monitoring based on qualitative data, which extends to other public health fields.

A comprehensive review of deep learning for medical image segmentation

Medical image segmentation provides detailed mappings of regions of interest, facilitating precise identification of critical areas and greatly aiding in the diagnosis, treatment, and understanding of diverse medical conditions. However, conventional techniques frequently rely on hand-crafted feature-based approaches, posing challenges when dealing with complex medical images, leading to issues such as low accuracy and sensitivity to noise. Recent years have seen substantial research focused on the effectiveness of deep learning models for segmenting medical images. In this study, we present a comprehensive review of the various deep learning-based approaches for medical image segmentation and provide a detailed analysis of their contributions to the domain. These methods can be broadly categorized into five groups: CNN-based methods, Transformer-based methods, Mamba-based methods, semi-supervised learning methods, and weakly supervised learning methods. Convolutional Neural Networks (CNNs), with their efficient feature self-learning, have driven major advances in medical image segmentation. Subsequently, Transformers, leveraging self-attention mechanisms, have achieved performance on par with or surpassing Convolutional Neural Networks. Mamba-based methods, as a novel selective state-space model, are emerging as a promising direction. Furthermore, due to the limited availability of annotated medical images, research in weakly supervised and semi-supervised learning continues to evolve. This review covers common evaluation methods, datasets, and deep learning applications in diagnosing and treating skin lesions, hippocampus, tumors, and polyps. Finally, we identify key challenges such as limited data, diverse modalities, noise, and clinical applicability, and propose future research in zero-shot segmentation, transfer learning, and multi-modal techniques to advance the development of medical image segmentation technology.

Advancing health coaching: A comparative study of large language model and health coaches

ObjectiveRecent advances in large language models (LLM) offer opportunities to automate health coaching. With zero-shot learning ability, LLMs could revolutionize health coaching by providing better accessibility, scalability, and customization. The aim of this study is to compare the quality of responses to clients' sleep-related questions provided by health coaches and an LLM. Design, setting, and participantsFrom a de-identified dataset of coaching conversations from a pilot randomized controlled trial, we extracted 100 question-answer pairs comprising client questions and corresponding health coach responses. These questions were entered into a retrieval-augmented generation (RAG)-enabled open-source LLM (LLaMa-2-7b-chat) to generate LLM responses. Out of 100 question-answer pairs, 90 were taken out and assigned to three groups of evaluators: experts, lay-users, and GPT-4. Each group conducted two evaluation tasks: (Task 1) a single-response quality assessment spanning five criteria—accuracy, readability, helpfulness, empathy, and likelihood of harm—rated on a five-point Likert scale, and (Task 2) a pairwise comparison to choose the superior response between pairs. A suite of inferential statistical methods, including the paired and independent sample t-tests, Pearson correlation, and chi-square tests, were utilized to answer the study objective. Recognizing potential biases in human judgment, the remaining 10 question-answer pairs were used to assess inter-evaluator reliability among the human evaluators, quantified using the interclass correlation coefficient and percentage agreement metrics. ResultsUpon exclusion of incomplete data, the analysis included 178 single-response evaluations (Task 1) and 83 pairwise comparisons (Task 2). Expert and GPT-4 assessments revealed no discernible disparities in health coach and LLM responses across the five metrics. Contrarily, lay-users deemed LLM responses significantly more helpful than that of human coaches (p < 0.05). LLM responses were preferred in the majority (62.25 %, n = 155) of the aggregate 249 assessments, with all three evaluator groups favoring LLM over health coach inputs. While GPT-4 rated both health coach and LLM responses significantly higher than experts in terms of readability, helpfulness, and empathy, its ratings on accuracy and likelihood of harm aligned with those of experts. Response length positively correlated with accuracy and empathy scores, but negatively affected readability across all evaluator groups. Expert and lay-user evaluators demonstrated moderate to high inter-evaluator reliability. ConclusionOur study showed encouraging findings by demonstrating that RAG-enabled LLM has comparable performance to health coaches in the domain tested. Serving as an initial step towards the creation of more sophisticated, adaptive, round-the-clock automated health coaching systems, our findings call for more extensive evaluation which could assist in the development of the model that could in the future lead to potential clinical implementation.

Instance-wise multi-view visual fusion for zero-shot learning

Zero-shot learning (ZSL) has become increasing popular in computer vision due to its ability to recognize categories unobserved in the training data. So far, most existing ZSL approaches adopt visual representations that are either derived from pretrained networks or learned using an end-to-end architecture. However, a single group of visual representations can hardly capture all features hidden in the images, yielding incomplete visual information. In numerous real-life scenarios, multi-view visual representations are often accessible which describe the instances more comprehensively and are potential for better learning performance. In this paper, we introduce an instance-wise multi-view visual fusion (IMVF) for zero-shot learning (ZSL) model. In accordance with the consensus principle, a multi-view visual-semantic mapping is created by minimizing the disparities of seen-class semantic projections on different views. Meanwhile, a straightforward linear constraint is performed on each seen-class instance to adhere to the complementary principle so that the cross-view information exchange is well motivated. In order to mitigate the domain shift problem, the predicted unseen-class semantic projections are further refined through a multi-view manifold alignment under the consensus principle. Our proposed IMVFZSL is compared with the State-of-the-Art ZSL methods on AwA2, CUB and SUN datasets. Exciting experimental results validate the effectiveness of the IMVF mechanism. To the best of our understanding, this is an initial attempt to fuse multi-view visual representations in ZSL, which will stimulate valuable contemplation in this field.

PMGNet: Disentanglement and entanglement benefit mutually for compositional zero-shot learning

Compositional zero-shot learning (CZSL) aims to model compositions of two primitives (i.e., attributes and objects) to classify unseen attribute-object pairs. Most studies are devoted to integrating disentanglement and entanglement strategies to circumvent the trade-off between contextuality and generalizability. Indeed, the two strategies can mutually benefit when used together. Nevertheless, they neglect the significance of developing mutual guidance between the two strategies. In this work, we take full advantage of guidance from disentanglement to entanglement and vice versa. Additionally, we propose exploring multi-scale feature learning to achieve fine-grained mutual guidance in a progressive framework. Our approach, termed Progressive Mutual Guidance Network (PMGNet), unifies disentanglement–entanglement representation learning, allowing them to learn from and teach each other progressively in one unified model. Furthermore, to alleviate overfitting recognition on seen pairs, we adopt a relaxed cross-entropy loss to train PMGNet, without an increase of time and memory cost. Extensive experiments on three benchmarks demonstrate that our method achieves distinct improvements, reaching state-of-the-art performance. Moreover, PMGNet exhibits promising performance under the most challenging open-world CZSL setting, especially for unseen pairs.

Zero-shot learning with visual-semantic mutual reinforcement for image recognition

Zero-shot learning with visual-semantic mutual reinforcement for image recognition

Comparative Study of GPT-4Vision and Convolutional Neural Networks in Histopathological Image Analysis

Abstract Introduction/Objective Recent advancements in artificial intelligence (AI), particularly with Large Language Models like ChatGPT-4Vision (GPT-4V), have expanded the potential for automated interpretation of medical images. This study evaluates the diagnostic accuracy of GPT-4V in histopathological analysis of neurodegenerative diseases and compares its performance with traditional Convolutional Neural Networks (CNNs). Methods/Case Report We utilized 1515 histopathological images, including hematoxylin and eosin (H&amp;E) staining and tau immunohistochemistry, from patients with various neurodegenerative diseases, such as Alzheimer’s disease, progressive supranuclear palsy, and corticobasal degeneration. GPT-4V’s performance was assessed through multi- step prompts to assess how textual context influences the image interpretation. In the quantitative analysis, the diagnostic accuracy of GPT-4V and the CNN-based model YOLOv8 was assessed for classifying three tau lesions—astrocytic plaques, neuritic plaques, and tufted astrocytes. The analysis used both zero-shot and few-shot learning methods, where GPT-4V and YOLOv8 were evaluated on their ability to distinguish these lesions using ten hold-out test images and 490 training images per lesion. Results (if a Case Study enter NA) GPT-4V accurately recognized Lewy bodies in all cases, but it showed limited accuracy in identifying neurofibrillary tangles (20%) and Pick bodies (0%) on H&amp;E slides. Notably, GPT-4V often suggested Alzheimer’s disease as a potential diagnosis, relying more on contextual clues, such as the prevalence of certain diseases, rather than direct image analysis. In the tau immunohistochemical analysis, GPT-4V’s diagnoses were strongly influenced by additional information about staining and brain region. However, few-shot learning markedly improved GPT-4V’s diagnostic capabilities from 40% accuracy with zero-shot learning to 90% with few-shot learning, reaching best accuracy with 20-shot learning. This performance compared favorably against YOLOv8, which also achieved 90% accuracy but required 100-shot learning. Conclusion While GPT-4V faces challenges in independently interpreting histopathological images, few-shot learning significantly improves its accuracy. This approach is particularly promising for neuropathology, where acquiring extensive labeled datasets is challenging. The findings highlight the need for ongoing refinement of AI applications in pathology, emphasizing balanced integration of textual and visual data to optimize diagnostic efficacy and reliability.

A Suite of Foundation Models Captures the Contextual Interplay Between Codons.

In the canonical genetic code, many amino acids are assigned more than one codon. Work by us and others has shown that the choice of these synonymous codon is not random, and carries regulatory and functional consequences. Existing protein foundation models ignore this context-dependent role of coding sequence in shaping the protein landscape of the cell. To address this gap, we introduce cdsFM, a suite of codon-resolution large language models, including both EnCodon and DeCodon models, with up to 1B parameters. Pre-trained on 60 million protein-coding sequences from more than 5,000 species, our models effectively learn the relationship between codons and amino acids, recapitualing the overall structure of the genetic code. In addition to outperforming state-of-the-art genomic foundation models in a variety of zero-shot and few-shot learning tasks, the larger pre-trained models were superior in predicting the choice of synonymous codons. To systematically assess the impact of synonymous codon choices on protein expression and our models' ability to capture these effects, we generated a large dataset measuring overall and surface expression levels of three proteins as a function of changes in their synonymous codons. We showed that our EnCodon models could be readily fine-tuned to predict the contextual consequences of synonymous codon choices. Armed with this knowledge, we applied EnCodon to existing clinical datasets of synonymous variants, and we identified a large number of synonymous codons that are likely pathogenic, several of which we experimentally confirmed in a cell-based model. Together, our findings establish the cdsFM suite as a powerful tool for decoding the complex functional grammar underlying the choice of synonymous codons.

Promoting Machine Abilities of Discovering and Utilizing Knowledge in a Unified Zero-shot Learning Paradigm

Knowledge discovery and utilization are two essential cognitive processes that enable humans to understand the world and extract new insights from their surroundings. These processes have motivated machine learning studies, particularly zero-shot learning (ZS), which seeks to identify unseen concepts through the use of side information. Previous ZS studies primarily focused on utilizing existing knowledge to infer unseen events, yet they overlook the crucial process of knowledge discovery and the integrated modeling of these knowledge-aware processes. In this study, we present a comprehensive ZS learning approach that explores and evaluates the machine's abilities of discovering and utilizing knowledge. More specifically, to emulate human-like knowledge discovery and utilization processes, we propose a novel visual-aware ZS knowledge graph completion task for evaluation, incorporating a traditional ZS image classification task. Technically, we develop a unified ZS learning paradigm named Cognitive Learner (CoLa) to foster the two knowledge-aware abilities. Including a knowledge representation learning (KRL) module and a knowledge adaptation (KA) module, CoLa adapts well to the two specified tasks with the corresponding data. Extensive experiments on large-scale datasets demonstrate CoLa models’ outstanding performance over compared methods in the two ZS tasks, illustrating their superior ability of discovering and utilizing knowledge.

Zero-Shot Learning for Accurate Project Duration Prediction in Crowdsourcing Software Development

Crowdsourcing Software Development (CSD) platforms, i.e., TopCoder, function as intermediaries connecting clients with developers. Despite employing systematic methodologies, these platforms frequently encounter high task abandonment rates, with approximately 19% of projects failing to meet satisfactory outcomes. Although existing research has focused on task scheduling, developer recommendations, and reward mechanisms, there has been insufficient attention to the support of platform moderators, or copilots, who are essential to project success. A critical responsibility of copilots is estimating project duration; however, manual predictions often lead to inconsistencies and delays. This paper introduces an innovative machine learning approach designed to automate the prediction of project duration on CSD platforms. Utilizing historical data from TopCoder, the proposed method extracts pertinent project attributes and preprocesses textual data through Natural Language Processing (NLP). Bidirectional Encoder Representations from Transformers (BERT) are employed to convert textual information into vectors, which are then analyzed using various machine learning algorithms. Zero-shot learning algorithms exhibit superior performance, with an average accuracy of 92.76%, precision of 92.76%, recall of 99.33%, and an f-measure of 95.93%. The implementation of the proposed automated duration prediction model is crucial for enhancing the success rate of crowdsourcing projects, optimizing resource allocation, managing budgets effectively, and improving stakeholder satisfaction.

Open-Pose 3D zero-shot learning: Benchmark and challenges

With the explosive 3D data growth, the urgency of utilizing zero-shot learning to facilitate data labeling becomes evident. Recently, methods transferring language or language-image pre-training models like Contrastive Language-Image Pre-training (CLIP) to 3D vision have made significant progress in the 3D zero-shot classification task. These methods primarily focus on 3D object classification with an aligned pose; such a setting is, however, rather restrictive, which overlooks the recognition of 3D objects with open poses typically encountered in real-world scenarios, such as an overturned chair or a lying teddy bear. To this end, we propose a more realistic and challenging scenario named open-pose 3D zero-shot classification, focusing on the recognition of 3D objects regardless of their orientation. First, we revisit the current research on 3D zero-shot classification and propose two benchmark datasets specifically designed for the open-pose setting. We empirically validate many of the most popular methods in the proposed open-pose benchmark. Our investigations reveal that most current 3D zero-shot classification models suffer from poor performance, indicating a substantial exploration room towards the new direction. Furthermore, we study a concise pipeline with an iterative angle refinement mechanism that automatically optimizes one ideal angle to classify these open-pose 3D objects. In particular, to make validation more compelling and not just limited to existing CLIP-based methods, we also pioneer the exploration of knowledge transfer based on Diffusion models. While the proposed solutions can serve as a new benchmark for open-pose 3D zero-shot classification, we discuss the complexities and challenges of this scenario that remain for further research development. The code is available publicly at https://github.com/weiguangzhao/Diff-OP3D

Digital Fingerprinting of Complex Liquids Using a Reconfigurable Multi-Sensor System with Foundation Models.

Combining chemical sensor arrays with machine learning enables designing intelligent systems to perform complex sensing tasks and unveil properties that are not directly accessible through conventional analytical chemistry. However, personalized and portable sensor systems are typically unsuitable for the generation of extensive data sets, thereby limiting the ability to train large models in the chemical sensing realm. Foundation models have demonstrated unprecedented zero-shot learning capabilities on various data structures and modalities, in particular for language and vision. Transfer learning from such models is explored by providing a framework to create effective data representations for chemical sensors and ultimately describe a novel, generalizable approach for AI-assisted chemical sensing. The translation of signals produced by remarkably simple and portable multi-sensor systems into visual fingerprints of liquid samples under test is demonstrated, and it is illustrated that how a pipeline incorporating pretrained vision models yields average classification accuracy in four unrelated chemical sensing tasks with limited domain-specific training measurements. This approach matches or outperforms expert-curated sensor signal features, thereby providing a generalization of data processing for ultimate ease-of-use and broad applicability to enable interpretation of multi-signal outputs for generic sensingapplications.

ChatGPT-4 extraction of heart failure symptoms and signs from electronic health records

BackgroundNatural language processing (NLP) can facilitate research utilizing data from electronic health records (EHRs). Large language models can potentially improve NLP applications leveraging EHR notes. The objective of this study was to assess the performance of zero-shot learning using Chat Generative Pre-trained Transformer 4 (ChatGPT-4) for extraction of symptoms and signs, and compare its performance to baseline machine learning and rule-based methods developed using annotated data. Methods and resultsFrom unstructured clinical notes of the national EHR data on the Veterans healthcare system, we extracted 1999 text snippets containing relevant keywords for heart failure symptoms and signs, which were then annotated by two clinicians. We also created 102 synthetic snippets that were semantically similar to snippets randomly selected from the original 1999 snippets. The authors applied zero-shot learning, using two different forms of prompt engineering in a symptom and sign extraction task with ChatGPT-4, utilizing the synthetic snippets. For comparison, baseline models using machine learning and rule-based methods were trained using the original 1999 annotated text snippets, and then used to classify the 102 synthetic snippets.The best zero-shot learning application achieved 90.6 % precision, 100 % recall, and 95 % F1 score, outperforming the best baseline method, which achieved 54.9 % precision, 82.4 % recall, and 65.5 % F1 score. Prompt style and temperature settings influenced zero-shot learning performance. ConclusionsZero-shot learning utilizing ChatGPT-4 significantly outperformed traditional machine learning and rule-based NLP. Prompt type and temperature settings affected zero-shot learning performance. These findings suggest a more efficient means of symptoms and signs extraction than traditional machine learning and rule-based methods.

Adaptive indefinite kernels in hyperbolic spaces

Learning embeddings in hyperbolic space has gained increasing interest in the community, due to its property of negative curvature, as a way of encoding data hierarchy. Recent works investigate the improvement of the representation power of hyperbolic embeddings through kernelization. However, existing developments focus on defining positive definite (pd) kernels, which may affect the intriguing property of hyperbolic spaces. This is due to the structures of hyperbolic spaces being modeled in indefinite spaces (e.g., Kreĭn space). This paper addresses this issue by developing adaptive indefinite kernels, which can better utilize the structures in the Kreĭn space. To this end, we first propose an adaptive embedding function in the Lorentz model and define indefinite Lorentz kernels (iLks) via the embedding function. Due to the isometric relationship between the Lorentz model and the Poincaré ball, these iLks are further extended to the Poincaré ball, resulting in the development of what are termed indefinite Poincaré kernels (iPKs). We evaluate the proposed indefinite kernels on a diversity of learning scenarios, including image classification, few-shot learning, zero-shot learning, person re-identification, knowledge distillation, etc. We show that the proposed indefinite kernels can bring significant performance gains over the baselines and enjoy better representation power from RKKSs than pd kernels.