Processing Tasks Research Articles

Leveraging ChatGPT for systematic reviews: how accurate is it for screening by title and abstract?

Abstract Background Systematic reviews are at the top of the evidence pyramid, but the whole process can be labor-intensive, especially in the screening by title and abstract of retrieved records. Large language models (LLMs), such as ChatGPT, are proficient in natural language processing tasks and offer potential for accelerating this phase. The aim of this study is to assess ChatGPT’s performance in screening records for an already published systematic review (doi: 10.1080/21645515.2023.2300848). Methods 1601 records with title and abstract were evaluated with ChatGPT 4, using two different prompts to instruct the model: one asking the model to classify each record as included or excluded (Prompt A), and one asking the model to rate each record from 1 to 5 based on inclusion confidence (Prompt B), to compute the model’s performance metrics. Results The review included 64 records after title and abstract screening, of which 18 after full text screening. Using records included by title and abstract as a reference, prompt B with a rating cut-off of 3 provided 82% sensitivity, 88% specificity and 99% negative predictive value (NPV). With articles included by full text as a reference, 100% sensitivity, 86% specificity, and 100% NPV. An 85% workload saving was reached. Prompts A showed higher workload savings (∼93%), high NPVs (∼100%) and specificity (∼96%), but lower sensitivity (62-72%). Conclusions Prompts with a rating cut-off of 3 achieved a better performance, and still relevant workload savings compared to binary classification prompts. These findings can inform prompt-engineering strategies to instruct LLMs and reach both high sensitivity and workload savings. Key messages • Systematic reviews provide evidence synthesis to guide policy and decision-making in healthcare and public health. Such process can be time-consuming, and need for timely evidence is often stringent. • LLMs can help reduce workload and speed up the title and abstract screening phase of a systematic review, which is considered a major bottleneck in the review process.

Self-Powered Dye-Sensitized Solar-Cell-Based Synaptic Devices for Multi-Scale Time-Series Data Processing in Physical Reservoir Computing.

Physical reservoir computing (PRC) using synaptic devices has attracted attention as a promising edge artificial intelligence device. To handle time-series data on various time scales, it is necessary to fabricate devices with the desired time scale. In this study, we fabricated a dye-sensitized solar-cell-based synaptic device with controllable time constants by changing the light intensity. This device showed synaptic features, such as paired-pulse facilitation and paired-pulse depression, in response to light intensity. Moreover, we found that the high computational performance of the time-series data processing task was achieved by changing the light intensity, even when the input pulse width was varied. In addition, the fabricated device can be used for motion recognition tasks. This study paves the way for realizing multiple time-scale PRC.

Analyzing Semantic Properties of Word Embeddings Using Eigenvectors

Dense word vectors have demonstrated their efficacy in several downstream natural language processing (NLP) tasks in recent years. Nevertheless, the interpretability of these embeddings' dimensions remains challenging. In this paper, we investigate how eigenvectors can reveal different semantic properties captured by word embedding models. Hence, we train word embeddings (e.g., Word2Vec) on English Wikipedia corpus to analyze the top eigenvectors, identify specific semantic properties (e.g., sentiment, formality) associated with each, and explore how these properties are encoded in the embedding space. This paper also discussed the limitations and potential benefits of this approach compared to other methods for analyzing word embeddings.

Robust projective measurements through measuring code-inspired observables

Quantum measurements are ubiquitous in quantum information processing tasks, but errors can render their outputs unreliable. Here, we present a scheme that implements a robust projective measurement through measuring code-inspired observables. Namely, given a projective POVM, a classical code, and a constraint on the number of measurement outcomes each observable can have, we construct commuting observables whose measurement is equivalent to the projective measurement in the noiseless setting. Moreover, we can correct t errors on the classical outcomes of the observables’ measurement if the classical code corrects t errors. Since our scheme does not require the encoding of quantum data onto a quantum error correction code, it can help construct robust measurements for near-term quantum algorithms that do not use quantum error correction. Moreover, our scheme works for any projective POVM, and hence can allow robust syndrome extraction procedures in non-stabilizer quantum error correction codes.

CPEQA: A Large Language Model Based Knowledge Base Retrieval System for Chinese Confidentiality Knowledge Question Answering

Large language models (LLMs) have exhibited remarkable performance on various natural language processing (NLP) tasks, particularly in the construction of the intelligent question-answering system. These systems, especially in specialized fields, usually rely on NLP through the retrieval of corpus and answering databases to efficiently provide accurate and concise answers. This paper focuses on the national confidentiality publicity and education field, aiming to address the dilemma of inaccurate knowledge retrieval in this field. Therefore, we design an intelligent confidentiality question-answering system CPEQA by comprehensively utilizing the LLMs platform and information retrieval technique. CPEQA is capable of providing professional answers to questions about Chinese confidentiality publicity and education raised by users. Additionally, we also integrate the conventional database retrieval technique and LLMs into the database query construction, enabling CPEQA to perform real-time queries and data analysis for both single-table and multi-table querying tasks. Through extensive experiments with generated query sentences, we show both methodological comparisons and empirical evaluations of CPEQA’s performance. Experimental results indicate that CPEQA has achieved competitive results on answering precision, recall rate and other metrics. Finally, we explore the challenges of the CPEQA system associated with these techniques and outline potential avenues for future research in this emerging field.

Topic aware probing: From sentence length prediction to idiom identification how reliant are neural language models on topic?

Abstract Transformer-based neural language models achieve state-of-the-art performance on various natural language processing tasks. However, an open question is the extent to which these models rely on word-order/syntactic or word co-occurrence/topic-based information when processing natural language. This work contributes to this debate by addressing the question of whether these models primarily use topic as a signal, by exploring the relationship between Transformer-based models’ (BERT and RoBERTa’s) performance on a range of probing tasks in English, from simple lexical tasks such as sentence length prediction to complex semantic tasks such as idiom token identification, and the sensitivity of these tasks to the topic information. To this end, we propose a novel probing method which we call topic-aware probing. Our initial results indicate that Transformer-based models encode both topic and non-topic information in their intermediate layers, but also that the facility of these models to distinguish idiomatic usage is primarily based on their ability to identify and encode topic. Furthermore, our analysis of these models’ performance on other standard probing tasks suggests that tasks that are relatively insensitive to the topic information are also tasks that are relatively difficult for these models.

Optimization of artificial intelligence in localized big data real-time query processing task scheduling algorithm

Optimization of artificial intelligence in localized big data real-time query processing task scheduling algorithm

Quantum-Based Maximum Likelihood Detection in MIMO-NOMA Systems for 6G Networks

As wireless networks advance toward the Sixth Generation (6G), which will support highly heterogeneous scenarios and massive data traffic, conventional computing methods may struggle to meet the immense processing demands in a resource-efficient manner. This paper explores the potential of quantum computing (QC) to address these challenges, specifically by enhancing the efficiency of Maximum-Likelihood detection in Multiple-Input Multiple-Output (MIMO) Non-Orthogonal Multiple Access (NOMA) communication systems, an essential technology anticipated for 6G. The study proposes the use of the Quantum Approximate Optimization Algorithm (QAOA), a variational quantum algorithm known for providing quantum advantages in certain combinatorial optimization problems. While current quantum systems are not yet capable of managing millions of physical qubits or performing high-fidelity, long gate sequences, the results indicate that QAOA is a promising QC approach for radio signal processing tasks. This research provides valuable insights into the potential transformative impact of QC on future wireless networks. This sets the stage for discussions on practical implementation challenges, such as constrained problem sizes and sensitivity to noise, and opens pathways for future research aimed at fully harnessing the potential of QC for 6G and beyond.

Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction

Syntactic annotation of corpora in the form of part-of-speech (POS) tags is a key requirement for both linguistic research and subsequent automated natural language processing (NLP) tasks. This problem is commonly tackled using machine learning methods, i.e., by training a POS tagger on a sufficiently large corpus of labeled data. While the problem of POS tagging can essentially be considered as solved for modern languages, historical corpora turn out to be much more difficult, especially due to the lack of native speakers and sparsity of training data. Moreover, most texts have no sentences as we know them today, nor a common orthography. These irregularities render the task of automated POS tagging more difficult and error-prone. Under these circumstances, instead of forcing the POS tagger to predict and commit to a single tag, it should be enabled to express its uncertainty. In this paper, we consider POS tagging within the framework of set-valued prediction, which allows the POS tagger to express its uncertainty via predicting a set of candidate POS tags instead of guessing a single one. The goal is to guarantee a high confidence that the correct POS tag is included while keeping the number of candidates small. In our experimental study, we find that extending state-of-the-art POS taggers to set-valued prediction yields more precise and robust taggings, especially for unknown words, i.e., words not occurring in the training data.

A Bibliometric Review of Large Language Models Research from 2017 to 2023

Large language models (LLMs), such as OpenAI's Generative Pre-trained Transformer (GPT), are a class of language models that have demonstrated outstanding performance across a range of natural language processing (NLP) tasks. LLMs have become a highly sought-after research area because of their ability to generate human-like language and their potential to revolutionize science and technology. In this study, we conduct bibliometric and discourse analyses of scholarly literature on LLMs. Synthesizing over 5,000 publications, this article serves as a roadmap for researchers, practitioners, and policymakers to navigate the current landscape of LLMs research. We present the research trends from 2017 to early 2023, identifying patterns in research paradigms and collaborations. We start with analyzing the core algorithm developments and NLP tasks that are fundamental in LLMs research. We then investigate the applications of LLMs in various fields and domains, including medicine, engineering, social science, and humanities. Our review also reveals the dynamic, fast-paced evolution of LLMs research. Overall, this article offers valuable insights into the current state, impact, and potential of LLMs research and its applications.

Sinusoidal PWM Generation for 3 Phase Inverter and RPM Measurement using Hercules TMS570LC43xx Launchpad

This project focuses on implementing a 3 phase Sinusoidal PWM generation using the Hercules TMS570LC43xx Launchpad Development Kit (Launchpad). The primary objective is to generate synchronized Sinusoidal nature PWM signals using the onboard High-End Timer (HET) and Enhanced Pulse Width Modulation (ePWM) module, which can be given to the Inverter for conversion of Direct current (DC) power into Alternating current (AC) power. This conversion is essential in various applications where AC power is required but the power source provides DC power. It is used in Solar photovoltaic (PV) systems and other renewable energy installations. These systems generate power suitable for powering household appliances or feeding into the electrical grid. Itis alsoused in electric vehicles (EVs) to drive the electric motor with variable speed. For verification of the wave nature, we have used an external lowpass filter (LPF) to transform the dynamic PWM signals into sinusoidal waveforms, ensuring compatibility with various applications like Inverters which can be further used in equipment and machinery such as Brushless DC motors, pumps and compressors. With the addon functionality to control the signal’s frequency which will be given to the Inverter to control the speed of the motor. Additionally, the project incorporates RPM measurement of the motor using an optical encoder setup interfaced with the Enhanced Quadrature Encoder Pulse (eQEP) module on the Launchpad. This feature enables accurate measurement of rotational speeds, position and Revolution per minute (RPM), enhancing the functionality of the system in real world applications like the speed of conveyor belts and other automated transport systems. Through successful implementation, this project demonstrates wide control for the Inverter, achieving reliable synchronized 3 phase signals with its variable speed having 120-degree phase shift signals alongside precise RPM measurement. The project highlights the Launchpad’s capabilities in handling complex signal processing tasks essential for modern power electronics applications. Looking forward, this project establishes a foundation for future enhancements and innovations in power electronics.

Single-Document Abstractive Text Summarization: A Systematic Literature Review

Abstractive text summarization is a task in natural language processing that automatically generates the summary from the source document in a human-written form with minimal loss of information. Research in text summarization has shifted towards abstractive text summarization due to its challenging aspects. This study provides a broad systematic literature review of abstractive text summarization on single-document summarization to gain insights into the challenges, widely used datasets, evaluation metrics, approaches, and methods. This study reviews research articles published between 2011 and 2023 from popular electronic databases. In total, 226 journal and conference publications were included in this review. The in-depth analysis of these papers helps researchers understand the challenges, widely used datasets, evaluation metrics, approaches, and methods. This paper identifies and discusses potential opportunities and directions, along with a generic conceptual framework and guidelines on abstractive summarization models and techniques for research in abstractive text summarization.

Using Generative Artificial Intelligence in Exams: A Research on KPSS with ChatGPT

ChatGPT is a language model developed by OpenAI that uses generative artificial intelligence (GenAI) algorithms for natural language processing (NLP) tasks. It is a popular Chatbot due to its impressive ability to respond consistently to users' various prompts. In this study, the Public Personnel Selection Examination (KPSS) questions, organized by ÖSYM (Turkish Student Selection and Placement Center) every year, were asked to ChatGPT, and its performance was evaluated. The evaluation was made based on fundamental text questions asked in the last three years (2021-2023) in Biology, History, Geography, Turkish Language Literature, and Social Studies in KPSS. This study also aims to investigate the accuracy and reliability of the questions asked to ChatGPT in the field of KPSS educational sciences. According to the research results, ChatGPT's KPSS knowledge and interpretation skills are significantly lower than the university graduate candidates who took the exam. Some suggestions are included for better Chatbot performance, more effective prompt engineering and ethical use of these tools.

Social cognition in adults with neurofibromatosis type 1.

Adult patients with the genetic disease neurofibromatosis type 1 (NF1) frequently report social difficulties. To date, however, only two studies have explored whether these difficulties are caused by social cognition deficits, and these yielded contradictory data. The aim of the present study was to exhaustively assess social cognition abilities (emotion, theory of mind, moral reasoning, and social information processing) in adults with NF1, compared with a control group, and to explore links between social cognition and disease characteristics (mode of inheritance, severity, and visibility). We administered a social cognition battery to 20 adults with NF1 (mean age = 26.5 years, SD = 7.4) and 20 healthy adults matched for sociodemographic variables. Patients scored significantly lower than controls on emotion, theory of mind, moral reasoning, and social information processing tasks. No effects of disease characteristics were found. These results appear to confirm that adults with NF1 have a social cognition weaknesses that could explain, at least in part, their social difficulties, although social abilities are not all impaired to the same extent. Regarding the impact of the disease characteristics, the patient sample seemed slightly insufficient for the power analyses performed. Thus, this exploratory study should form the basis of further research, with the objective of replicating these results with larger and more appropriately matched samples.

Relationship Between Socioeconomic Status and Basic Auditory Processing in Young Adults.

The current study aimed at investigating the relationship between socioeconomic status (SES) and basic auditory processing in young adults with normal hearing. Specifically, we sought to determine whether SES metrics, including parental education, parental occupation, and individual education attainment, influenced performance on tasks of basic auditory processing. Secondarily, we also aimed at understanding the interactive effects of SES and working memory on basic auditory processing. A total of 38 young adults were included in the study. Self-reported SES measures were recorded through Likert scales, and auditory processing measures included tests for temporal fine structure (TFS) and spectrotemporal sensitivity (STS). Pearson correlation analyses were conducted to probe the relationships between SES indicators and auditory processing performance. Multiple linear regressions were conducted to understand the interactive effects of SES and working memory on auditory processing performance. The analysis revealed no significant correlation between SES measures and basic auditory processing performance across TFS and STS tasks. Furthermore, analysis investigating the interaction between SES and working memory also yielded nonsignificant results. These findings indicate that, within this specific age cohort, SES information may not be indispensable for assessing basic auditory processing. Our study suggests that SES may not significantly influence basic auditory processing in young adults with normal hearing. However, further research with larger sample sizes and longitudinal designs is warranted to confirm these findings and explore potential age-related differences in the impact of SES on auditory processing across the lifespan.

Exploring noise-induced techniques to strengthen deep learning model resilience

AbstractIn artificial intelligence, combating overfitting and enhancing model generalization is crucial. This research explores innovative noise-induced regularization techniques, focusing on natural language processing tasks. Inspired by gradient noise and Dropout, this study investigates the interplay between controlled noise, model complexity, and overfitting prevention. Utilizing long short-term memory and bidirectional long short term memory architectures, this study examines the impact of noise-induced regularization on robustness to noisy input data. Through extensive experimentation, this study shows that introducing controlled noise improves model generalization, especially in language understanding. This contributes to the theoretical understanding of noise-induced regularization, advancing reliable and adaptable artificial intelligence systems for natural language processing.

Enhancing Recommendation Diversity by Re-ranking with Large Language Models

Recommender Systems (RS) should provide diverse recommendations, not just relevant ones. Diversity helps handle uncertainty and offers users meaningful choices. The literature proposes various methods to improve diversity, most notably by re-ranking and selecting from a larger set of candidate recommendations. Driven by promising insights from the literature on how to incorporate versatile Large Language Models (LLMs) into the RS pipeline, in this paper we show how LLMs can be used for diversity re-ranking. We prompt LLMs to generate a diverse ranking from a candidate ranking using various prompt templates with different re-ranking instructions in a zero-shot fashion. We conduct experiments testing state-of-the-art LLMs from the GPT and Llama families. We compare their re-ranking capabilities with random re-ranking and various traditional re-ranking methods from the literature. We open-source the code of our experiments for reproducibility. Our findings suggest that the trade-offs (in terms of performance and costs, among others) of LLM-based re-rankers are superior to those of random re-rankers but, as yet, inferior to the ones of traditional re-rankers. However, because LLMs exhibit improved performance on many natural language processing and recommendation tasks and lower inference costs, we can expect LLM-based re-ranking to become more competitive soon.

Testing and Evaluation of Health Care Applications of Large Language Models

Large language models (LLMs) can assist in various health care activities, but current evaluation approaches may not adequately identify the most useful application areas. To summarize existing evaluations of LLMs in health care in terms of 5 components: (1) evaluation data type, (2) health care task, (3) natural language processing (NLP) and natural language understanding (NLU) tasks, (4) dimension of evaluation, and (5) medical specialty. A systematic search of PubMed and Web of Science was performed for studies published between January 1, 2022, and February 19, 2024. Studies evaluating 1 or more LLMs in health care. Three independent reviewers categorized studies via keyword searches based on the data used, the health care tasks, the NLP and NLU tasks, the dimensions of evaluation, and the medical specialty. Of 519 studies reviewed, published between January 1, 2022, and February 19, 2024, only 5% used real patient care data for LLM evaluation. The most common health care tasks were assessing medical knowledge such as answering medical licensing examination questions (44.5%) and making diagnoses (19.5%). Administrative tasks such as assigning billing codes (0.2%) and writing prescriptions (0.2%) were less studied. For NLP and NLU tasks, most studies focused on question answering (84.2%), while tasks such as summarization (8.9%) and conversational dialogue (3.3%) were infrequent. Almost all studies (95.4%) used accuracy as the primary dimension of evaluation; fairness, bias, and toxicity (15.8%), deployment considerations (4.6%), and calibration and uncertainty (1.2%) were infrequently measured. Finally, in terms of medical specialty area, most studies were in generic health care applications (25.6%), internal medicine (16.4%), surgery (11.4%), and ophthalmology (6.9%), with nuclear medicine (0.6%), physical medicine (0.4%), and medical genetics (0.2%) being the least represented. Existing evaluations of LLMs mostly focus on accuracy of question answering for medical examinations, without consideration of real patient care data. Dimensions such as fairness, bias, and toxicity and deployment considerations received limited attention. Future evaluations should adopt standardized applications and metrics, use clinical data, and broaden focus to include a wider range of tasks and specialties.

How can geostatistics help us understand deep learning? An exploratory study in SAR-based aircraft detection

Deep Neural Networks (DNNs) have garnered significant attention across various research domains due to their impressive performance, particularly Convolutional Neural Networks (CNNs), known for their exceptional accuracy in image processing tasks. However, the opaque nature of DNNs has raised concerns about their trustworthiness, as users often cannot understand how the model arrives at its predictions or decisions. This lack of transparency is particularly problematic in critical fields such as healthcare, finance, and law, where the stakes are high. Consequently, there has been a surge in the development of explanation methods for DNNs. Typically, the effectiveness of these methods is assessed subjectively via human observation on the heatmaps or attribution maps generated by eXplanation AI (XAI) methods. In this paper, a novel GeoStatistics Explainable Artificial Intelligence (GSEAI) framework is proposed, which integrates spatial pattern analysis from Geostatistics with XAI algorithms to assess and compare XAI understandability. Global and local Moran’s I indices, commonly used to assess the spatial autocorrelation of geographic data, assist in comprehending the spatial distribution patterns of attribution maps produced by the XAI method, through measuring the levels of aggregation or dispersion. Interpreting and analyzing attribution maps by Moran’s I scattergram and LISA clustering maps provide an accurate global objective quantitative assessment of the spatial distribution of feature attribution and achieves a more understandable local interpretation. In this paper, we conduct experiments on aircraft detection in SAR images based on the widely used YOLOv5 network, and evaluate four mainstream XAI methods quantitatively and qualitatively. By using GSEAI to perform explanation analysis of the given DNN, we could gain more insights about the behavior of the network, to enhance the trustworthiness of DNN applications. To the best of our knowledge, this is the first time XAI has been integrated with geostatistical algorithms in SAR domain knowledge, which expands the analytical approaches of XAI and also promotes the development of XAI within SAR image analytics.

Federated learning-based natural language processing: a systematic literature review

Federated learning (FL) is a decentralized machine learning (ML) framework that allows models to be trained without sharing the participants’ local data. FL thus preserves privacy better than centralized machine learning. Since textual data (such as clinical records, posts in social networks, or search queries) often contain personal information, many natural language processing (NLP) tasks dealing with such data have shifted from the centralized to the FL setting. However, FL is not free from issues, including convergence and security vulnerabilities (due to unreliable or poisoned data introduced into the model), communication and computation bottlenecks, and even privacy attacks orchestrated by honest-but-curious servers. In this paper, we present a systematic literature review (SLR) of NLP applications in FL with a special focus on FL issues and the solutions proposed so far. Our review surveys 36 recent papers published in relevant venues, which are systematically analyzed and compared from multiple perspectives. As a result of the survey, we also identify the most outstanding challenges in the area.