Design and implementation of an end-to-end AI-driven colonoscopy recall workflow at scale.

Background and Aims Large language models (LLMs) have gained significant attention in the field of natural language processing (NLP), marking a shift from traditional techniques like Term Frequency-Inverse Document Frequency (TF-IDF). We developed a traditional NLP model to predict arteriovenous fistula (AVF) failure within next 30 days using clinical notes. The goal of this analysis was to investigate whether LLMs would outperform traditional NLP techniques, specifically in the context of predicting AVF failure within the next 30 days using clinical notes. Method We defined AVF failure as the change in status from active to permanently unusable status or temporarily unusable status. We used data from a large kidney care network from January 2021 to December 2021. Two models were created using LLMs and traditional TF-IDF technique. We used “distilbert-base-uncased”, a distilled version of BERT base model [1], and compared its performance with traditional TF-IDF-based NLP techniques. The dataset was randomly divided into 60% training, 20% validation and 20% test dataset. The test data, comprising of unseen patients’ data was used to evaluate the performance of the model. Both models were evaluated using metrics such as area under the receiver operating curve (AUROC), accuracy, sensitivity, and specificity. Results The incidence of 30 days AVF failure rate was 2.3% in the population. Both LLMs and traditional showed similar overall performance as summarized in Table 1. Notably, LLMs showed marginally better performance in certain evaluation metrics. Both models had same AUROC of 0.64 on test data. The accuracy and balanced accuracy for LLMs were 72.9% and 59.7%, respectively, compared to 70.9% and 59.6% for the traditional TF-IDF approach. In terms of specificity, LLMs scored 73.2%, slightly higher than the 71.2% observed for traditional NLP methods. However, LLMs had a lower sensitivity of 46.1% compared to 48% for traditional NLP. However, it is worth noting that training on LLMs took considerably longer than TF-IDF. Moreover, it also used higher computational resources such as utilization of graphics processing units (GPU) instances in cloud-based services, leading to higher cost. Conclusion In our study, we discovered that advanced LLMs perform comparably to traditional TF-IDF modeling techniques in predicting the failure of AVF. Both models demonstrated identical AUROC. While specificity was higher in LLMs compared to traditional NLP, sensitivity was higher in traditional NLP compared to LLMs. LLM was fine-tuned with a limited dataset, which could have influenced its performance to be similar to that of traditional NLP methods. This finding suggests that while LLMs may excel in certain scenarios, such as performing in-depth sentiment analysis of patient data for complex tasks, their effectiveness is highly dependent on the specific use case. It is crucial to weigh the benefits against the resources required for LLMs, as they can be significantly more resource-intensive and costly compared to traditional TF-IDF methods. This highlights the importance of a use-case-driven approach in selecting the appropriate NLP technique for healthcare applications.

Natural Language Processing in Nephrology.

RATIONALE: Accurate text interpretation using traditional natural language processing (NLP) approaches has historically been a roadblock for incorporating text-based findings into clinical decision support tools. Advances in NLP, most notably large language models (LLMs), have unlocked the ability to identify clinically relevant results from unstructured text reports. Here we apply the open-source Llama-3 LLM to chest radiograph reports and evaluate the accuracy of Llama-3 at labeling radiology text reports for features relevant to creating pneumonia severity clinical decision support. METHODS: Chest radiograph reports were obtained from patients with an ICD-10 primary diagnosis of pneumonia present on admission (J9-J18) who were admitted to a single center in New York City from January 2017 to September 2019. Llama-3 was run locally using the Ollama python package to evaluate a random selection of 200 radiology reports. The beginning of all prompts included an initial text to the LLM setting the context. These initial instructions were created with trial-and-error tuning. A python script prompted Llama-3 with three questions for each radiology report: “Does the chest x-ray show a consolidation?”; “How many sides of the lungs show evidence of pneumonia?”; and “Does the chest x-ray show evidence of an atypical pneumonia?”; The LLM was instructed to answer either a “Yes” or “No” for the first prompt; “No sides,” “One side,” or “Two sides” for the second prompt; and “Yes,” “No,” or “Maybe” for the third prompt. To evaluate the performance of the LLM, two clinicians reviewed the radiology reports and answered the above questions. Percent agreement and Cohen's Kappa were calculated between the clinicians. Any disagreements were arbitrated by a third clinician, with the resulting clinician labels used as the reference for comparison to Llama-3. RESULTS: The percent agreement and Cohen's Kappa for the initial two clinician reviewers and between Llama-3 and the clinician reference is shown in Table 1. CONCLUSIONS: Clinicians demonstrated moderate to good agreement at labeling features from radiology reports while Llama-3 demonstrated moderate to poor agreement, depending on the feature. Radiology reports have some ambiguity, and even clinicians could not achieve perfect agreement labeling features. In the future, further tuning of the LLM prompt and clarifying the questions may improve performance. Forthcoming new model releases will also likely improve performance. Llama-3 has potential for labeling features from radiology reports for a variety of potential uses, including creating clinical decision support and predictive modeling.

1557 Background: Patients with prostate cancer are diagnosed through a prostate needle biopsy (PNB). Information contained in PNB pathology reports is critical for informing clinical risk stratification and treatment; however, patient comprehension of PNB pathology reports is low, and formats vary widely by institution. Natural language processing (NLP) models trained to automatically extract key information from unstructured PNB pathology reports could be used to generate personalized educational materials for patients in a scalable fashion and expedite the process of collecting registry data or screening patients for clinical trials. As proof of concept, we trained and tested four NLP models for accuracy of information extraction. Methods: Using 403 positive PNB pathology reports from over 80 institutions, we converted portable document formats (PDFs) into text using the Tesseract optical character recognition (OCR) engine, removed protected health information using the Philter open-source tool, cleaned the text with rule-based methods, and annotated clinically relevant attributes as well as structural attributes relevant to information extraction using the Brat Rapid Annotation Tool. Text pre-processing for classification and extraction was done using Scispacy and rule-based methods. Using a 75:25 train:test split (N = 302, 101), we tested conditional random field (CRF), support vector machine (SVM), bidirectional long-short term memory network (Bi-LSTM), and Bi-LSTM-CRF models, reserving 46 training reports as a validation subset for the latter two models. Model-extracted variables were compared with values manually obtained from the unprocessed PDF reports for clinical accuracy. Results: Clinical accuracy of model-extracted variables is reported in the Table. CRF was the highest performing model, with accuracies of 97% for Gleason grade, 82% for percentage of positive cores ( < 50% vs. ≥50%), 90% for perineural or lymphovascular invasion, and 100% for presence of non-acinar carcinoma histology. On manual review of inaccurate results, model performance was limited by PDF image quality, errors in OCR processing of tables or columns, and practice variability in reporting number of biopsy cores. Conclusions: Our results demonstrate successful proof of concept for the use of NLP models in accurately extracting information from PNB pathology reports, though further optimization is needed before use in clinical practice.[Table: see text]

In the context of the BiblIndex project, which is an online index of biblical textual reuses by the Church Fathers, intrabiblical intertextuality must be considered to better understand the underlying basis of the Church Fathers’ thought. This paper examines the reuse of Psalmic texts in the New Testament as a test case for experimenting with a detection tool that uses traditional natural language processing (NLP) methods exclusively. Biblical verses are compared using similarity measures based on various NLP operations, such as tokenization, lemmatization, part-of-speech tagging, stop word filtering and synset assignment. Textometric measures provide a framework for the numerical assessment of grammatical, lexical and semantic similarities between textual units. We demonstrate the efficiency of this reproducible method, which does not involve a ‘black box’ effect, for detecting and characterizing literal quotations and a significant range of echoes.

This research presents the development and evaluation of a prototype AI-powered glasses attachment designed to enhance the daily mobility and independence of individuals with visual impairments. The work addresses the significant challenges faced by millions who experience limitations in navigating, recognising objects, and accessing information. It aims to contribute to the field of assistive technologies by creating a cost-effective and versatile solution. The project emphasises the importance of designing inclusive technology that is not only functional but also user-friendly and accessible to all. The system has several strengths:* Compact and portable design making it a discrete companion in daily life.* Versatile functionality combining object/text recognition with speech interaction.* Cost-effective approach making it accessible to a broader range of users.The core of the prototype integrates real-time object and text recognition, along with speech interaction capabilities. The system uses image processing algorithms, particularly convolutional neural networks (CNNs) to identify objects. Optical character recognition (OCR) is implemented to transform printed text into digital formats, enabling text-to-speech functionality. The prototype also incorporates a natural language processing (NLP) powered chatbot, facilitating spoken interaction and information retrieval. The project involved a thorough analysis of user requirements, focusing on the needs of individuals with varying degrees of visual impairment. This included understanding the challenges faced by people with different levels of sight loss, from moderate to complete blindness. The study also highlighted the importance of intuitive design and accessibility. The hardware design incorporates a compact ESP32-S3 microcontroller coupled with an OV5640 camera module, chosen for their balance of performance and power efficiency. The software is developed using Python for data processing and C++ for the microcontroller. Cloud-based AI services, including Google Cloud Text-to-Speech API and OpenAI's GPT API and Whisper API, are used for text-to-speech, object and text recognition, and speech recognition, respectively. The 3D-printed enclosure for the attachment was designed with several key considerations, including compactness, component integration, and user comfort. Key aspects of the hardware design are:* Component Layout: The enclosure was designed to house the ESP32 microcontroller, the OV5640 camera module, and a TTP223B touch sensor. The internal structure includes a separate compartment for the camera with a heatsink to manage its operating temperature.* Touch Sensor Integration: The touch sensor is located on the underside of the enclosure, designed with a concave depression for easy access and covered with a rubber membrane. This placement allows for user interaction and control of the system’s functions.* Size and Weight: The prototype has a weight of 23 grams. Its design aimed to be as small as possible whilst being able to accommodate the necessary hardware.The conducted functional tests focused on the accuracy of text and object recognition, and the efficacy of the chatbot. The text and object recognition performed well, however, issues with the camera quality, particularly in low-light conditions, limited the system's performance with smaller details and text. The speech interaction, while functional, encountered some difficulties with complex questions. The prototype is promising, future development should focus on addressing its limitations. Potential improvements include an automated configuration process, the integration of a navigation system and a more robust camera. Enhancing the user interface with a more tactile feedback system could improve accessibility.

Graph Neural Networks (GNNs) have become increasingly prominent in Natural Language Processing (NLP) due to their ability to model intricate relationships and contextual connections between texts. Unlike traditional NLP methods, which typically process text linearly, GNNs utilize graph structures to represent the complex relationships between texts more effectively. This capability has led to significant advancements in various NLP applications, such as social media interaction analysis, sentiment analysis, text classification, and information extraction. Notably, GNNs excel in scenarios with limited labeled data, often outperforming traditional approaches by providing deeper, context-aware solutions. Their versatility in handling different data types has made GNNs a popular choice in NLP research. In this study, we thoroughly explored the application of GNNs across various NLP tasks, demonstrating their advantages in understanding and representing text relationships. We also examined how GNNs address traditional NLP challenges, showcasing their potential to deliver more meaningful and accurate results. Our research underscores the value of GNNs as a potent tool in NLP and suggests future research directions to enhance their applicability and effectiveness further.

Telugu is one of South India’s oldest languages. It has a complicated orthography with various different character shapes. For so many decades, offline character recognition has always been a popular area of study. Handwriting segmentation and recognition are difficult, which has stimulated the interest of industry and academic researchers. Methods for recognizing handwriting have become much more prominent in recent years. Until now, the connected component approach, vertical and horizontal profiles, and other approaches were presented in the literature for the segmentation of printed character's work. The selection of the best discriminative features is an important concern in character recognition. Various statistical and structural features, as well as various combinations of them, are discussed in the research work. Researchers used ANN, SVM, KNN, and CNN to classify offline Telugu characters. The purpose of this article is to closely analyze various feature extraction methods and classification models to understand the problem and challenges encountered by earlier research. This identification is meant to provide numerous recommendations for advancements and scope.Keywordsk-nearest neighbor (KNN)Support vector machines (SVM)Convolution neural networks (CNN)Optical character recognition (OCR)Character recognition (CR)Natural language processing (NLP)Connected component (CC)Histogram of oriented gradients (HOG)Fast Fourier transform (FFT)Discrete cosine transform (DCT)Discrete wavelet transform (DWT)Principal component analysis (PCA)Fourier transform (FT)Particle swarm optimization (PSO)Differential evolution (DE)

Background Accurate prediction of intravenous (IV) fluid utilization in emergency departments (ED) is essential for optimal clinical decision-making, resource allocation, and patient management. However, existing predictive models typically rely solely on structured clinical variables, overlooking the rich, contextual insights available in unstructured patient narratives. This study aimed to develop and evaluate predictive models that integrate structured clinical data with natural language processing (NLP)-derived embeddings from unstructured patient narratives to enhance prediction accuracy for IV fluid administration in ED settings. Methods We analyzed a large dataset from the National Hospital Ambulatory Medical Care Survey—Emergency Department (NHAMCS-ED, n = 13,115), comprising both structured patient demographics and clinical variables, alongside unstructured chief complaints. Patient narratives were processed using three distinct Natural Language Processing (NLP) techniques: CountVectorizer, Word2Vec embeddings, and pre-trained GPT-2 embeddings. An early fusion strategy was employed, concatenating structured and unstructured features before feeding them into Logistic Regression (LR) and Gradient Boosting Classifier (GBC) models. Model performance was evaluated using 5-fold cross-validation, with the area under the receiver operating characteristic curve (AUC) as the primary metric. Results The integrated models, which combined structured clinical variables with NLP features from patient narratives, consistently demonstrated superior predictive performance over models using either data type alone. We compared several NLP techniques, including CountVectorizer, Word2Vec, and pre-trained Generative Pre-trained Transformer 2 (GPT-2) embeddings. Unexpectedly, the integrated models using traditional NLP features, particularly CountVectorizer, achieved the best performance, reaching an AUC of 0.786 with the Gradient Boosting Classifier (GBC) and 0.781 with logistic regression. These results outperformed the integrated model using GPT-2 embeddings (GBC AUC = 0.772). This suggests that for the short, keyword-driven patient complaints found in the emergency department, frequency-based methods may be more effective at extracting predictive signals. Conclusions Our findings confirm that integrating patient narratives with structured clinical data significantly enhances predictive accuracy for IV fluid utilization. This multimodal approach, particularly when leveraging traditional NLP models like CountVectorizer, provides a robust framework for improving clinical decision-making, resource allocation, and patient care in the ED. The study highlights the critical importance of aligning NLP model choice with the specific characteristics of clinical text data.

IntroductionDiabetes management increasingly relies on telehealth platforms in which patients generate structured and unstructured data. This unstructured data, in the form of free-text notes often contain additional information beyond the structured data. Extracting this information can enhance patient profiles and optimize treatment. In particular, the extraction of physical activity information from these notes is considered important. This study evaluates rule-based/regex algorithms and a locally deployed Mistral LLM for physical activity information extraction and data augmentation, with their performances benchmarked against a state-of-the-art GPT-4.1.MethodsData from 943 patients collected over 12 years in the DiabMemory system, supplemented by 100 synthetic notes, were analyzed. Patients’ privacy was preserved by applying a free text pseudonymization algorithm to all notes and by using locally deployed approaches, thereby avoiding third-party cloud services. Three tasks were conducted: (1) extraction of physical activity (PA) data from free-text notes using regex and a locally deployed Mistral LLM, (2) integration of extracted data with structured activity records using a rule-based approach and the local Mistral LLM, and (3) benchmarking local approaches against GPT-4.1 based on the synthetic notes.ResultsBoth local methods achieved strong performance in task 1, with minimum F1-scores of 0.84. In task 2, rule-based augmentation (F1 = 0.73) surpassed the Mistral LLM (F1 = 0.37). Task 3 showed GPT-4.1 outperforming the local LLM but not consistently surpassing regex. The rule-based algorithms also required substantially less computation time than either LLM.DiscussionThe regex algorithm achieved superior accuracy and efficiency but required extensive dataset-specific development, while prompt engineering for the LLM required less knowledge and the development time for regex exceeded that of LLM prompt engineering. Findings of this work generally align with prior studies but are limited by the rather small test set and use of synthetic data.ConclusionsLocal NLP approaches can enhance structured PA data in diabetes telehealth. Rule-based algorithms remain a strong option where computational resources are limited, though future work should validate these findings on larger and more diverse datasets.

Drilling capital expenditure represents significant portion of any oil and gas project. Drilling investment accounts to about 40% of the cost of the well. Therefore monitoring the health of equipments and processess associated with rigs are critical in bringing down both the cost of the well and the Non Productive Time (NPT). At the moment there are several Key Performance Indicators (KPIs) to monitor NPT but they are reactive in nature. Proactive measures can be taken to avoid NPT and reduce Invisible Lost Time (ILT) by exploiting the power of big data. The scope of this work is to create a framework to predict NPT and ILT causes from massive degree of unstructured data collected during the drilling operations. KPIs and performance values come from numerical metrics. In addition, the industry has also gathered an enormous amount of unstructured data from drilling and other reports. In many industries, Natural Language Processing (NLP) has been used to create value from unstructured data. Upstream reports are not exactly natural language ready reports. Our approach on generating value from these reports has two major steps (1) transforming NLP challenge to Technology language processing (TLP) in drilling context, and (2) classification of NPT causes. The two-step approach mentioned above results in a framework for mining unstructured reports to detect anomalies like symptoms, events, and actions. A modified NLP, referred here as TLP in the drilling context is used to achieve the first set of data extraction. Furthermore, a predictive model for classification of NPT and ILT causes is performed on the extracted data. The accuracy of these predictive models is an on-going effort, as it is dependent upon the transformation and detection of complex technical language that is inconsistent due to multiple reasons. However, this framework provides a foundation for predicting NPT / ILT during different phases of drilling operations. Use cases that predict attributes related to hole packoff or equipment failure shows the accuracy in the range of 50% to 70% due to a limited amount of extracted technical terms that are relevant. The framework calls for continual improvement of employing deep learning algorithm based on enhanced recovery of correlated technical terms across unstructured reports. While there are many attempts in using NLP for mining data in E&P sector, in this paper we present a framework for mining unstructured reports that leverage contextual Technology Language processing a step ahead NLP. Our framework incorporates the use of various machine learning algorithms to detect multiple attributes necessary to calculate NPT and ILT, and guides on how to enhance the predictive model.

Patients with rare diseases often struggle to find accurate medical information, and large language model (LLM)-based chatbots may help meet this need. However, evaluating LLM-generated free-text answers typically requires physician review, which is time-consuming and difficult to scale. This study compared traditional natural language processing (NLP) metrics to emerging LLM-based evaluation approaches for assessing answer quality in the context of Complex Lymphatic Anomalies (CLAs). We compiled 25 common patients' questions about CLAs and generated 175 responses to these questions from seven LLMs. Three expert physicians scored these responses for accuracy. We compared these physician-assigned scores with automated scores, generated by four NLP sentence similarity metrics (BLEU, ROUGE, METEOR, BERTScore) and six LLM evaluators (GPT-4, GPT-4o, Qwen3-32B, DeepSeek-R1-14B, Gemma3-27B, LLaMA3.3-70B). We examined both LLM-based scoring with and without reference answers (reference-guided vs reference-free). We calculated Spearman, Phi, and Kendall's Tau correlation coefficients to assess alignment between automated and physician-assigned scores. LLM-based evaluation demonstrated stronger alignment with physician-assigned scores than NLP metrics. The reference-guided GPT-4 evaluator achieved the highest correlation with physician-assigned scores (ρ = 0.758), followed by GPT-4o (ρ = 0.727). NLP metrics showed weak to moderate correlations with physician-assigned scores (ρ = 0.240-0.403). Reference-guided scoring outperformed reference-free methods. Reference-guided LLM-based evaluation methods approximate expert physicians' judgment better than traditional NLP metrics, offering an effective, scalable approach for assessing LLM-generated responses to patient questions about rare disease. LLM-based evaluation, particularly reference-guided scoring with GPT models, can support the scalable development and evaluation of LLM-based rare disease-specific chatbot systems.

Scientific growth is iterative, with existing knowledge serving as the foundation for new discoveries. Citations serve as the primary channel for information propagation in science, shaping which ideas and findings persist in the literature and which do not. While natural language processing (NLP) is increasingly used in citation context analysis, it is underutilized in studies that examine the actual scientific content of citations. In this pilot study, we explored how NLP can be used to track the propagation of scientific findings by replicating a prior citation context study that relied on manual extraction. We compared two approaches: a traditional NLP pipeline (named entity recognition and relation extraction) and a generative large language model (LLM). We formulated a two-step automated pipeline: (1) extracting findings from a reference paper and (2) mapping citation contexts to the findings they reference. Our findings indicate that LLMs are superior to traditional NLP techniques in both steps of the pipeline. However, they are also more prone to errors, mapping citation contexts to findings they do not reference. While the two-step automated pipeline was effective, integrating manual annotation of findings with LLM-based mapping of citation contexts yields the best results. To our knowledge, this study is one of the first to explore how NLP, particularly LLMs, can be leveraged to track the flow of information in science. Future research should further evaluate the application of LLMs and other NLP techniques on a larger scale to assess their effectiveness in supporting citation-focused scientometric and informetric studies.

This article outlines our practical approach to digitizing historical text sources via Optical Character Recognition (OCR) for subsequent Natural Language Processing (NLP) and corpus analysis. For this purpose we developed two processing pipelines based on pyFlow for parallel computing using Tesseract OCR for OCR and spaCy for NLP. To ensure that the software is developed in terms of reusability and sustainability, the importance of free and open source software and the use of Linux containers via Docker during the development process and within a production environment is also described. Details regarding OCR preprocessing steps, e.g., binarization are also discussed. Besides the software development aspects the article contains some learnings and best practices for end users on how to create high quality input images (avoiding noise, skew, etc.) for the OCR pipeline. By following these steps, OCR results can be significantly improved. For both, our OCR and NLP pipelines, the accuracies and respective error rates are discussed at the ennd of the corresponding chapters.

In the last decades, a huge number of documents has been digitised, before undergoing optical character recognition (OCR) to extract their textual content. This step is crucial for indexing the documents and to make the resulting collections accessible. However, the fact that documents are indexed through their OCRed content is posing a number of problems, due to the varying performance of OCR methods over time. Indeed, OCR quality has a considerable impact on the indexing and therefore the accessibility of digital documents. Named entities are among the most adequate information to index documents, in particular in the case of digital libraries, for which log analysis studies have shown that around 80% of user queries include a named entity. Taking full advantage of the computational power of modern natural language processing (NLP) systems, named entity recognition (NER) can be operated over enormous OCR corpora efficiently. Despite progress in OCR, resulting text files still have misrecognised words (or noise for short) which are harming NER performance. In this paper, to handle this challenge, we apply a spelling correction method to noisy versions of a corpus with variable OCR error rates in order to quantitatively estimate the contribution of post-OCR correction to NER. Our main finding is that we can indeed consistently improve the performance of NER when the OCR quality is reasonable (error rates respectively between 2% and 10% for characters (CER) and between 10% and 25% for words (WER)). The noise correction algorithm we propose is both language-independent and with low complexity.