Confidence In Predictions Research Articles

Future cities demand smart and equitable infrastructure resilience modeling perspectives

AbstractRisk-informed decisions that promote infrastructure resilience (or the ability to withstand, recover from, and adapt to stressors like natural hazards) require confident predictions of system performance now and into the future. We propose a perspective shift–one capable of handling uncertain and dynamic conditions, leveraging emerging observations from smart systems, and guided by demands for social equity. This shift requires collective efforts, but our future cities demand and deserve it.

Repairs and Breaks Prediction for Deep Neural Networks

With the increasing prevalence of software incorporating deep neural networks (DNNs), quality assurance for these software systems has become a crucial concern. To this end, various methods have been proposed to repair the misbehavior of DNNs by modifying their weights. However, these repair methods may not meet the developer's needs for a given dataset and model. In this study, we build prediction models for repair outcomes (i.e., repairs and breaks) to help determine whether the repair method is likely to work. By using our prediction models, developers and operators of DNNs can decide whether or not to apply a repair method, and if so, which method to use. Our prediction models utilize four metrics as explanatory metrics that represent the confidence or ambiguity in the DNN predictions. We experimented with four repair methods and 10 datasets. The experimental results demonstrate that our prediction models successfully select a repair method that meets developers’ needs in 16 out of 24 cases, resulting in an average time saving of 16.29% compared to the naive method. Based on these results, our prediction models can reduce costs for developers and operators when deciding whether to employ repair methods for real-world applications of DNNs.

Physics-Informed Data-Driven Approaches to State of Health Prediction of Maritime Battery Systems

Battery systems are increasingly being used for powering ocean going ships, and the number of fully electric or hybrid ships relying on battery power for propulsion and maneuvering is growing. In order to ensure the safety of such electric ships, it is of paramount importance to monitor the available energy that can be stored in the batteries, and classification societies typically require that the state of health (SOH) of the batteries can be verified by independent tests – annual capacity tests. However, this paper discusses physics-informed data-driven approaches to online diagnostics for state of health monitoring of maritime battery systems based on a combination of physical knowledge, physic-based models, insights from extensive characterization tests and operational sensor data collected from the batteries during actual operation. This represents an alternative approach to the annual capacity tests for electric ships that is found to be sufficiently robust and accurate under certain conditions. Previous attempts with purely data-driven models, including both cumulative and snapshot models, semi-supervised learning and simple models based on the state of charge did not achieve the required reliability and accuracy for them to be utilized in a ship classification perspective, as presented at previous PHM conferences. However, preliminary results from the physics-informed data-driven method presented in this paper indicate that it can be relied on for independent verification of state of health as an alternative to physical tests. It has been tested on battery cells cycled in laboratory degradation tests as well as on field returns from batteries onboard ships in service. Notwithstanding, further validation and verification of the method is recommended to further build confidence in the model predictions.

Validation of vibration reduction in barge-type floating offshore wind turbines with oscillating water columns through experimental and numerical analyses

Floating offshore wind turbines (FOWTs) are highly susceptible to vibrations caused by wind and sea wave oscillations, necessitating effective vibration reduction strategies to ensure stability and optimal performance. This study investigates the effectiveness of a barge-type FOWT integrated with oscillating water columns (OWCs) in reducing oscillations, particularly in rotational modes. A hybrid FOWT-OWCs system was designed, and its vibration mitigation capabilities were assessed through both numerical simulations and experimental tests. The numerical approach focused on controlling airflow in the OWCs, while the experimental tests validated these results under similar conditions. A strong agreement between the simulations and experiments was observed, particularly in reducing platform pitch oscillations, even under irregular wave conditions. The open OWC-based platform outperformed the closed design, reducing pitch angle oscillations from 17.51° to 14.38° for waves with a 10-s dominant frequency. Benchmark tests confirmed this trend, with the open moonpool-based platform achieving a reduction from 18.41° to 12.23°. These findings demonstrate the potential of OWCs to improve the stability and performance of FOWTs, with experimental validation providing confidence in the numerical predictions.

A probabilistic forecast for multi-year ENSO using Bayesian convolutional neural network

Abstract A robust El Niño Southern Oscillation (ENSO) prediction is essential for monitoring the global climate, regional monsoons, and weather extremes. Despite dedicated efforts spanning decades, the precise prediction of ENSO events through numerical modeling beyond a couple of seasonal lead times remains a daunting challenge. The advent of deep learning-based approaches marks a transformative era in climate and weather prediction. However, many machine learning-based studies attempting ENSO prediction are confined to singular estimates, lacking adequate quantification of uncertainty in learned parameters and overlooking the crucial need for a nuanced understanding of ENSO prediction confidence. Here, we introduce a deep learning-based Bayesian convolutional neural network model that provides robust probabilistic predictions for ENSO with a lead time of up to 9–10 months across all seasons. The Bayesian layers within the convolutional neural network maintain the capability to predict a distribution of learned parameters. Augmented with bias correction, our model reproduces the amplitude of the Niño 3.4 index with fidelity for lead up to 9–10 months. The inherent capacity for uncertainty modeling enhances the reliability of bayesian neural networks (BNNs), making them particularly valuable in operational services. This research holds substantial socio-economic implications as it enhances our forecasting capabilities and rigorously quantifies forecast uncertainties, providing valuable insights for planning and policymaking.

Short-Term Prediction Method for Gas Concentration in Poultry Houses Under Different Feeding Patterns

Ammonia (NH3) and carbon dioxide (CO2) are the main gases that affect indoor air quality and the health of the chicken flock. Currently, the environmental control strategy for poultry houses mainly relies on real-time temperature, resulting in lag and singleness. Indoor air quality can be improved by predicting the change in CO2 concentration and proposing an optimal control strategy. Combining the advantages of seasonal-trend decomposition using loess (STL), Granger causality (GC), long short-term memory (LSTM), and extreme gradient boosting (XGBoost), an ensemble method called the STL-GC-LSTM-XGBoost model is proposed. This model can set fast response prediction results at a lower cost and has strong generalization ability. The comparative analysis shows that the proposed STL-GC-LSTM-XGBoost model achieved high prediction accuracy, performance, and confidence in predicting CO2 levels under different environmental regulation modes and data volumes. However, its prediction accuracy for NH3 was slightly lower than that of the STL-GC-LSTM model. This may be due to the limited variability and regularity of the NH3 dataset, which likely increased model complexity and decreased predictive ability with the introduction of XGBoost. Nevertheless, in general, the proposed integrated model still provides a feasible approach for gas concentration prediction and health-related risk control in poultry houses.

Estimating and Modeling Pinus contorta Transpiration in a Montane Meadow Using Sap-Flow Measurements

This study quantifies the transpiration of encroached lodgepole pine (Pinus contorta var. murryana (Grev. & Balf.) Engelm.) in a montane meadow using pre-restoration sap-flow measurements. Lodgepole pine transpiration and its response to environmental variables were examined in Rock Creek Meadow (RCM), Southern Cascade Range, CA, USA. Sap-flow data from lodgepole pines were scaled to the meadow using tree survey data and then validated with MODIS evapotranspiration estimates for the 2019 and 2020 growing seasons. A modified Jarvis–Stewart model calibrated to 2020 sap-flow data analyzed lodgepole pine transpiration’s correlation with solar radiation, air temperature, vapor pressure deficit, and soil volumetric water content. Model validation utilized 2021 growing season sap-flow data. Calibration and validation employed a Markov Chain Monte Carlo (MCMC) approach through the DREAM(ZS) algorithm with a generalized likelihood (GL) function, enabling parameter and total uncertainty assessment. The model’s scaling was compared with simple scaling estimates. Average lodgepole pine transpiration at RCM ranged between 220.6 ± 25.3 and 393.4 ± 45.7 mm for the campaign (mid-July 2019 to mid-August 2020) and 100.2 ± 11.5 to 178.8 ± 20.7 mm for the 2020 partial growing season (April to mid-August), akin to MODIS ET. The model aligned well with observed normalized sap-velocity during the 2020 growing season (RMSE = 0.087). However, sap-velocity, on average, was underpredicted by the model (PBIAS = −6.579%). Model validation mirrored calibration in performance metrics (RMSE = 0.1233; PBIAS = −2.873%). The 95% total predictive uncertainty confidence intervals generated by GL-DREAM(ZS) enveloped close to the theoretically expected 95% of total observations for the calibration (94.5%) and validation (81.8%) periods. The performance of the GL-DREAM(ZS) approach and uncertainty assessment in this study shows promise for future MJS model applications, and the model-derived 2020 transpiration estimates highlight the MJS model utility for scaling sap-flow measurements from individual trees to stands of trees.

The effect of postfire regeneration pattern on soil respiration in the boreal forest of China

BackgroundAs the second largest carbon flux between the atmosphere and terrestrial ecosystems, soil respiration involves multiple components of ecosystem production. Revealing soil respiration in forests with different postfire regeneration patterns is critical for determining appropriate restoration strategies in response to increasing wildfire disturbances. Here, we examined the influence of five postfire regeneration patterns (L: Larix gmelinii monocultures, LB: L. gmelinii and Betula platyphylla mixed plantations, P: Pinus sylvestris var. mongolica monocultures, PB: P. sylvestris var. mongolica and B. platyphylla mixed plantations, N: naturally regenerated forests) on soil heterotrophic respiration (Rh) and total respiration (Rs).MethodsTrenching was implemented to monitor soil heterotrophic respiration. We used partial least squares path modeling methods to estimate the different environmental factors regulating soil respiration across forest types.ResultsThe results showed that forest type and season had significant effects on Rs and Rh. Rh was the dominant part of Rs for all forest types (68.84 ~ 90.20%). Compared to naturally regenerated forests, Rs and Rh under L, LB, and PB had higher rates (P < 0.05), while Rs and Rh under P had lower rates (P < 0.05). The temperature sensitivities of Rs under different forest types were 2.316 (L), 1.840 (LB), 1.716 (P), 1.665 (PB), and 2.096 (N).ConclusionsForests regenerated with artificial participation established their plant communities visibly faster than naturally regenerated forests. Mixed species plantation regeneration demonstrated an improvement in soil respiration compared to naturally regenerated forests but had a lower temperature sensitivity of soil respiration than their respective monocultures. Soil temperature and moisture dominated the influence factors on soil respiration throughout the broader seasonal shifts. However, for a single season, forest productivity and soil properties have a greater impact on soil respiration. This study extends our knowledge of the interaction mechanism between soil respiration and environmental variables in boreal forests and contributes to improving confidence in global carbon cycling model predictions.

Special applets for elucidating the difference between prediction intervals and confidence intervals—Why are arising ovals actually ellipses?

AbstractIn the last years special “ovals” appear increasingly often in diagrams and applets for discussing crucial items of statistical inference (when dealing with confidence intervals for an unknown probability p; approximation of the binomial distribution by the normal distribution; especially in German literature, see e.g. [Meyer, Stochastik in der Schule 2011; 31:18–22] or [Riemer, Statistik unterrichten. Eine handlungsorientierte Didaktik der Stochastik; 2023, p. 69ff]). These applets concerning confidence intervals aim at illustrating vividly the crucial differences (as we know, often vague for students) and connections between confidence intervals and prediction intervals. In English literature such applets seem to be not so well known, thus at the beginning, we describe how they work and highlight their high potential for teaching and learning. The fact that the mentioned ovals are, actually, ellipses is not important for stochastic purposes which is probably the reason why one can hardly find related explanations in the literature. Another aim of this paper is to explain the ellipse shape in a preferably elementary manner involving geometry, even spatial geometry.

Establishing a Relationship between In Vitro Potency in Cell-Based Assays and Clinical Efficacious Concentrations for Approved GLP-1 Receptor Agonists.

Background: Glucagon-like peptide-1 receptor agonists (GLP-1RAs) play an important role in the treatment of type 2 diabetes (T2D) and obesity. The relationship between efficacy and dosing regimen has been studied extensively for this class of molecules. However, a comprehensive analysis of the translation of in vitro data to in vivo efficacious exposure is still lacking. Methods: We collected clinical pharmacokinetics for five approved GLP-1RAs to enable the simulation of exposure profiles and compared published clinical efficacy endpoints (HbA1c and body weight) with in-house in vitro potency values generated in different cell-based assays. Additionally, we investigated the correlation with target coverage, expressed as a ratio between the steady state drug exposure and unbound potency, body weight, or HbA1c reduction in patients with T2D. Results: We found that the best correlation with in vivo efficacy was seen for in vitro potency data generated in cellular assays performed in the absence of any serum albumin or using ovalbumin. Residual variability was larger using in vitro potency data generated in endogenous cell lines or in the presence of human serum albumin. For the human receptor assay with no albumin, exposures above 100-fold in vitro EC50 resulted in >1.5% point HbA1c reduction, while a 5% BW reduction was related to approximately 3× higher exposures. A similar relationship was seen in the ovalbumin assay. Conclusions: Overall, the relationship established for in vitro potency and in vivo efficacy will help to increase confidence in human dose prediction and trial design for new GLP-1RAs in the discovery and early clinical phases.

Early-Exit Deep Neural Network - A Comprehensive Survey

Deep neural networks (DNNs) typically have a single exit point that makes predictions by running the entire stack of neural layers. Since not all inputs require the same amount of computation to reach a confident prediction, recent research has focused on incorporating multiple ”exits” into the conventional DNN architecture. Early-exit DNNs are multi-exit neural networks that attach many side branches to the conventional DNN, enabling inference to stop early at intermediate points. This approach offers several advantages, including speeding up the inference process, mitigating the vanishing gradients problems, reducing overfitting and overthinking tendencies. It also supports DNN partitioning across devices and is ideal for multi-tier computation platforms such as edge computing. This paper decomposes the early-exit DNN architecture and reviews the recent advances in the field. The study explores its benefits, designs, training strategies, and adaptive inference mechanisms. Various design challenges, application scenarios, and future directions are also extensively discussed.

Revisiting the functional annotation of TriTryp using sequence similarity tools

Trypanosomatids are the causative agents of deadly diseases in humans and livestock. Given the high phylogenetic distance of trypanosomatids from model organisms, these organisms have ample unannotated genes. Manual functional annotation is time-consuming, highlighting the importance of automated functional annotation tools. The development of automated functional tools is a hot research topic, and multiple tools have been developed for the task. PANNZER2 is an automated functional annotation tool that merely relies on the sequence similarity of the query to the annotated proteins. We tried PANNZER2 on Trypanosoma brucei, the most studied organism among trypanosomatids, to see if it could improve our knowledge of the functions of the genes.Even with the availability of automated annotation tools like InterPro2GO in databases such as TriTrypDB, PANNZER2 has made surprisingly confident predictions for some hypothetical proteins in T. brucei. In this study, we identify gaps in such annotations because of not employing pairwise sequence alignment tools in TriTrypDB's automated annotation process. Our findings demonstrate that even the use of stringent cutoffs can successfully annotate a significant number of proteins. Additionally, we discovered that adjusting the open reading frames in certain genes leads to sequences with increased sequence signature coverage—characterized by the length covered by at least one sequence signature—compared to the original sequences. This enhanced sequence signature coverage suggests these genomic fragments could be pseudogenes. To facilitate further exploration, we developed a script to help identify potential pseudogenes within an organism's genome, offering researchers a new tool for genomic analysis and understanding. We extended all our analysis to Trypanosoma cruzi and Leishmania major to assess the impact of this approach across different species.Our study demonstrates that by utilizing pairwise sequence similarity alignment, even with stringent cutoffs, we can attribute 2986, 3953, and 3798 new GO terms to the genomes of T. brucei, T. cruzi, and L. major. Additionally, we found that 210, 239, and 29 genes exhibit increased sequence signature coverage following frame correction, suggesting the presence of pseudogenes.

Water Resources’ AI–ML Data Uncertainty Risk and Mitigation Using Data Assimilation

Artificial intelligence (AI), including machine learning (ML) and deep learning (DL), learns by training and is restricted by the amount and quality of training data. Training involves a tradeoff between prediction bias and variance controlled by model complexity. Increased model complexity decreases prediction bias, increases variance, and increases overfitting possibilities. Overfitting is a significantly smaller training prediction error relative to the trained model prediction error for an independent validation set. Uncertain data generate risks for AI–ML because they increase overfitting and limit generalization ability. Specious confidence in predictions from overfit models with limited generalization ability, leading to misguided water resource management, is the uncertainty-related negative consequence. Improved data is the way to improve AI–ML models. With uncertain water resource data sets, like stream discharge, there is no quick way to generate improved data. Data assimilation (DA) provides mitigation for uncertainty risks, describes data- and model-related uncertainty, and propagates uncertainty to results using observation error models. A DA-derived mitigation example is provided using a common-sense baseline, derived from an observation error model, for the confirmation of generalization ability and a threshold identifying overfitting. AI–ML models can also be incorporated into DA to provide additional observations for assimilation or as a forward model for prediction and inverse-style calibration or training. The mitigation of uncertain data risks using DA involves a modified bias–variance tradeoff that focuses on increasing solution variability at the expense of increased model bias. Increased variability portrays data and model uncertainty. Uncertainty propagation produces an ensemble of models and a range of predictions.

Model-agnostic confidence measurement for aggregating multimodal ensemble models in automatic diagnostic systems

Automatic diagnostic systems (ADSs) have been garnering increased attention because they can alleviate the workload of clinicians by assisting in diagnosis and offering low-cost access to healthcare for people in medically underserved areas. ADS can suggest potential diseases by analyzing a patient's self-report. Previous research on ADS has leveraged diagnostic case data from various patients and medical knowledge to diagnose diseases, with multimodal ensemble methods proving particularly effective. However, the existing multimodal ensemble method combines the probabilities of different models in the aggregating process, which can not properly combine the probabilities that are produced by different criteria. To address these issues, we propose an effective aggregation framework for multimodal ensembles that can properly aggregate model-agnostic confidence scores and predictions from each model. Our framework transforms probability scores from different criteria into unified aggregation rule-based scores and reflects the gap between the probabilities that may be blurred in the aggregation process through the confidence score. In particular, The proposed confidence measurement method employs a post-analysis approach with the developed model or algorithm, making it adaptable in a model-agnostic manner and suitable for multimodal ensemble learning that utilizes heterogeneous prediction results. Our experimental results demonstrate that our framework outperforms existing approaches by more effectively leveraging the strengths of each ensemble member.

LesionMix data enhancement and entropy minimization for semi-supervised lesion segmentation of lung cancer

Determining the location and contour of the lesion is a crucial prerequisite for medical diagnosis, subsequent personalized treatment plan and prognostic prediction of lung cancer. Semi-supervised learning and data augmentation methods facilitate deep learning to be used in many fields of medical imaging. In this paper, we introduce a novel data enhancement technique called LesionMix. This method involves extracting and reusing lesions from a limited amount of labeled CT data, thereby enhancing the efficiency of utilizing those labeled data. Meanwhile, we propose a two-stage semi-supervised training strategy called Entropy Minimization LesionMix (EMLM). In the first stage, features containing lesion contour information are rapidly learned through LesionMix data augmentation. Entropy minimization strategy optimizes the model parameters to alleviate overfitting as much as possible in the second stage and improves prediction confidence. Our proposed method is validated on the public dataset LIDC-IDRI and the in-house dataset GDPHLUAD. Extensive experiments demonstrate that our method achieves promising performance and outperforms seven state-of-the-art semi-supervised models; Moreover, ablation experiments validate the effectiveness of various aspects of our approach.

SMoRe GloS: An efficient and flexible framework for inferring global sensitivity of agent-based model parameters.

Agent-based models (ABMs) have become essential tools for simulating complex biological, ecological, and social systems where emergent behaviors arise from the interactions among individual agents. Quantifying uncertainty through global sensitivity analysis is crucial for assessing the robustness and reliability of ABM predictions. However, most global sensitivity methods demand substantial computational resources, making them impractical for highly complex models. Here, we introduce SMoRe GloS (Surrogate Modeling for Recapitulating Global Sensitivity), a novel, computationally efficient method for performing global sensitivity analysis of ABMs. By leveraging explicitly formulated surrogate models, SMoRe GloS allows for comprehensive parameter space exploration and uncertainty quantification without sacrificing accuracy. We demonstrate our method's flexibility by applying it to two biological ABMs: a simple 2D cell proliferation assay and a complex 3D vascular tumor growth model. Our results show that SMoRe GloS is compatible with simpler methods like the Morris one-at-a-time method, and more computationally intensive variance-based methods like eFAST. SMoRe GloS accurately recovered global sensitivity indices in each case while achieving substantial speedups, completing analyses in minutes. In contrast, direct implementation of eFAST amounted to several days of CPU time for the complex ABM. Remarkably, our method also estimates sensitivities for ABM parameters representing processes not explicitly included in the surrogate model, further enhancing its utility. By making global sensitivity analysis feasible for computationally expensive models, SMoRe GloS opens up new opportunities for uncertainty quantification in complex systems, allowing for more in depth exploration of model behavior, thereby increasing confidence in model predictions.

Evaluation of Nontargeted Mass Spectral Data Acquisition Strategies for Water Analysis and Toxicity-Based Feature Prioritization by MS2Tox.

The machine-learning tool MS2Tox can prioritize hazardous nontargeted molecular features in environmental waters, by predicting acute fish lethality of unknown molecules based on their MS2 spectra, prior to structural annotation. It has yet to be investigated how the extent of molecular coverage, MS2 spectra quality, and toxicity prediction confidence depend on sample complexity and MS2 data acquisition strategies. We compared two common nontargeted MS2 acquisition strategies with liquid chromatography high-resolution mass spectrometry for structural annotation accuracy by SIRIUS+CSI:FingerID and MS2Tox toxicity prediction of 191 reference chemicals spiked to LC-MS water, groundwater, surface water, and wastewater. Data-dependent acquisition (DDA) resulted in higher rates (19-62%) of correct structural annotations among reference chemicals in all matrices except wastewaters, compared to data-independent acquisition (DIA, 19-50%). However, DIA resulted in higher MS2 detection rates (59-84% DIA, 37-82% DDA), leading to higher true positive rates for spectral library matching, 40-73% compared to 34-72%. DDA resulted in higher MS2Tox toxicity prediction accuracy than DIA, with root-mean-square errors of 0.62 and 0.71 log-mM, respectively. Given the importance of MS2 spectral quality, we introduce a "CombinedConfidence" score to convey relative confidence in MS2Tox predictions and apply this approach to prioritize potentially ecotoxic nontargeted features in environmental waters.

ITER materials irradiation within the D–T neutron environment at JET: post-irradiation radioactivity analysis following the DTE2 experimental campaign

Abstract This work presents the results following the first irradiation of ITER materials samples in a tokamak D--T plasma environment operating at significant fusion power. The materials exposed to this nuclear environment at the Joint European Torus during the DTE2 experimental campaign that took place in 2021 include representative ITER samples from various components such as poloidal field (PF) coil jacket samples, toroidal field coil radial closure plate steels, EUROFER 97 steel, W and CuCrZr materials from the divertor, Inconel-718, CuCrZr, and 316L stainless steel for blanket modules, as well as vacuum vessel forging samples.&amp;#xD;&amp;#xD;The experimental results discussed include high-resolution gamma spectrometry measurements and analysis conducted with the post-irradiated samples, of which there were 68 in total. These samples were exposed through different experimental campaigns, including deuterium, deuterium--tritium and tritium phases. Supporting the analysis were 25 dosimetry foil-based neutron diagnostics and two 'VERDI' neutron spectrometry diagnostics. A further 12 samples for positron annihilation spectroscopy (PALS) were also irradiated. The irradiation of all these samples took place in a long-term irradiation assembly located near the JET vacuum vessel.&amp;#xD;&amp;#xD;The post-irradiation analysis of the ITER material samples has yielded valuable insights into their material activation levels and radiation fields. Comparative assessments between experimental measurements and comprehensive neutronics simulations have demonstrated a significant level of agreement in this work, while also revealing some discrepancies in specific material instances. The data and interpretation from this work not only serve as a robust experimental foundation for enhancing the precision and predictability of neutronics simulation approaches for ITER and next-step devices but also present some opportunities for the refinement of simulation methodologies. In light of these findings, a series of recommendations have been proposed, aimed at improving confidence in nuclear predictions associated with materials that have been exposed to fusion nuclear environments and advancing understanding in this important domain.

Clinical usability of deep learning-based saliency maps for occlusion myocardial infarction identification from the prehospital 12-Lead electrocardiogram

IntroductionDeep learning (DL) models offer improved performance in electrocardiogram (ECG)-based classification over rule-based methods. However, for widespread adoption by clinicians, explainability methods, like saliency maps, are essential. MethodsOn a subset of 100 ECGs from patients with chest pain, we generated saliency maps using a previously validated convolutional neural network for occlusion myocardial infarction (OMI) classification. Three clinicians reviewed ECG-saliency map dyads, first assessing the likelihood of OMI from standard ECGs and then evaluating clinical relevance and helpfulness of the saliency maps, as well as their confidence in the model's predictions. Questions were answered on a Likert scale ranging from +3 (most useful/relevant) to −3 (least useful/relevant). ResultsThe adjudicated accuracy of the three clinicians matched the DL model when considering area under the receiver operating characteristics curve (AUC) and F1 score (AUC 0.855 vs. 0.872, F1 score = 0.789 vs. 0.747). On average, clinicians found saliency maps slightly clinically relevant (0.96 ± 0.92) and slightly helpful (0.66 ± 0.98) in identifying or ruling out OMI but had higher confidence in the model's predictions (1.71 ± 0.56). Clinicians noted that leads I and aVL were often emphasized, even when obvious ST changes were present in other leads. ConclusionIn this clinical usability study, clinicians deemed saliency maps somewhat helpful in enhancing explainability of DL-based ECG models. The spatial convolutional layers across the 12 leads in these models appear to contribute to the discrepancy between ECG segments considered most relevant by clinicians and segments that drove DL model predictions.

College Chatbot Using RASA

The rapid advancement of Artificial Intelligence, Machine Learning, etc., has brought about significant changes in the technological sphere for decades. One such transformative technology is a Chatbot, a Conversational AI capable of addressing users’ queries. These chatbots are tailored to serve various objectives, one of which is to streamline tasks in education through automation. In line with this, a paper titled “College Chatbot Using RASA” has been written, which can effectively respond to questions based on college and other college-related information. The chatbot has been made accessible via an API utilizing the Streamlit framework. The RASA framework, a key open-source technology, has been harnessed to enable the chatbot. RASA plays a vital role in the chatbot's functionality by combining two main components, Rasa Core and Rasa Natural Language Understanding (NLU). Rasa NLU component is used to deduce intent and extract necessary entities from user input, while Rasa Core generates output by constructing a probabilistic model. The model is evaluated by getting a confusion matrix and prediction confidence distribution. The chatbot, upon testing, showcases the best result of 1 for each accuracy, precision, recall, and F1 score.