Model Performance Measures Research Articles

A Systematic Review and Meta-analysis of Convolutional Neural Network in the Diagnosis of Colorectal Polyps and Cancer.

Convolutional neural networks are a class of deep neural networks used for different clinical purposes, including improving the detection rate of colorectal lesions. This systematic review and meta-analysis aimed to assess the performance of convolutional neural network-based models in the detection or classification of colorectal polyps and colorectal cancer. A systematic search was performed in MEDLINE, SCOPUS, Web of Science, and other related databases. The performance measures of the convolutional neural network models in the detection of colorectal polyps and colorectal cancer were calculated in the 2 scenarios of the best and worst accuracy. Stata and R software were used for conducting the meta-analysis. From 3368 searched records, 24 primary studies were included. The sensitivity and specificity of convolutional neural network models in predicting colorectal polyps in worst and best scenarios ranged from 84.7% to 91.6% and from 86.0% to 93.8%, respectively. These values in predicting colorectal cancer varied between 93.2% and 94.1% and between 94.6% and 97.7%. The positive and negative likelihood ratios varied between 6.2 and 14.5 and 0.09 and 0.17 in these scenarios, respectively, in predicting colorectal polyps, and 17.1-41.2 and 0.07-0.06 in predicting colorectal polyps. The diagnostic odds ratio and accuracy measures of convolutional neural network models in predicting colorectal polyps in worst and best scenarios ranged between 36% and 162% and between 80.5% and 88.6%, respectively. These values in predicting colorectal cancer in the worst and the best scenarios varied between 239.63% and 677.47% and between 88.2% and 96.4%. The area under the receiver operating characteristic varied between 0.92 and 0.97 in the worst and the best scenarios in colorectal polyps, respectively, and between 0.98 and 0.99 in colorectal polyps prediction. Convolutional neural network-based models showed an acceptable accuracy in detecting colorectal polyps and colorectal cancer.

Radiomic-Based Prediction of Lesion-Specific Systemic Treatment Response in Metastatic Disease.

Cancer is a heterogenous, dynamic target. Individual patients, lesions, and cell populations may present with different characteristics and varying sensitivities to anticancer therapies. The purpose of this study was to investigate the utility of image-derived (radiomic) biomarkers for prediction of lesion-specific treatment response. Using a database of serial imaging from a Phase III clinical trial (NCT01440088), all visible lung metastases were contoured on chest computed tomography performed at baseline and after 2 cycles of chemotherapy. Metastatic leiomyosarcoma patients received either doxorubicin monotherapy (DM) or doxorubicin plus evofosfamide (DE). Percent change in volume with respect to baseline was evaluated as a metric of treatment response. Lesions that experienced a positive volume change greater than 50% were labeled as 'progressors'; all other lesions were labeled as 'non-progressors'. Radiomic features were extracted at baseline from the whole lesion and three peritumoral regions using PyRadiomics (version 3.0.1). Features were compared using the Mann-Whitney U-test and further used to classify between response categories. Logistic classification models were evaluated with 10-fold cross-validation using Area Under the Curve for the Receiver Operating Characteristic (AUROC) and Precision-Recall (AUPRC) curves as measures of model performance. Significance was evaluated by permutation tests. All code used to derive features is open source and available for download. In total, 204 lung metastases from 80 patients were evaluated. The fraction of progressing lesions was 19% and 27% for the DM and DE regimes, respectively. Differences in individual lesion response were observed in 20% of patients with multiple lung metastases. Significant differences in entropy were observed between response categories (false discovery rate < 5% after multiple testing correction): interior regions for lesions receiving DM, and peripheral regions for lesions receiving DE. The best performing models for the DM and DE regimens had AUROC of 0.87 and 0.76 and AUPRC 0.60 and 0.50, respectively (p < 0.05). Predicting lesion-specific responses using radiomic features represents a paradigm shift. Lesion-specific radiomic models indicate a 2 to 3-fold increase in predictive capacity in comparison to a no-skill classifier. These models, although preliminary, achieved a strong predictive value and could be used to predict lesion-specific treatment response.

Skeletal Muscle Health, Physical Performance, and Lower Urinary Tract Symptoms in Older Adults: The Study of Muscle, Mobility, and Aging.

Lower urinary tract symptoms (LUTS) and mobility limitations are bidirectionally associated among older adults, but the role of skeletal muscle remains unknown. We evaluated cross-sectional associations of muscle health and physical performance with LUTS. We used data from 377 women and 264 men aged >70 years in the Study of Muscle, Mobility and Aging (SOMMA). LUTS and urinary bother were assessed using the LURN Symptom Index-10 (SI-10; higher = worse symptoms). Muscle mass and volume were assessed using D3-creatine dilution (D3Cr) and magnetic resonance imaging. Grip strength and peak leg power assessed upper/lower extremity physical performance. 400-m walk, Short Physical Performance Battery (SPPB), and Four Square Step Test (FSST) assessed global physical performance. Mobility Assessment Tool-short form (MAT-sf) assessed self-reported mobility. We calculated Spearman correlation coefficients adjusted for age, body mass index, multimorbidity, and polypharmacy, chi-square tests, and Fisher's Z-test to compare correlations. Among women, LURN SI-10 total scores were inversely correlated with FSST (rs = 0.11, p = .045), grip strength (rs = -0.15, p = .006), and MAT-sf (rs = -0.18, p = .001), but not other muscle and physical performance measures in multivariable models. LURN SI-10 was not associated with any of these measures among men. Forty-four percent of women in the lowest tertile of 400-m walk speed versus 24% in the highest tertile reported they were at least "somewhat bothered" by urinary symptoms (p < .001), whereas differences among men were not significant. Balance and grip strength were associated with LUTS severity in older women but not men. Associations with other muscle and physical performance measures varied by LUTS subtype but remained strongest among women.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis.

Poster Session: Deep Active Learning Using Retinal Image Embedding Vectors to Optimize Training Data Selections.

In recent years, deep learning (DL) has made breakthroughs in many fields. However, the success of DL requires constant re-training with large amounts of data, which is often costly, causing data paucity problems in the medical space. Active learning (AL), on the other hand, aims to reduce the amount of training data while still retaining similar performance through the exploration of the added value of data. In this study, we designed an AL system for training Artificial Intelligence (AI) models to explore faster learning curves while minimizing the need for new data. The model was designed for vessel object detection, using a Deep Neural Network (DNN), for scanning laser ophthalmoscopy (SLO) retinal images. Image embedding vectors were used to proactively select the most informative data. We established a baseline of k-fold cross validation frameworks to measure model performance, where 809 annotated images were randomly selected from our database and divided into multiple regions of interest with minimal selection bias. Using this validation framework, we observed the detection model for vessel junctions could achieve a mean average precision (mAP) of 0.648, precision of 0.645, and recall of 0.703; tested on 135 images. We then tested our active learning system by dividing the dataset into two clusters using image embedding vectors and machine learning clustering algorithms. We observed one cluster converged faster to the desired mAP performance than the other by 2.3x.

Identifying the Neurocognitive Difference Between Two Groups Using Supervised Learning

&#x0D; &#x0D; &#x0D; Brain Imaging Analysis is a dynamic and exciting field within neuroscience. This study is conducted with two main objectives. First, to develop a classification framework to enhance predictive performance, and second, to conduct a comparative analysis of accuracy versus inference using brain imaging data. The dataset of chess masters and chess novices is utilized to identify neurocognitive differences between the two groups, based on their resting-state functional magnetic resonance imaging data. A network of connections between brain regions is created and analyzed. Standard statistical learning techniques and machine learning models are then applied to distinguish connectivity patterns between the groups. The trade-off between model precision and interpretability is also assessed. Finally, model performance measures, including accuracy, sensitivity, specificity, and AUC, are reported to demonstrate the effectiveness of the model framework.&#x0D; &#x0D; &#x0D;

Chemometric analysis of voltammetric-based sensing of heavy metal ions in aqueous solution for lead (Pb+2) ions

In the present work, we have developed a sensitive sensing scheme to identify lead ions in the aqueous water system at ultra-trace levels. The electrochemical technique, anodic stripping voltammetry (ASV) was used to quantify lead ion (Pb2+) in aqueous samples within the range of 10–50 μg/L concentration. The linear ASV signals were found in the 10–50 μg/L concentration range from the experimental work with the regression coefficient R2 0.975. Based on the standard deviation (0.14), the detection limit and quantification limit of the sensor was found to be 1.49 μg/L (S/N = 3) and 4.92 μg/L (S/N = 10), which is a smaller order of magnitude below the 10 μg/L thresholds for drinking water imposed by the World Health Organization.Further, Chemometric modelling was imposed to analyse the data appropriately. Principal component analysis (PCA) was used to conduct a quantitative study and multiple response analysis of the sensor's behaviour to lead metal ions, demonstrating the proposed sensing scheme's excellent performance and stability. The PLSR model performance measures obtained include- R-square (0.99) and RMSE (0.90), indicating the reliability and robustness of the prediction model for testing unknown samples. Therefore, the proposed sensing scheme's effectiveness can be used for various environmental applications, making it a potential platform.

Optimizing Clinical Decision Making with Decision Curve Analysis: Insights for Clinical Investigators.

A large number of prediction models are published with the objective of allowing personalized decision making for diagnostic or prognostic purposes. Conventional statistical measures of discrimination, calibration, or other measures of model performance are not well-suited for directly and clearly assessing the clinical value of scores or biomarkers. Decision curve analysis is an increasingly popular technique used to assess the clinical utility of a prognostic or diagnostic score/rule, or even of a biomarker. Clinical utility is expressed as the net benefit, which represents the net balance of patients' benefits and harms and considers, implicitly, the consequences of clinical actions taken in response to a certain prediction score, rule, or biomarker. The net benefit is plotted against a range of possible exchange rates, representing the spectrum of possible patients' and clinicians' preferences. Decision curve analysis is a powerful tool for judging whether newly published or existing scores may truly benefit patients, and represents a significant advancement in improving transparent clinical decision making. This paper is meant to be an introduction to decision curve analysis and its interpretation for clinical investigators. Given the extensive advantages, we advocate applying decision curve analysis to all models intended for use in clinical practice.

Gender Classification from Fingerprint Using Hybrid CNN-SVM

Gender classification is used in numerous applications such as biometrics, criminology, surveillance, HCI, and business profiling. Although biometric factors like gait, face, hand shape, and iris have been used to classify people into genders, the majority of research has focused on facial traits due to their more recognisable qualities. This research employs fingerprints to classify gender, with the intention of being relevant for future studies. Several methods for gender classification utilising fingerprints have been presented in the literature, including ANN, KNN, Naive Bayes, the Gaussian mixture model, and deep learning-based classifiers. Although these classifiers have shown good classification accuracy, gender classification remains an unexplored field of study that necessitates the development of new approaches to enhance recognition accuracy, computation, and running time. In this paper, a CNN-SVM hybrid framework for gender classification from fingerprints is proposed, where preprocessing, feature extraction, and classification are the three main components. The main goal of this study is to use CNN to extract fingerprint information. These features are then sent to an SVM classifier to determine gender. The hybrid model's performance measures are examined and compared to those of the conventional CNN model. Using a CNN-SVM hybrid model, the accuracy of gender classification based on fingerprints was 99.25%.

Prediction and analysis of atmospheric visibility in five terrain types with artificial intelligence

Scattering visiometers are widely used to measure atmospheric visibility; however, visibility is difficult to measure accurately because the extinction coefficient decays exponentially with visual range according to the Koschmid's law. Moreover, models for predicting visibility are lacking due to the lack of accurate visibility observations to verify. This study formulated an artificial intelligence method for measuring atmospheric visibility in five topographical regions: hills, basins, plains, alluvial plains, and rift valleys. Four air pollution factors and five meteorological factors were selected as independent variables for predicting visibility by using three artificial intelligence models, namely a support vector machine (SVM) model, a multilayer perceptron (MLP) model, and an extreme gradient boosting (XGBoost) model. The GridSearchCV function was used to automatically tune model hyperparameters to determine the optimal parameter values of the three models for the five target areas. The predictions of the aforementioned three models underwent considerable considerably scale shrinking relative to observed values. The inappropriately low predicted visibility values might have been caused by the use of inaccurate observations for training. To solve this problem, formulas of scale ratio and downshift were used to adjust the predicted values. Statistical measurements of model performance measures by five quantitative methods (e.g., correlation coefficient, mean absolute error) showed that adjusted predictions were in strong agreement with the observation data for the five target areas. Therefore, the adjusted prediction has high reliability. Because of obvious differences in the topography, weather, and air quality of the five target areas, different models provided optimal predictions for different areas. In densely populated western Taiwan, the MLP model is most suitable for predicting visibility on hills whereas the XGBoost model is most suitable for predicting visibility on basins and plains. In eastern Taiwan, the SVM model is most suitable for predicting visibility on alluvial plains and rift valleys. Thus, the optimal prediction model should be identified according to the conditions in each area. These results can inform decision-making processes or improve visibility predicting in specific areas.

Risk prediction models for antineoplastic-associated cardiotoxicity in treatment of breast cancer: A systematic review.

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time. The objective of this systematic review is to assess methodology of published models to predict the risk of antineoplastic-associated cardiotoxicity in patients with breast cancer. We searched PubMed and Embase for studies that developed or validated a multivariable risk prediction model. Data extraction and quality assessments were performed according to the Prediction Model Risk of Bias Assessment Tool (PROBAST). We identified 2,816 unique publications and included 8 eligible studies (7 new risk models and 1 validation of a risk stratification tool) that modeled risk with trastuzumab (n = 5), anthracyclines (n = 2), and anthracyclines with or without trastuzumab (n = 1). The most common final predictors were previous or concomitant chemotherapy (n = 5) and age (n = 4). Three studies incorporated measures of myocardial mechanics that may not be frequently available. Model discrimination was reported in 7 studies (range of area under the receiver operating characteristic curve, 0.56-0.88), while calibration was reported in 1 study. Internal and external validation were performed in 4 studies and 1 study, respectively. Using the PROBAST methodology, we rated the overall risk of bias as high for 7 of 8 studies and unclear for 1 study. Concerns for applicability were low for all studies. Among 8 models to predict the risk of cardiotoxicity of antineoplastic agents for breast cancer, 7 were rated as having a high risk of bias and all had low concerns for clinical applicability. Most evaluated studies reported positive measures of model performance but did not perform external validation. Efforts to improve development and reporting of these models to facilitate their use in practice are warranted.

Role of artificial intelligence in pharmacy practice: A narrative review

AbstractArtificial intelligence (AI) has emerged as a potentially useful tool in transforming healthcare. Roles for AI include drug safety, operations, as well as precision medicine to improve patient outcomes. Within pharmacy practice, limited guidance is available describing the purpose of AI. This gap is important to address given the current growth of AI in medicine. The purpose of this narrative review was to review the literature evaluating the potential role of AI in three relevant areas of pharmacy practice. A review of Medline and Embase from January 2004 to June 2022 evaluating search terms involving AI and drug safety, pharmacy operations, or precision medicine. Articles had to be focused on human subjects, be either an observational cohort study or randomized control trial, and involve medication use to be included in this review. Data collection included the study design, AI methodology, and primary outcome and associated AI model performance measures. A total of 24 232 articles were identified from the search strategy, with 403 articles meeting inclusion criteria (1.7%). A total of 22 articles were included in this review. Articles focused on drug safety included safe use of opioids and the role of social media in identifying adverse drug reactions. In terms of pharmacy operations, articles included the identification of medications at risk for shortage and visual identification of medications. Finally, for precision medicine, articles included determining likelihood of beneficial effects of medications for new diagnoses and dosing of warfarin. The role of AI in improving patient outcomes is numerous. However, challenges that remain include transparency of generated AI models, generalizability, and lack of external validation. Future areas of focus include education of pharmacy clinicians on the role of AI and the increased use of open‐source models/data to allow for external validation of AI.

A new key performance indicator model for demand forecasting in inventory management considering supply chain reliability and seasonality

Forecasting demand and determining safety stocks are key aspects of supply chain planning. Demand forecasting involves predicting future demand for a product or service using historical data and other external and internal drivers. Stockouts and excess production can be reduced by accurately forecasting demand. This allows companies to plan production, inventory, and logistics more effectively. Companies maintain safety stocks in their inventory to protect against unexpected changes in demand or supply. A company must find the appropriate safety stock level to meet customer demands while avoiding excess inventory and carrying costs. Forecasting demand and determining safety stocks work together to help companies reduce costs, improve customer service, and optimize inventory levels. Key Performance Indicators (KPIs) are commonly used to measure model performance. Classical forecasting models mostly concern themselves with minimizing forecast errors. However, the impact on inventory costs is not directly considered. In this paper, we introduce a Key Performance Indicator to be used in the demand forecasting process that produces more efficient results in terms of inventory costs. We also propose a novel approach to determining the best level for safety stock. This approach considers logistic network supply reliability and seasonality indices identified within historical demand patterns. We use real-life data and show that the proposed method can improve efficiency in forecasting and safety stock levels by reducing the risk of stockouts and excess inventory.

Multi-modal deep learning for automated assembly of periapical radiographs

ObjectivesPeriapical radiographs are oftentimes taken in series to display all teeth present in the oral cavity. Our aim was to automatically assemble such a series of periapical radiographs into an anatomically correct status using a multi-modal deep learning model. Methods4,707 periapical images from 387 patients (on average, 12 images per patient) were used. Radiographs were labeled according to their field of view and the dataset split into a training, validation, and test set, stratified by patient. In addition to the radiograph the timestamp of image generation was extracted and abstracted as follows: A matrix, containing the normalized timestamps of all images of a patient was constructed, representing the order in which images were taken, providing temporal context information to the deep learning model. Using the image data together with the time sequence data a multi-modal deep learning model consisting of two residual convolutional neural networks (ResNet-152 for image data, ResNet-50 for time data) was trained. Additionally, two uni-modal models were trained on image data and time data, respectively. A custom scoring technique was used to measure model performance. ResultsMulti-modal deep learning outperformed both uni-modal image-based learning (p<0.001) and time-based learning (p<0.05). The multi-modal deep learning model predicted tooth labels with an F1-score, sensitivity and precision of 0.79, respectively, and an accuracy of 0.99. 37 out of 77 patient datasets were fully correctly assembled by multi-modal learning; in the remaining ones, usually only one image was incorrectly labeled. ConclusionsMulti-modal modeling allowed automated assembly of periapical radiographs and outperformed both uni-modal models. Dental machine learning models can benefit from additional data modalities. Clinical significanceLike humans, deep learning models may profit from multiple data sources for decision-making. We demonstrate how multi-modal learning can assist assembling periapical radiographs into an anatomically correct status. Multi-modal learning should be considered for more complex tasks, as clinically a wealth of data is usually available and could be leveraged.

Predictive Maintenance of Lead-Acid Batteries with Sparse Vehicle Operational Data

Predictive maintenance aims to predict failures in components of a system, a heavy-duty vehicle in this work, and do maintenance before any actual fault occurs. Predictive maintenance is increasingly important in the automotive industry due to the development of new services and autonomous vehicles with no driver who can notice first signs of a component problem. The lead-acid battery in a heavy vehicle is mostly used during engine starts, but also for heating and cooling the cockpit, and is an important part of the electrical system that is essential for reliable operation. This paper develops and evaluates two machine-learning based methods for battery prognostics, one based on Long Short-Term Memory (LSTM) neural networks and one on Random Survival Forest (RSF). The objective is to estimate time of battery failure based on sparse and non-equidistant vehicle operational data, obtained from workshop visits or over-the-air readouts. The dataset has three characteristics: 1) no sensor measurements are directly related to battery health, 2) the number of data readouts vary from one vehicle to another, and 3) readouts are collected at different time periods. Missing data is common and is addressed by comparing different imputation techniques. RSF- and LSTM-based models are proposed and evaluated for the case of sparse multiple-readouts. How to measure model performance is discussed and how the amount of vehicle information influences performance.

Explainable AI and machine learning algorithms to predict treatment failures for patients with cancer.

e13577 Background: Cancer patients may undergo lengthy and painful chemotherapy treatments, comprising several successive regimens or plans. Treatment inefficacy and other adverse events can lead to discontinuation (or failure) of these plans, or prematurely changing them, which results in a significant amount of physical, financial, and emotional toxicity to the patients and their families. In this research work, we build AI driven treatment failure models that utilize the real-world evidence gathered from patients’ profiles available in an oncology EMR/EHR system, with a goal of predicting the likelihood of a plan being discontinued at the time of its prescription. The selected AI models achieve a prediction accuracy of more than 80% and also provide reasons for their inference. Methods: Inclusion and Exclusion Criteria: Deidentified and anonymized electronic health records of patients, with their prescribed chemotherapies, for five different primary cancer diagnoses - ICD10 codes C18, C34, C50, C61 and C90 - that have the highest plan discontinuation rates between the years 2015 and 2022 are analyzed. All patients of other cancer types are excluded. AI Models: Unique features, that influence the treatment failure, for each cancer type are engineered by using therapeutic classification of drugs, diagnoses codes, comorbidity scores, tumor and biomarker information that is extracted from the notes and lab tests. We only use features that are available at the time of selecting a treatment plan. Several machine learning classifiers are investigated, and three tree ensembles - random forests, Xgboost and boosted forests - are further evaluated on the validation set to fine tune learning parameters with an objective to reduce the complexity of decision trees for providing better interpretability without significantly compromising the accuracy. Results: Our pilot studies reveal that boosted forests comprising of 5 random forests, each with 5 trees of depth 10 offer the best compromise between performance and interpretability. The models once trained are evaluated on unseen datasets and four performance measures of AI models are reported. On average, 15 rules are autonomously generated for a treatment failure inference for each cancer type and generally 6 of them have a significant support of 30 samples or greater. Conclusions: Machine learning algorithms for predicting treatment efficacy of chemotherapy regimens by deriving inference from the patients’ EMR/EHR data is an emerging yet challenging research domain. Our studies demonstrate that AI models like boosted forests provide the optimal models for treatment failure use case. In future, we want to validate the system in controlled clinical trials with the help of oncologists. [Table: see text]

Development and validation of explainable machine-learning models for carotid atherosclerosis early screening

BackgroundCarotid atherosclerosis (CAS), an important factor in the development of stroke, is a major public health concern. The aim of this study was to establish and validate machine learning (ML) models for early screening of CAS using routine health check-up indicators in northeast China.MethodsA total of 69,601 health check-up records from the health examination center of the First Hospital of China Medical University (Shenyang, China) were collected between 2018 and 2019. For the 2019 records, 80% were assigned to the training set and 20% to the testing set. The 2018 records were used as the external validation dataset. Ten ML algorithms, including decision tree (DT), K-nearest neighbors (KNN), logistic regression (LR), naive Bayes (NB), random forest (RF), multiplayer perceptron (MLP), extreme gradient boosting machine (XGB), gradient boosting decision tree (GBDT), linear support vector machine (SVM-linear), and non-linear support vector machine (SVM-nonlinear), were used to construct CAS screening models. The area under the receiver operating characteristic curve (auROC) and precision-recall curve (auPR) were used as measures of model performance. The SHapley Additive exPlanations (SHAP) method was used to demonstrate the interpretability of the optimal model.ResultsA total of 6315 records of patients undergoing carotid ultrasonography were collected; of these, 1632, 407, and 1141 patients were diagnosed with CAS in the training, internal validation, and external validation datasets, respectively. The GBDT model achieved the highest performance metrics with auROC of 0.860 (95% CI 0.839–0.880) in the internal validation dataset and 0.851 (95% CI 0.837–0.863) in the external validation dataset. Individuals with diabetes or those over 65 years of age showed low negative predictive value. In the interpretability analysis, age was the most important factor influencing the performance of the GBDT model, followed by sex and non-high-density lipoprotein cholesterol.ConclusionsThe ML models developed could provide good performance for CAS identification using routine health check-up indicators and could hopefully be applied in scenarios without ethnic and geographic heterogeneity for CAS prevention.

Cooperative-Aware Radio Resource Allocation Scheme for 5G Network Slicing in Cloud Radio Access Networks.

The 5G network is designed to serve three main use cases: enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), and ultra-reliable and low-latency communications (uRLLC). There are many new technological enablers, including the cloud radio access network (C-RAN) and network slicing, that can support 5G and meet its requirements. The C-RAN combines both network virtualization and based band unit (BBU) centralization. Using the network slicing concept, the C-RAN BBU pool can be virtually sliced into three different slices. 5G slices require a number of Quality of service (QoS) metrics, such as average response time and resource utilization. In order to enhance the C-RAN BBUs utilization while protecting the minimum QoS of the coexisting three slices, a priority-based resource allocation with queuing model is proposed. The uRLLC is given the highest priority, while eMBB has a higher priority than mMTC services. The proposed model allows the eMBB and mMTC to be queued and the interrupted mMTC to be restored in its queue to increase its chance to reattempt the service later. The proposed model's performance measures are defined and derived using a continuous-time Markov chain (CTMC) model and evaluated and compared using different methodologies. Based on the results, the proposed scheme can increase C-RAN resource utilization without degrading the QoS of the highest-priority uRLLC slice. Additionally, it can reduce the forced termination priority of the interrupted mMTC slice by allowing it to re-join its queue. Therefore, the comparison of the results shows that the proposed scheme outperforms the other states of the art in terms of improving the C-RAN utilization and enhancing the QoS of eMBB and mMTC slices without degrading the QoS of the highest priority use case.

Leave-one-out cross-validation, penalization, and differential bias of some prediction model performance measures—a simulation study

BackgroundThe performance of models for binary outcomes can be described by measures such as the concordance statistic (c-statistic, area under the curve), the discrimination slope, or the Brier score. At internal validation, data resampling techniques, e.g., cross-validation, are frequently employed to correct for optimism in these model performance criteria. Especially with small samples or rare events, leave-one-out cross-validation is a popular choice.MethodsUsing simulations and a real data example, we compared the effect of different resampling techniques on the estimation of c-statistics, discrimination slopes, and Brier scores for three estimators of logistic regression models, including the maximum likelihood and two maximum penalized likelihood estimators.ResultsOur simulation study confirms earlier studies reporting that leave-one-out cross-validated c-statistics can be strongly biased towards zero. In addition, our study reveals that this bias is even more pronounced for model estimators shrinking estimated probabilities towards the observed event fraction, such as ridge regression. Leave-one-out cross-validation also provided pessimistic estimates of the discrimination slope but nearly unbiased estimates of the Brier score.ConclusionsWe recommend to use leave-pair-out cross-validation, fivefold cross-validation with repetitions, the enhanced or the .632+ bootstrap to estimate c-statistics, and leave-pair-out or fivefold cross-validation to estimate discrimination slopes.

Prediction Models for Intravenous Immunoglobulin Resistance in Kawasaki Disease: A Meta-analysis.

Approximately 10% to 20% of patients with Kawasaki disease (KD) are refractory to initial intravenous immunoglobulin (IVIG) therapy. KD is mainly associated with coronary artery abnormalities. To identify and evaluate all developed prediction models for IVIG resistance in patients with KD and synthesize evidence from external validation studies that evaluated their predictive performances. PubMed Medline, Dialog Embase, the Cochrane Central Register of Controlled Trials, the World Health Organization International Clinical Trials Registry Platform, and ClinicalTrials.gov were searched from inception until October 5, 2021. All cohort studies that reported patients diagnosed with KD who underwent an initial IVIG of 2 g/kg were selected. Study and patient characteristics and model performance measures. Two authors independently extracted data from the studies. The Kobayashi, Egami, Sano, Formosa, and Harada scores were the only prediction models with 3 or more external validation of the161 model analyses in 48 studies. The summary C-statistics were 0.65 (95% confidence interval [CI]: 0.57-0.73), 0.63 (95% CI: 0.55-0.71), 0.58 (95% CI: 0.55-0.60), 0.50 (95% CI: 0.36-0.63), and 0.63 (95% CI: 0.44-0.78) for the Kobayashi, Egami, Sano, Formosa, and Harada models, respectively. All 5 models showed low positive predictive values (0.14-0.39) and high negative predictive values (0.85-0.92). Potential differences in the characteristics of the target population among studies and lack of assessment of calibrations. None of the 5 prediction models with external validation accurately distinguished between patients with and without IVIG resistance.