Validation Of Prediction Model Research Articles

Quality and customer loyalty in the port services of Manzanillo

The customs industry, particularly within the Mexican context, plays a significant role both domestically and internationally, owing to its contribution to the global economy. With the expansion and standardization of customs clearance services, there has emerged a necessity for providers of these services to seek customer loyalty. Grounded in the principles of quality, the study examines the elements of the SERVQUAL model: tangibles, reliability, responsiveness, assurance, and empathy. Goals:The aim of the study was to extend the SERVQUAL model by incorporating customer attitude as an additional component within the three primary dimensions for measuring service quality. Methodology: A survey was employed to gather data from 100 customs brokerage firms through simple random sampling. Given the utilization of structural equation modeling (SEM) methodology to assess the theoretical model, Partial Least Squares (PLS) analysis technique was applied using Smart PLS 3.3.0 software. The findings indicated that the extended model had a significant impact on both customer satisfaction and loyalty within the customs facilities of Manzanillo, Colima. The coefficients of determination were 61.5% for customer satisfaction and 54% for customer loyalty, with a root mean square error of approximation (SRMR) of 0.065, confirming the model's predictive validity. Contributions: Customs facilities should focus their strategies on addressing issues that may be impacting service quality, particularly concerning sensitivity, empathy, and reliability factors, as these have a moderate level of influence on quality determination, according to customs agencies' perceptions. Regarding the inherent limitations of online surveys and their application in specialized groups, it is crucial to expand the research to include other clients of the customs facility.

Effect of optical diagnosis training on recognition and treatment of submucosal invasive colorectal cancer in community hospitals: a prospective multicenter intervention study.

Recognition of submucosal invasive colorectal cancer (T1 CRC) is difficult, with sensitivities of 35 %-60 % in Western countries. We evaluated the real-life effects of training in the OPTICAL model, a recently developed structured and validated prediction model, in Dutch community hospitals. In this prospective multicenter study (OPTICAL II), 383 endoscopists from 40 hospitals were invited to follow an e-learning program on the OPTICAL model, to increase sensitivity in detecting T1 CRC in nonpedunculated polyps. Real-life recognition of T1 CRC was then evaluated in 25 hospitals. Endoscopic and pathologic reports of T1 CRCs detected during the next year were collected retrospectively, with endoscopists unaware of this evaluation. Sensitivity for T1 CRC recognition, R0 resection rate, and treatment modality were compared for trained vs. untrained endoscopists. 1 year after e-learning, 528 nonpedunculated T1 CRCs were recorded for endoscopies performed by 251 endoscopists (118 [47 %] trained). Median T1 CRC size was 20 mm. Lesions were mainly located in the distal colorectum (66 %). Trained endoscopists recognized T1 CRCs more frequently than untrained endoscopists (sensitivity 74 % vs. 62 %; mixed model analysis odds ratio [OR] 2.90, 95 %CI 1.54-5.45). R0 resection rate was higher for T1 CRCs detected by trained endoscopists (69 % vs. 56 %; OR 1.73, 95 %CI 1.03-2.91). Training in optical recognition of T1 CRCs in community hospitals was associated with increased recognition of T1 CRCs, leading to higher en bloc and R0 resection rates. This may be an important step toward more organ-preserving strategies.

Development and validation of a social alienation predictive model for older maintenance hemodialysis patients based on latent profile analysis—a cross-sectional study

BackgroundSocial alienation refers to the state of feeling isolated, helpless, and unsatisfied due to maintaining distance from others or avoiding social interaction and activities. This phenomenon is caused by a lack of social skills, social anxiety, physical health problems, and other reasons. Older maintenance hemodialysis patients are exposed to a higher risk of social alienation. However, previous studies have been performed using the total score of the scale, which does not allow the identification of the characteristics of various patient groups with different levels of social alienation. In contrast, latent profile analysis can classify individuals into different categories based on continuous observational indicators, which improves accuracy and provides a more objective assessment by accounting for the uncertainty of variables. Given the concealed nature of social alienation and the differences in characteristics and treatment measures between different profiles, developing a predictive model for social alienation in older maintenance hemodialysis patients holds significance.ObjectiveTo explore the latent profile analysis of social alienation in older maintenance hemodialysis patients and to develop and validate a predictive model for social alienation in this population.MethodsA total of 350 older maintenance hemodialysis patients were selected as the study subjects using convenience sampling. A cross-sectional survey was conducted using a general information questionnaire, the Generalized Alienation Scale, and the Self-Perceived Burden Scale. Based on the results of the Generalized Alienation Scale, a latent profile analysis was performed, followed by univariate analysis and multinomial logistic regression to develop a predictive model. The effectiveness of the predictive model was evaluated in terms of its authenticity, reliability, and predictive ability.ResultsThree hundred nineteen valid questionnaires were collected. The social alienation of older maintenance hemodialysis patients based on latent profile analysis were divided into three profiles, which were named the low/medium/high-symptom groups, comprising 21%, 38.9%, and 40.1% of participants, respectively. Based on male, monthly social activity hours, Age-Adjusted Charlson Comorbidity Index, dialysis age, and Self-Perceived Burden Scale, a predictive model of social alienation for older maintenance hemodialysis patients was developed, and the Hosmer–Lemeshow tests showed no statistical significance (P > 0.05). The model has high predictive efficiency in authenticity, reliability and predictability.ConclusionOlder maintenance hemodialysis patients exhibited moderate to high levels of social alienation. The latent profile analysis based method was used to divide patients into low/medium/high-symptom profiles, and the predictive model demonstrates excellent authenticity, reliability, and predictability.

Prognostic biomarkers of intracerebral hemorrhage identified using targeted proteomics and machine learning algorithms.

Early prognostication of patient outcomes in intracerebral hemorrhage (ICH) is critical for patient care. We aim to investigate protein biomarkers' role in prognosticating outcomes in ICH patients. We assessed 22 protein biomarkers using targeted proteomics in serum samples obtained from the ICH patient dataset (N = 150). We defined poor outcomes as modified Rankin scale score of 3-6. We incorporated clinical variables and protein biomarkers in regression models and random forest-based machine learning algorithms to predict poor outcomes and mortality. We report Odds Ratio (OR) or Hazard Ratio (HR) with 95% Confidence Interval (CI). We used five-fold cross-validation and bootstrapping for internal validation of prediction models. We included 149 patients for 90-day and 144 patients with ICH for 180-day outcome analyses. In multivariable logistic regression, UCH-L1 (adjusted OR 9.23; 95%CI 2.41-35.33), alpha-2-macroglobulin (aOR 5.57; 95%CI 1.26-24.59), and Serpin-A11 (aOR 9.33; 95%CI 1.09-79.94) were independent predictors of 90-day poor outcome; MMP-2 (aOR 6.32; 95%CI 1.82-21.90) was independent predictor of 180-day poor outcome. In multivariable Cox regression models, IGFBP-3 (aHR 2.08; 95%CI 1.24-3.48) predicted 90-day and MMP-9 (aOR 1.98; 95%CI 1.19-3.32) predicted 180-day mortality. Machine learning identified additional predictors, including haptoglobin for poor outcomes and UCH-L1, APO-C1, and MMP-2 for mortality prediction. Overall, random forest models outperformed regression models for predicting 180-day poor outcomes (AUC 0.89), and 90-day (AUC 0.81) and 180-day mortality (AUC 0.81). Serum biomarkers independently predicted short-term poor outcomes and mortality after ICH. Further research utilizing a multi-omics platform and temporal profiling is needed to explore additional biomarkers and refine predictive models for ICH prognosis.

Harnessing Computational Modeling for Efficient Drug Design Strategies

Abstract: Computational modeling has become a crucial tool in drug design, offering efficiency and cost-effectiveness. This paper discusses the various computational modeling techniques used in drug design and their role in enabling efficient drug discovery strategies. Molecular docking predicts the binding affinity of a small molecule to a target protein, allowing the researchers to identify potential lead compounds and optimize their interactions. Molecular dynamics simulations provide insights into protein-ligand complexes, enabling the exploration of conformational changes, binding free energies, and fundamental protein-ligand interactions. Integrating computational modeling with machine learning algorithms, such as QSAR modeling and virtual screening, enables the prediction of compound properties and prioritizes potential drug candidates. High-performance computing resources and advanced algorithms are essential for accelerating drug design workflows, with parallel computing, cloud computing, and GPU acceleration reducing computational time. The paper also addresses the challenges and limitations of computational modeling in drug design, such as the accuracy of scoring functions, protein flexibility representation, and validation of predictive models. It emphasizes the need for experimental validation and iterative refinement of computational predictions to ensure the reliability and efficacy of designed drugs.

Establishment and validation of novel predictive models to predict bone metastasis in newly diagnosed prostate adenocarcinoma based on single-photon emission computed tomography radiomics.

To establish and validate novel predictive models for predicting bone metastasis (BM) in newly diagnosed prostate adenocarcinoma (PCa) via single-photon emission computed tomography radiomics. In a retrospective review of the clinical single-photon emission computed tomography (SPECT) database, 176 patients (training set: n = 140; validation set: n = 36) who underwent SPECT/CT imaging and were histologically confirmed to have newly diagnosed PCa from June 2016 to June 2022 were enrolled. Radiomic features were extracted from the region of interest (ROI) in a targeted lesion in each patient. Clinical features, including age, total prostate-specific antigen (t-PSA), and Gleason grades, were included. Statistical tests were then employed to eliminate irrelevant and redundant features. Finally, four types of optimized models were constructed for the prediction. Furthermore, fivefold cross-validation was applied to obtain sensitivity, specificity, accuracy, and area under the curve (AUC) for performance evaluation. The clinical usefulness of the multivariate models was estimated through decision curve analysis (DCA). A radiomics signature consisting of 27 selected features which were obtained by radiomics' LASSO treatment was significantly correlated with bone status (P < 0.01 for both training and validation sets). Collectively, the models showed good predictive efficiency. The AUC values ranged from 0.87 to 0.98 in four models. The AUC values of the human experts were 0.655 and 0.872 in the training and validation groups, respectively. Most radiomic models showed better diagnostic accuracy than human experts in the training and validation groups. DCA also demonstrated the superiority of the radiomics models compared to human experts. Radiomics models are superior to humans in differentiating between benign bone and prostate cancer bone metastases; it can be used to facilitate personalized prediction of BM in newly diagnosed PCa patients.

PREVECOL, a predictive model for early detection of colorectal cancer and precancerous lesions in liquid biopsy.

3629 Background: Colorectal cancer (CRC) is the second cause of cancer-related deaths worldwide. Most of these deaths could have been prevented by increasing the effectiveness of CRC screening programs, which exhibit a low participation rate, partly due to the limited available screening options, alongside discomfort associated with the screening procedure (faecal test). Liquid biopsy-based technology offers new alternatives to improve CRC early detection. CRC and advanced adenoma (AA) patients show molecular alterations at different biological levels that can be detected in peripheral blood and used for diagnostic purposes. AMADIX has developed PreveCol, an innovative blood test based on molecular biomarkers and clinical features for early detection of CRC and AA, before the onset of the first symptoms. We present the validation of PreveCol predictive model through a European prospective clinical study involving average-risk population actively engaged in a real CRC screening program. PreveCol has been granted with the Breakthrough Device Designation by the FDA in December 2023. Methods: Samples from two prospective clinical studies were used for training and validation. A case-control study (n= 538 individuals: 173 CRC, 181 AA, 184 Controls; 13 Spanish sites) and a clinical study based on a real CRC screening program (n= 3163 individuals: 78 CRC, 156 AA, 146 HRPL, and 2783 Controls; 9 European sites). PreveCol molecular signature (11 proteins and 10 miRNAs), was determined in plasma samples using RT-qPCR and ELISA/CLIA. Machine learning techniques were used to develop an ensemble algorithm. K-fold cross validation techniques were used in the training to set the hyperparameters for each model. Hyperparameter optimalization was performed to find the best parameters of the grid. Once the algorithm was locked and the clinical cut-off was established in the training set, the signature was validated in the European prospective study sub-cohort. Results: For CRC detection, PreveCol demonstrates an Area Under the ROC curve (AUC) of 92%, with a sensitivity (SS) of 81.8% and specificity (SP) of 86%. Moreover, the model exhibits high sensitivity (above 90%) in identifying CRC in early stages (stages 1 and 2), as well as lesions located in the proximal colon. In addition, the AUC of PreveCol for Advanced Adenomas detection is 86.88%, (SS of 70.8% and SP of 86%), showing that PreveCol presents higher sensitivity compared to other identified tests for detecting precancerous lesions. Conclusions: These findings highlight PreveCol's potential to significantly impact on patient prognosis and survival by facilitating the early detection of CRC and premalignant lesions as well as lesions in critical areas of the colon, thereby contributing to improved screening efficacy and patient outcomes.

Incorporating preoperative frailty to assist in early prediction of postoperative pneumonia in elderly patients with hip fractures: an externally validated online interpretable machine learning model

BackgroundThis study aims to implement a validated prediction model and application medium for postoperative pneumonia (POP) in elderly patients with hip fractures in order to facilitate individualized intervention by clinicians.MethodsEmploying clinical data from elderly patients with hip fractures, we derived and externally validated machine learning models for predicting POP. Model derivation utilized a registry from Nanjing First Hospital, and external validation was performed using data from patients at the Fourth Affiliated Hospital of Nanjing Medical University. The derivation cohort was divided into the training set and the testing set. The least absolute shrinkage and selection operator (LASSO) and multivariable logistic regression were used for feature screening. We compared the performance of models to select the optimized model and introduced SHapley Additive exPlanations (SHAP) to interpret the model.ResultsThe derivation and validation cohorts comprised 498 and 124 patients, with 14.3% and 10.5% POP rates, respectively. Among these models, Categorical boosting (Catboost) demonstrated superior discrimination ability. AUROC was 0.895 (95%CI: 0.841–0.949) and 0.835 (95%CI: 0.740–0.930) on the training and testing sets, respectively. At external validation, the AUROC amounted to 0.894 (95% CI: 0.821–0.966). The SHAP method showed that CRP, the modified five-item frailty index (mFI-5), and ASA body status were among the top three important predicators of POP.ConclusionOur model’s good early prediction ability, combined with the implementation of a network risk calculator based on the Catboost model, was anticipated to effectively distinguish high-risk POP groups, facilitating timely intervention.

Optimizing Insulated-Gate Bipolar Transistors’ Lifetime Estimation: A Critical Evaluation of Lifetime Model Adjustments Based on Power Cycling Tests

This paper presents a detailed refinement and validation of two well-known lifetime prediction models for IGBTs, namely CIPS08 and SKiM63, using experimental power cycling test data. This study focuses on adapting these models to reflect the operational conditions and degradation patterns to more accurately fit different IGBT types and applications. Key modifications include recalibrating the scale factor and temperature coefficients in the SKiM63 model and refining the CIPS08 model coefficients (β1 = −2.910, β2 = 1083.714, β3 = −4.521) based on the impact of temperature fluctuations, bond wire diameter, and electrical stresses observed during power cycling tests. These adjustments provide a significant shift from traditional values, with the recalibrated models offering a better fit, as evidenced by a reasonable coefficient of determination (R2) and root mean square error (RMSE). Utilizing Monte Carlo simulations with a 5% uncertainty, the study calculates the B10 lifetimes of PV inverters, demonstrating a substantial reduction from 43 years in the unmodified model to 13 years in the modified model. This emphasizes the critical need for ongoing modification and validation of predictive models based on the actual operational data to enhance the reliability and efficiency of IGBTs in power electronic systems.

Development and Validation of Machine Learning-Based Predictive Model for Prolonged Hospital Stay after Decompression Surgery for Lumbar Spinal Canal Stenosis.

Precise prediction of hospital stay duration is essential for maximizing resource utilization during surgery. Existing lumbar spinal stenosis (LSS) surgery prediction models lack accuracy and generalizability. Machine learning can improve accuracy by considering preoperative factors. This study aimed to develop and validate a machine learning-based model for estimating hospital stay duration following decompression surgery for LSS. Data from 848 patients who underwent decompression surgery for LSS at three hospitals were examined. Twelve prediction models, using 79 preoperative variables, were developed for postoperative hospital stay estimation. The top five models were chosen. Fourteen models predicted prolonged hospital stay (≥14 days), and the most accurate model was chosen. Models were validated using a randomly divided training sample (70%) and testing cohort (30%). The top five models showed moderate linear correlations (0.576-0.624) between predicted and measured values in the testing sample. The ensemble of these models had moderate prediction accuracy for final length of stay (linear correlation 0.626, absolute mean error 2.26 days, standard deviation 3.45 days). The c5.0 decision tree model was the top predictor for prolonged hospital stay, with accuracies of 89.63% (training) and 87.2% (testing). Key predictors for longer stay included JOABPEQ social life domain, facility, history of vertebral fracture, diagnosis, and Visual Analogue Scale (VAS) of low back pain. A machine learning-based model was developed to predict postoperative hospital stay after LSS decompression surgery, using data from multiple hospital settings. Numerical prediction of length of stay was not very accurate, although favorable prediction of prolonged stay was accomplished using preoperative factors. The JOABPEQ social life domain score was the most important predictor.

Modeling Cost-Estimation Factors for Public Building Projects with Hybrid Approach in Addis Ababa

Assessing the most important cost-influencing factors is essential for enhancing the predictive ability of cost estimation for building construction projects. The goal of this study is to examine and design a valid cost prediction model for assessing factors that impact the cost estimation of public buildings in Addis Ababa. This research solves these issues that typically arise in predictive cost estimation models in two major processes. First, the insights of 133 professionals gathered on the 38 cost-impacting elements, and 15 top factors design, time or cost, and parties’ experience were determined. The suggested hybrid approach is based on the Akaike information criterion (AIC) and principal component regression (PCR) employed, coupling a stepwise linear regression model. According to the findings of the study, principal component analysis reduced important factors to 14 and efficiently solved the problem of multicollinearity with a variance inflation factor of less than 2, while stepwise cross-validation solved the overfitting problem at the lowest AIC. The cost prediction model sorted out five factors: design completion by the public body when bids are invited; completion of the project scope definition when bids are invited; level of construction complexity; importance of project completion within budget; and subcontractor experience and capability have all been identified as the main cost-determining factors. The study’s contribution is the first approach (PCR–AIC) utilized in this work to explore numerous cost-estimating components, eliminate those that were related to one another, and identify the most crucial ones that consisted of the majority of the original variables’ attributes.

Development and validation of a machine learning-based prediction model for sudden cardiac death in the general population: insights from the community cardiovascular cohort

Abstract Background/Introduction Accurately predicting and managing the risk of sudden cardiac death (SCD) in the general population is a formidable challenge. Purpose This study aimed to develop and validate a machine learning (ML)-based prediction model for SCD using data from the Community Cardiovascular Cohort (CCCC) in Taiwan. Methods A community-based cohort study was used for model construction . Four ML algorithms, including extreme gradient boosting (XGBoost), random forests (RF), logistic regression, and deep neural network (DNN), underwent meticulous evaluation. Performance assessment encompassed receiver operating characteristic (ROC) curves, area under the curves (AUC), and accuracy metrics. Significant risk factors were identified through feature importance analysis. Results We employed a random partitioning strategy, allocating 1744 participants (55%) to the training dataset, 952 participants (30%) to the test dataset, and 476 participants (15%) to the validation dataset. In the data training stage, the XGBoost algorithm demonstrated outstanding prediction performance for SCD (AUC: 1.000); In the validation stage, the XGBoost algorithm demonstrated good prediction performance for SCD prediction (AUC: 0.791, 95%CI: 0.617-0.964; Accuracy: 0.971). Using XGBoost-generated predicted probabilities (P(XGB)), we identified individuals with P(XGB) ≥ 0.5 (Group 1) who displayed significantly higher SCD risk (HR 32.0; 95% CI: 9.29-109.9) compared to those with P(XGB) &amp;lt; 0.5 (Group 0). Conclusions Our study demonstrates the effectiveness of ML algorithms, particularly XGBoost, in predicting event of SCD in the general population. The identification of these pivotal risk factors holds profound implications, empowering the identification of high-risk individuals and facilitating the implementation of precisely tailored preventive strategies.Figure 1:KM sruvival curveFigure 2:Feature importance analysis

Development and validation of early prediction models for new-onset functional impairment in patients after being transferred from the ICU

A significant number of intensive care unit (ICU) survivors experience new-onset functional impairments that impede their activities of daily living (ADL). Currently, no effective assessment tools are available to identify these high-risk patients. This study aims to develop an interpretable machine learning (ML) model for predicting the onset of functional impairment in critically ill patients. Data for this study were sourced from a comprehensive hospital in China, focusing on adult patients admitted to the ICU from August 2022 to August 2023 without prior functional impairments. A least absolute shrinkage and selection operator (LASSO) model was utilized to select predictors for inclusion in the model. Four models, logistic regression, support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost), were constructed and validated. Model performance was assessed using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Additionally, the DALEX package was employed to enhance the interpretability of the final models. The study ultimately included 1,380 patients, with 684 (49.6%) exhibiting new-onset functional impairment on the seventh day after leaving the ICU. Among the four models evaluated, the SVM model demonstrated the best performance, with an AUC of 0.909, accuracy of 0.838, sensitivity of 0.902, specificity of 0.772, PPV of 0.802, and NPV of 0.886. ML models are reliable tools for predicting new-onset functional impairments in critically ill patients. Notably, the SVM model emerged as the most effective, enabling early identification of patients at high risk and facilitating the implementation of timely interventions to improve ADL.

Validation of predictive models for lesion transmurality and steam-pop using ex-vivo porcine bipolar ablation model

Abstract Introduction Bipolar ablation has been reported to create deeper myocardial ablation lesion as compared to unipolar ablation, however, there is currently no standardized strategy for bipolar ablation settings such as energy and ablation time. Purpose The aim of the present study is to test our hypothesis that both transmural lesion creation and steam-pop occurrence during bipolar ablation can be predicted by the settings of bipolar ablation and the myocardial tissue characteristics such as tissue thickness and initial impedance prior to RF. Methods Using porcine cardiac tissues, we conducted ex-vivo ablation experiments. Commercially available radiofrequency (RF) ablation catheters were utilized for the procedures. Each ablation catheter was positioned bilaterally on the myocardial slices at a 45-degree angle, maintaining a targeted contact force of 10g. We documented the occurrence of steam-pop under various conditions, modulating tissue thickness (5-20 mm), RF power (20, 30, 40, 50 Watt) and ablation duration (20 sec, 40 sec, 60 sec, 180 sec). Generalized linear models (GLM) were developed to forecast the creation of transmural lesion and steam-pop incidents, drawing on the bipolar ablation settings and myocardial tissue characteristics. Subsequently, we engaged in an additional series of experiments to validate the models. Results In total, 194 bipolar applications were utilized as training data. Transmural lesions and steam-pops were observed in 95 (49%), and 11 applications (5.7%), respectively. We constructed GLM using the RF energy (Watt), tissue thickness (mm), initial bipolar impedance (Ohm), impedance drop during the first 5 seconds of application (Ohm), and RF duration (seconds). For transmurality prediction the model validation was conducted with another independent 111 RF applications (102 transmural, and 25 steam-pops). The GLM, which utilized RF energy, tissue thickness, initial bipolar impedance, and RF duration, performed the best performance in predicting transmural lesion creation (AUC 0.94 [0.90-0.99], sensitivity 86 %, specificity 100 %). However, the exclusion of ‘tissue thickness’ from this model led to a significant decrease in its predictive performance (AUC 0.69 [0.48-0.89], sensitivity 47%, specificity 100%, P=0.0057). For pop prediction in validation data, the GLM utilizing RF energy, initial bipolar impedance and impedance drop showed the best performance (AUC 0.90 [0.83-0.97], sensitivity 76 %, specificity 92 %). Conclusion Our data show that accurate assessment of tissue thickness is pivotal for predicting transmural lesion creation. To predict lesion transmurality and steam-pop incidents through RF parameters and tissue characteristics within a beating heart, further research is warranted. This could facilitate the calibration of RF settings, potentially enhancing the safety of bipolar ablation procedures.

#1584 Development and validation of risk prediction models according to different clinical settings of IgA nephropathy: nationwide multicenter cohort study

Abstract Background and Aims IgA nephropathy (IgAN) is the most common glomerulonephritis with a high risk of progression to end-stage renal disease (ESRD). Because machine learning models developed for research have limited use in clinical settings, this study developed and validated prediction models for IgAN according to different clinical settings. Method In this study, we analyzed data from 1,174 patients in the Japanese Nationwide Retrospective Cohort Study in IgAN, focusing on a 10-year observation period. We defined the derivation data from January 2002 to April 2004, and temporal validation data from May 2004 to December 2004. The composite renal outcome was defined as a 1.5-fold increase in creatinine or progression to ESRD. We developed and evaluated the three prediction models according to the different clinical settings: the primary care model for general physicians in primary care settings, the tertiary care model for specialists in tertiary care hospitals, and the research institute model, a machine learning-based approach for academic research institute settings. Results After excluding 82 patients for missing data, 114 and 14 patients experienced the composite renal outcome in the derivation (n = 874) and validation (n = 218) cohorts, respectively. The primary care model identified three variables—eGFR &amp;lt;45 mL/min/1.73 m², proteinuria ≥0.5 g/day, and non-use of corticosteroids—as significant predictors (C-statistic: 0.796; 95% CI: 0.686-0.895). The tertiary care model composed of three variables including glomerular number, histological severity and tubular/interstitial severity showed a slightly higher C-statistic (0.807; 95% CI: 0.713-0.886). When stratifying patients into risk groups, the primary care model showed a higher incidence of composite renal events in high-risk patients, and the tertiary care model effectively discriminated outcomes in all groups. The research institute model, which included 38 variables including proteinuria level, hypertension, eGFR, and non-use of corticosteroids as key predictors, showed comparable performance (C-statistic: 0.808; 95% CI: 0.689-0.886). Conclusion The primary care and tertiary care models provide simple yet effective predictive tools, competitive with the machine learning-based research institution model. These models are critical for predicting renal outcomes in patients with IgA nephropathy and for patient management according to different clinical settings.

#560 The representation of multimorbidity and frailty in the development and validation of kidney failure risk prediction models – a systematic review

Abstract Background and Aims Prognostic models that identify individuals with chronic kidney disease (CKD) at highest risk of developing kidney failure help clinicians to make decisions and deliver precision medicine. People with CKD are very likely to have multiple long-term health conditions (multimorbidity) and often experience frailty. These factors impact progression of kidney disease and influence the risk of other outcomes such as death. It is unclear to what extent prognostic models, that estimate the risk of kidney failure, consider the impact of multimorbidity and frailty and whether these models are valid in such sub-populations. This systematic review (CRD42022347295) describes and evaluates the representation of multimorbidity and frailty within cohorts used to develop and/or validate prognostic models assessing the risk of kidney failure in individuals with CKD. The review aims to determine if multimorbidity or frailty has been considered in relation to prediction of kidney failure and if reliable prognostic models exist for use in this population. Method We included studies that described the derivation, validation or update of a kidney failure prognostic model (outcome assessed at ≥2 years) for use in adults with CKD and reported at least one measure of either discrimination or calibration. The primary outcome for the review was the representation of multimorbidity or frailty in these models. An electronic search for published peer-reviewed articles involved MEDLINE, CINAHL Plus and the Cochrane Library – CENTRAL and was supplemented with manual review of references from previous systematic reviews and clinical guidelines. Results A total of 97 studies, reporting 121 different kidney failure prognostic models were identified. A total of 2 925 413 participants and 149 380 kidney failure events were included across all studies. Included participants had a mean age of 58.9 years (SD 9.6), 44.4% were female, with a mean eGFR of 47.5 ml/min/1.73 m2 (SD 10.7). Only two studies reported multimorbidity, measured by Elixhauser Comorbidity Index and Charlson Co-morbidity Index (CCI), and a single study reported a self-reported proxy frailty measure. The rates of specific co-morbidities were reported in a greater proportion of studies: 67.0% (n = 65) studies reported baseline data on diabetes, 54.6% (n = 53) reported hypertension, and 39.2% (n = 38) reported cardiovascular disease. No studies included frailty in model development and only one study considered multimorbidity as a predictor variable (via CCI). No studies assessed model performance in populations with multimorbidity. A single study assessed clinical utility of a model and referral algorithms considering frailty and the risk of kidney failure and death. Conclusion There is a paucity of kidney failure risk prediction models that consider the impact of multimorbidity and/or frailty in adults with CKD, resulting in a lack of clear evidence-based practice and guidance for multimorbid or frail individuals. These knowledge gaps should be explored to help clinicians know whether these models can be used for patients who experience multimorbidity and/or frailty.

Using 30-day modified rankin scale score to predict 90-day score in patients with intracranial hemorrhage: Derivation and validation of prediction model.

Whether 30-day modified Rankin Scale (mRS) scores can predict 90-day scores is unclear. This study derived and validated a model to predict ordinal 90-day mRS score in an intracerebral hemorrhage (ICH) population using 30-day mRS values and routinely available baseline variables. Adults enrolled in the Antihypertensive Treatment of Acute Cerebral Hemorrhage-2 (ATACH-2) trial between May 2011 and September 2015 with acute ICH, who were alive at 30 days and had mRS scores reported at both 30 and 90 days were included in this post-hoc analysis. A proportional odds regression model for predicting ordinal 90-day mRS scores was developed and internally validated using bootstrapping. Variables in the model included: mRS score at 30 days, age (years), hematoma volume (cm3), hematoma location (deep [basal ganglia, thalamus], lobar, or infratentorial), presence of intraventricular hemorrhage (IVH), baseline Glasgow Coma Scale (GCS) score, and National Institutes of Health Stroke Scale (NIHSS) score at randomization. We assessed model fit, calibration, discrimination, and agreement (ordinal, dichotomized functional independence), and EuroQol-5D ([EQ-5D] utility weighted) between predicted and observed 90-day mRS. A total of 898/1000 participants were included. Following bootstrap internal validation, our model (calibration slope = 0.967) had an optimism-corrected c-index of 0.884 (95% CI = 0.873-0.896) and R2 = 0.712 for 90-day mRS score. The weighted ĸ for agreement between observed and predicted ordinal 90-day mRS score was 0.811 (95% CI = 0.787-0.834). Agreement between observed and predicted functional independence (mRS score of 0-2) at 90 days was 74.3% (95% CI = 69.9-78.7%). The mean ± SD absolute difference between predicted and observed EQ-5D-weighted mRS score was negligible (0.005 ± 0.145). This tool allows practitioners and researchers to utilize clinically available information along with the mRS score 30 days after ICH to reliably predict the mRS score at 90 days.

Development, comparison, and internal validation of prediction models to determine the visual prognosis of patients with open globe injuries using machine learning approaches

IntroductionOpen globe injuries (OGI) represent a main preventable reason for blindness and visual impairment, particularly in developing countries. The goal of this study is evaluating key variables affecting the prognosis of open globe injuries and validating internally and comparing different machine learning models to estimate final visual acuity.Materials and methodsWe reviewed three hundred patients with open globe injuries receiving treatment at Khatam-Al-Anbia Hospital in Iran from 2020 to 2022. Age, sex, type of trauma, initial VA grade, relative afferent pupillary defect (RAPD), zone of trauma, traumatic cataract, traumatic optic neuropathy (TON), intraocular foreign body (IOFB), retinal detachment (RD), endophthalmitis, and ocular trauma score (OTS) grade were the input features. We calculated univariate and multivariate regression models to assess the association of different features with visual acuity (VA) outcomes. We predicted visual acuity using ten supervised machine learning algorithms including multinomial logistic regression (MLR), support vector machines (SVM), K-nearest neighbors (KNN), naïve bayes (NB), decision tree (DT), random forest (RF), bagging (BG), adaptive boosting (ADA), artificial neural networks (ANN), and extreme gradient boosting (XGB). Accuracy, positive predictive value (PPV), recall, F-score, brier score (BS), Matthew correlation coefficient (MCC), receiver operating characteristic (AUC-ROC), and calibration plot were used to assess how well machine learning algorithms performed in predicting the final VA.ResultsThe artificial neural network (ANN) model had the best accuracy to predict the final VA. The sensitivity, F1 score, PPV, accuracy, and MCC of the ANN model were 0.81, 0.85, 0.89, 0.93, and 0.81, respectively. In addition, the estimated AUC-ROC and AUR-PRC of the ANN model for OGI patients were 0.96 and 0.91, respectively. The brier score and calibration log-loss for the ANN model was 0.201 and 0.232, respectively.ConclusionAs classic and ensemble ML models were compared, results shows that the ANN model was the best. As a result, the framework that has been presented may be regarded as a good substitute for predicting the final VA in OGI patients. Excellent predictive accuracy was shown by the open globe injury model developed in this study, which should be helpful to provide clinical advice to patients and making clinical decisions concerning the management of open globe injuries.

Enhancing the validity of mortality prediction models in orthopedic surgery: two critical evaluations.

Enhancing the validity of mortality prediction models in orthopedic surgery: two critical evaluations.

Predicting Hospitalization Risk Among Home Care Residents in the United Kingdom: Development and Validation of a Machine Learning-Based Predictive Model

Preventable hospital admissions in elderly home care residents are a major socioeconomic burden, whilst early detection of deterioration may improve outcomes. Our goal was to develop and validate a machine learning-based algorithm to predict hospitalization risk among home care users. Our primary outcome was hospitalization. An existing risk score (1-5) was assessed for its discriminatory capacity over time. We subsequently developed a new machine learning model using carer concerns, service user demographics, and other home care data between January and July 2021. We randomly selected 150 service user records for validation, which were evaluated by both the model and 10 clinicians (9 doctors and 1 nurse) to compare prediction time and accuracy to human experts. Comparison between model and human was via area under the receiver operating characteristic curve (AUC). A score of 5 conferred an 8x higher likelihood of hospitalization in the subsequent 7 days (15.4% vs 1.8%, p &lt; .05), compared to a score of 1. The new model and risk score increased performance, detecting 182 hospitalizations/month (3.7x chance). The AUC for the model was significantly higher than for clinicians (0.87 vs 0.41-0.57, respectively; p &lt; .05). The model took &lt;1 minute, while clinicians typically took over 40 minutes. A risk prediction model using carer concerns and other home care data features detects 3.7x more hospitalizations than chance. The model is faster and more accurate than human clinicians, enabling low-cost scale-up. This study supports linking the model to a triage and intervention service to reduce preventable hospitalizations in the home care sector.