Prediction Model Risk Of Bias Assessment Tool Research Articles

Association between sociodemographic factors and mobility limitation among older adults: a systematic review and meta-analysis protocol

BackgroundMobility is an independent predictor of physical functionality, healthy ageing, and quality of life. Various literatures have associated mobility limitation in older adulthood with demographic and socioeconomic factors. Hence, we propose a systematic review and meta-analysis to synthesise the association between sociodemographic factors and mobility limitations in older adults.Methods and analysesThis protocol was written according to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) guidelines. We will perform a comprehensive search of all observational studies that assessed the relationship between age, gender, race, place, education, income, occupation, social status, and walking distance, time, or speed. Electronic databases (MEDLINE, Web of Science, EMBASE, CINAHL, AgeLine, and SPORTDiscus) will be searched from inception to 28 February 2023. We will supplement the database search by manually searching the reference lists of all identified and relevant full-text articles. Two independent reviewers will be responsible for screening articles, data extraction, and assessment of bias. We will appraise the study quality and risk of bias using the Prediction Model Risk of Bias Assessment Tool (PROBAST). A meta-analysis will be considered if data from the selected studies are homogeneous, otherwise, a narrative synthesis of the extracted data will be presented.DiscussionMobility limitation leads to frequent falls, dependency, morbidity, and death among older adults. This review is necessary, to identify and prioritise important sociodemographic factors during older adults’ clinical assessment and policy development. It is the first phase of a multi-methods study seeking to develop a prognostic mobility trajectory for community-dwelling older adults.Systematic review registrationPROSPERO CRD42022298570

Prognosticating Acute Ischemic Stroke and Estimating the Feasibility of Mapping Stroke Volume to the Functional Outcomes Using Diffusion-Weighted Images: A Systematic Review Protocol.

Diffusion-weighted image or DWI is commonly used to provide valuable and diverse information on acute stroke in tertiary care hospitals. DWI is a sensitive and accurate method for identifying the infarct core and can expose the area of cerebral infarction within a few hours of onset. This systematic review is planned to evaluate the measurement of stroke volume on DWI and correlate it with functional outcomes (modified ranking scale). We have adhered to the PRISMA-P checklist to report this systematic review protocol. PubMed, Web of Science, Scopus, and TRIP (Turning Research into Practice) databases will be searched. Two independent reviewers will screen the records, extract data, and critically appraise the studies. A checklist for critical appraisal will be applied for data abstraction, and data extraction will be done using predictive modeling for systematic reviews. The risk of bias will be measured by the Prediction Model Risk of Bias Assessment Tool (PROBAST). The meta-analysis will be considered only if included studies have adequate data, and STATA statistical package version 13.1 will be used for performing a meta-analysis. A narrative synthesis will be performed if meta-analysis is not possible. As this review will focus on secondary information, there is no ethical consideration required. We will disseminate our findings by publishing our analysis in a peer-reviewed journal. In Prospective Register of Systematic Reviews (CRD42019141840).

Application of machine learning methods in predicting schizophrenia and bipolar disorders: A systematic review.

Schizophrenia and bipolar disorder (BD) are critical and high-risk inherited mental disorders with debilitating symptoms. Worldwide, 3% of the population suffers from these disorders. The mortality rate of these patients is higher compared to other people. Current procedures cannot effectively diagnose these disorders because it takes an average of 10 years from the onset of the first symptoms to the definitive diagnosis of the disease. Machine learning (ML) techniques are used to meet this need. This study aimed to summarize information on the use of ML techniques for predicting schizophrenia and BD to help early and timely diagnosis of the disease. A systematic literature search included articles published until January 19, 2020 in 3 databases. Two reviewers independently assessed original papers to determine eligibility for inclusion in this review. PRISMA guidelines were followed to conduct the study, and the Prediction Model Risk of Bias Assessment Tool (PROBAST) to assess included papers. In this review, 1243 papers were retrieved through database searches, of which 15 papers were included based on full-text assessment. ML techniques were used to predict schizophrenia and BDs. The main algorithms applied were support vector machine (SVM) (10 studies), random forests (RF) (5 studies), and gradient boosting (GB) (3 studies). Input and output characteristics were very diverse and have been kept to enable future research. RFs algorithms demonstrated significantly higher accuracy and sensitivity than SVM and GB. GB demonstrated significantly higher specificity than SVM and RF. We found no significant difference between RF and SVM in terms of specificity. ML can precisely predict results and assist in making clinical decisions-concerning schizophrenia and BD. RF often performed better than other algorithms in supervised learning tasks. This study identified gaps in the literature and opportunities for future psychological ML research.

Predictive value of machine learning in diagnosing cognitive impairment in patients with Parkinson's disease: a systematic review and meta-analysis.

The current process used to diagnose cognitive impairment in patients with Parkinson's disease (PD) is unsatisfactory. More and more researchers had introduced machine learning into this field in recent years. This study explored the application of machine learning and its diagnostic performance in this field. Since Parkinson's concurrent cognitive impairment is currently divided into different periods, most studies focus on the prodromal or early stages of Parkinson's cognitive impairment, and a few focuses on the dementia stage of Parkinson's. To ensure comprehensiveness, and model stability, we included patients with Parkinson's concurrent cognitive impairment in different periods who met the nadir criteria. A comprehensive literature search was carried out of the PubMed, Cochrane, Embase, and Web of Science databases. We used Prediction Model Risk of Bias Assessment Tool (PROBAST) to assess the risk of bias for the machine learning models covered by the included original studies. The outcome indicators included the concordance-index (C-index), sensitivity, and specificity. A meta-analysis using the random-effects model was conducted to determine the C-index, and a double variable mixed-effects model was used to determine the sensitivity and specificity. The meta-analysis in this article was completed in STATA. A total of 32 articles, comprising 10,778 patients and 51 prognostic models [summary c-statistic: 0.857, 95% confidence interval (CI) (0.842-0.873)], met the selection criteria and were included in this analysis. The total sensitivity and specificity of all models were 0.77 (95% CI: 0.72-0.81) and 0.83 (95% CI: 0.80-0.85), respectively, and those of the testing test were 0.85 (95% CI: 0.79-0.89), and 0.74 (95% CI: 0.70-0.78), respectively. A large part of the model showed a high risk of bias mainly because the study design was almost retrospective investigation. This study constitutes a detailed mapping and assessment of the machine learning for prediction in PD patients with cognitive decline, which may provide stronger discriminative performance and can be used as a potential tool for early diagnosis.

Value of genomics- and radiomics-based machine learning models in the identification of breast cancer molecular subtypes: a systematic review and meta-analysis.

In the era of precision therapy, early classification of breast cancer (BRCA) molecular subtypes has clinical significance for disease management and prognosis. We explored the accuracy of machine learning (ML) models for early classification of BRCA molecular subtypes through a systematic review of the literature currently available. We retrieved relevant studies published in PubMed, EMBASE, Cochrane, and Web of Science until 15 April 2022. A prediction model risk of bias assessment tool (PROBAST) was applied for the assessment of risk of bias of a genomics-based ML model, and the Radiomics Quality Score (RQS) was simultaneously used to evaluate the quality of this radiomics-based ML model. A random effects model was adopted to analyze the predictive accuracy of genomics-based ML and radiomics-based ML for Luminal A, Luminal B, Basal-like or triple-negative breast cancer (TNBC), and human epidermal growth factor receptor 2 (HER2). The PROSPERO of our study was prospectively registered (CRD42022333611). Of the 38 studies were selected for analysis, 14 ML models were based on gene-transcriptomic, with only 4 external validations; and 43 ML models were based on radiomics, with only 14 external validations. Meta-analysis results showed that c-statistic values of the ML based on radiomics for the identification of BRCA molecular subtypes Luminal A, Luminal B, Basal-like or TNBC, and HER2 were 0.76 [95% confidence interval (CI): 0.60-0.96], 0.78 (95% CI: 0.69-0.87), 0.89 (95% CI: 0.83-0.91), and 0.83 (95% CI: 0.81-0.86), respectively. The c-statistic values of ML based on the gene-transcriptomic analysis cohort for the identification of the previously described BRCA molecular subtypes were 0.96 (95% CI: 0.93-0.99), 0.96 (95% CI: 0.93-0.99), 0.98 (95% CI: 0.95-1.00), and 0.97 (95% CI: 0.96-0.98) respectively. Additionally, the sensitivity of the ML model based on radiomics for each molecular subtype ranged from 0.79 to 0.85, while the sensitivity of the ML model based on gene-transcriptomic was between 0.92 and 0.99. Both radiomics and gene transcriptomics produced ideal effects on BRCA molecular subtype prediction. Compared with radiomics, gene transcriptomics yielded better prediction results, but radiomics was simpler and more convenient from a clinical point of view.

PP37 A Systematic Review of Machine Learning and Statistical Models For Predicting Coronary Heart Disease In Diabetic Patients

IntroductionRisk prediction models, using either machine learning or statistical algorithms, can act as inputs of a cost-effectiveness model when predicting costs and effectiveness of an intervention. This systematic review has two objectives: to evaluate methodological quality of the published models to predict diabetic coronary heart disease (CHD) risk; to evaluate whether the models were sufficiently reported to judge their applicability to the cost-effectiveness modelling.MethodsA targeted review of journal articles published in English, Dutch, Chinese, or Spanish was undertaken in PubMed, Embase, Scopus, Web of Science, and IEEE Explore from 1 January, 2016 to 31 May, 2021. To assess the methodological quality and reporting of the models, we used PROBAST (Prediction model Risk Of Bias Assessment Tool), CHARMS (a Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies), and a checklist (Betts 2019) summarizing the application of cardiovascular risk prediction models to health technology assessment.ResultsOur search retrieved 6,579 hits, of which 18 models were eligible for inclusion. Among them, four studies developed machine learning models (2 recurrent neural networks, 1 random forest models, and 1 multi-task learning model) while 14 studies developed statistical models (8 Cox models, 5 logistic models, and 1 microsimulation model). More than 70 percent of models were of high methodological quality in aspects of participants (89%), predictors (72%), and outcomes (72%), while only five models (28%) in aspects of statistical analysis. For the reporting, only two models provided sufficient evidence in all aspects (i.e., participants, predictors, and outcomes) for judging their applicability to the cost-effectiveness modelling. Most models were reported sufficiently regarding participants (78%) and outcomes (72%), but only three models regarding predictors (17%).ConclusionsTo apply the CHD risk prediction models to cost-effectiveness modelling, concerns remain regarding the potential risk of bias due to inappropriate use of analysis methods, and regarding insufficient reporting on how to measure and assess the predictors.

Passive Sensing in the Prediction of Suicidal Thoughts and Behaviors: Protocol for a Systematic Review.

Suicide is a severe public health problem, resulting in a high number of attempts and deaths each year. Early detection of suicidal thoughts and behaviors (STBs) is key to preventing attempts. We discuss passive sensing of digital and behavioral markers to enhance the detection and prediction of STBs. The paper presents the protocol for a systematic review that aims to summarize existing research on passive sensing of STBs and evaluate whether the STB prediction can be improved using passive sensing compared to prior prediction models. A systematic search will be conducted in the scientific databases MEDLINE, PubMed, Embase, PsycINFO, and Web of Science. Eligible studies need to investigate any passive sensor data from smartphones or wearables to predict STBs. The predictive value of passive sensing will be the primary outcome. The practical implications and feasibility of the studies will be considered as secondary outcomes. Study quality will be assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). If studies are sufficiently homogenous, we will conduct a meta-analysis of the predictive value of passive sensing on STBs. The review process started in July 2022 with data extraction in September 2022. Results are expected in December 2022. Despite intensive research efforts, the ability to predict STBs is little better than chance. This systematic review will contribute to our understanding of the potential of passive sensing to improve STB prediction. Future research will be stimulated since gaps in the current literature will be identified and promising next steps toward clinical implementation will be outlined. OSF Registries osf-registrations-hzxua-v1; https://osf.io/hzxua. DERR1-10.2196/42146.

Prognostic scores in status epilepticus: A systematic review and meta-analysis.

The performance of prognostic scores of status epilepticus (SE) has been reported in very heterogeneous cohorts. We aimed to provide a summary of the available evidence on their respective performance. PubMed and EMBASE were searched for relevant articles. Studies were reviewed for eligibility for meta-analysis of the area under the receiver-operating characteristic curve (AUC) and for meta-analysis of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) in predicting in-hospital mortality with scores in which at least two external evaluations had been published. This study was registered with PROSPERO (international prospective register of systematic reviews) (CRD42022325766). Study quality was assessed using Prediction model Risk Of Bias ASsessment Tool (PROBAST). In the meta-analysis of AUC, 21 studies were pooled for STESS (Status Epilepticus Severity Score), five for EMSE-EAC (Epidemiology-based Mortality Score in Status Epilepticus - Etiology, Age, level of Consciousness), five for EMSE-EACE (EMSE - Etiology, Age, level of Consciousness, EEG), and two for ENDIT (Encephalitis, nonconvulsive status epilepticus, Diazepam resistance, Imaging abnormalities, Tracheal intubation). The pooled AUC of STESS, EMSE-EAC, EMSE-EACE, and ENDIT was 0.74 (95% CI: 0.71-0.78), 0.68 (95% CI 0.63-0.72), 0.77 (95% CI: 0.72-0.81), and 0.78 (95% CI: 0.70-0.87), respectively. The pooled sensitivity of STESS-3, STESS-4, EMSE-EACE-64, and ENDIT-4 was 0.83 (95% CI: 0.80-0.86), 0.60 (95% CI: 0.55-0.65), 0.76 (95% CI: 0.67-0.83), and 0.70 (95% CI: 0.55-0.82), respectively. Their pooled specificity was 0.50 (95% CI: 0.48-0.52), 0.74 (95% CI: 0.72-0.76), 0.63 (95% CI: 0.59-0.67), and 0.65 (95% CI: 0.61-0.70), respectively. Their pooled PPV was 0.27 (95% CI: 0.24-0.30), 0.35 (95% CI: 0.29-0.41), 0.33 (95% CI: 0.24-0.43), and 0.20 (95% CI: 0.13-0.27). Their pooled NPV was 0.94 (95% CI: 0.93-0.96), 0.90 (95% CI: 0.89-0.92), 0.89 (95% CI: 0.80-0.98), and 0.95 (95% CI: 0.92-0.98). Variations in performance were observed in patients' subgroups, such as critically ill patients and refractory cases. Investigated scores only have acceptable AUC, sensitivity, and specificity for predicting in-hospital mortality, with the EMSE-EAC having a lower discriminative power. STESS-3 has the highest sensitivity, and STESS-4 the highest specificity, but neither combines acceptable sensitivity and specificity. All these scores had high NPV but very low PPV. Caution should be exercised in their clinical use. Further studies are required to develop more accurate scores.

Performance of the J-CTO score versus other risk scores for predicting procedural difficulty in coronary chronic total occlusion interventions

Background Although the Japanese chronic total occlusion (J-CTO) score is widely used to assess the complexity of revascularization for CTO lesions, ambiguous and conflicting results are reported in validation studies. Therefore, we aimed to quantitatively evaluate the effectiveness of the J-CTO score and explore the heterogeneity of its comparison with other CTO scores. Methods PubMed, Embase, the Cochrane Library, and ClinicalTrials.gov databases were systematically searched from January 1st, 2011 to December 23rd, 2021. Studies that examined the accuracy of the J-CTO score were eligible. Where feasible, estimates of discrimination and calibration were pooled with a random-effects model. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used for risk-of-bias assessment. This study was reported according to PRISMA guidelines and prospectively registered with PROSPERO (CRD42019126161). Results Of 28 included studies (N = 34,944 lesions), 24 were eligible for meta-analysis. The J-CTO score demonstrated significant discrimination for 30-min wire crossing (summary C-statistic 0.76; 95% CI 0.68–0.84) and technical success (0.68; 95% CI 0.61–0.74) despite significant heterogeneity. Only 19 (33%) of the 58 pairwise comparisons with 14 competing scores that were based on discrimination reported a statistical result. The J-CTO score performed worse (relative difference of C-statistics >5%) in eight out of 33 independent comparisons but better in another 13. Methodological shortcomings resulted from only one study evaluating model calibration appropriately. Conclusion The discrimination power of the J-CTO score was useful for time-efficient wire crossing and moderate for angiographic success. Head-to-head comparisons of CTO scores would benefit from standardized reporting and appropriate statistical methods. Key messages The J-CTO score has useful discrimination in predicting 30-min wire crossing while performing moderately for technical success. After excluding optimism bias, there is insufficient independent evidence supporting the superiority of newly introduced models over the J-CTO score. Standardized methodology and assessment are needed to achieve a better understanding of CTO scores, especially for their calibration.

Can quantitative peritumoral CT radiomics features predict the prognosis of patients with non-small cell lung cancer? A systematic review

ObjectivesTo provide an overarching evaluation of the value of peritumoral CT radiomics features for predicting the prognosis of non-small cell lung cancer and to assess the quality of the available studies.MethodsThe PubMed, Embase, Web of Science, and Cochrane Library databases were searched for studies predicting the prognosis in patients with non-small cell lung cancer (NSCLC) using CT-based peritumoral radiomics features. Information about the patient, CT-scanner, and radiomics analyses were all extracted for the included studies. Study quality was assessed using the Radiomics Quality Score (RQS) and the Prediction Model Risk of Bias Assessment Tool (PROBAST).ResultsThirteen studies were included with 2942 patients from 2017 to 2022. Only one study was prospective, and the others were all retrospectively designed. Manual segmentation and multicenter studies were performed by 69% and 46% of the included studies, respectively. 3D-Slicer and MATLAB software were most commonly used for the segmentation of lesions and extraction of features. The peritumoral region was most frequently defined as dilated from the tumor boundary of 15 mm, 20 mm, or 30 mm. The median RQS of the studies was 13 (range 4–19), while all of included studies were assessed as having a high risk of bias (ROB) overall.ConclusionsPeritumoral radiomics features based on CT images showed promise in predicting the prognosis of NSCLC, although well-designed studies and further biological validation are still needed.Key Points• Peritumoral radiomics features based on CT images are promising and encouraging for predicting the prognosis of non-small cell lung cancer.• The peritumoral region was often dilated from the tumor boundary of 15 mm or 20 mm because these were considered safe margins.• The median Radiomics Quality Score of the included studies was 13 (range 4–19), and all of studies were considered to have a high risk of bias overall.

Prognostic value of patient-reported outcome measures (PROMs) in adults with non-small cell Lung Cancer: a scoping review

BackgroundThere is growing interest in the collection and use of patient-reported outcome measures (PROMs) to support clinical decision making in patients with non-small cell lung cancer (NSCLC). However, an overview of research into the prognostic value of PROMs is currently lacking.AimTo explore to what extent, how, and how robustly the value of PROMs for prognostic prediction has been investigated in adults diagnosed with NSCLC.MethodsWe systematically searched Medline, Embase, CINAHL Plus and Scopus for English-language articles published from 2011 to 2021 that report prognostic factor study, prognostic model development or validation study. Example data charting forms from the Cochrane Prognosis Methods Group guided our data charting on study characteristics, PROMs as predictors, predicted outcomes, and statistical methods. Two reviewers independently charted the data and critically appraised studies using the QUality In Prognosis Studies (QUIPS) tool for prognostic factor studies, and the risk of bias assessment section of the Prediction model Risk Of Bias ASsessment Tool (PROBAST) for prognostic model studies.ResultsOur search yielded 2,769 unique titles of which we included 31 studies, reporting the results of 33 unique analyses and models. Out of the 17 PROMs used for prediction, the EORTC QLQ-C30 was most frequently used (16/33); 12/33 analyses used PROM subdomain scores instead of the overall scores. PROMs data was mostly collected at baseline (24/33) and predominantly used to predict survival (32/33) but seldom other clinical outcomes (1/33). Almost all prognostic factor studies (26/27) had moderate to high risk of bias and all four prognostic model development studies had high risk of bias.ConclusionThere is an emerging body of research into the value of PROMs as a prognostic factor for survival in people with NSCLC but the methodological quality of this research is poor with significant bias. This warrants more robust studies into the prognostic value of PROMs, in particular for predicting outcomes other than survival. This will enable further development of PROM-based prediction models to support clinical decision making in NSCLC.

Prediction Model Risk-of-Bias Assessment Tool for coronary artery lesions in Kawasaki disease

ObjectiveTo review and critically appraise articles on prediction models for coronary artery lesions (CALs) in Kawasaki disease included in PubMed, Embase, and Web of Science databases from January 1, 1980, to December 23, 2021.Materials and methodsStudy screening, data extraction, and quality assessment were performed by two independent reviewers, with a statistics expert resolving discrepancies. Articles that developed or validated a prediction model for CALs in Kawasaki disease were included. The Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies checklist was used to extract data from different articles, and Prediction Model Risk-of-Bias Assessment Tool (PROBAST) was used to assess the bias risk in different prediction models. We screened 19 studies from a pool of 881 articles.ResultsThe studies included 73–5,151 patients. In most studies, univariable logistic regression was used to develop prediction models. In two studies, external data were used to validate the developing model. The most commonly included predictors were C-reactive protein (CRP) level, male sex, and fever duration. All studies had a high bias risk, mostly because of small sample size, improper handling of missing data, and inappropriate descriptions of model performance and the evaluation model.ConclusionThe prediction models were suitable for the subjects included in the studies, but were poorly effective in other populations. The phenomenon may partly be due to the bias risk in prediction models. Future models should address these problems and PROBAST should be used to guide study design.

Clinical First-Trimester Prediction Models for Gestational Diabetes Mellitus: A Systematic Review and Meta-Analysis.

Gestational diabetes mellitus (GDM) is a common pregnancy complication that negatively impacts the health of both the mother and child. Early prediction of the risk of GDM may permit prompt and effective interventions. This systematic review and meta-analysis aimed to summarize the study characteristics, methodological quality, and model performance of first-trimester prediction model studies for GDM. Five electronic databases, one clinical trial register, and gray literature were searched from the inception date to March 19, 2022. Studies developing or validating a first-trimester prediction model for GDM were included. Two reviewers independently extracted data according to an established checklist and assessed the risk of bias by the Prediction Model Risk of Bias Assessment Tool (PROBAST). We used a random-effects model to perform a quantitative meta-analysis of the predictive power of models that were externally validated at least three times. We identified 43 model development studies, six model development and external validation studies, and five external validation-only studies. Body mass index, maternal age, and fasting plasma glucose were the most commonly included predictors across all models. Multiple estimates of performance measures were available for eight of the models. Summary estimates range from 0.68 to 0.78 (I2 ranged from 0% to 97%). Most studies were assessed as having a high overall risk of bias. Only eight prediction models for GDM have been externally validated at least three times. Future research needs to focus on updating and externally validating existing models.

Risk assessment models for venous thromboembolism in pregnancy and in the puerperium: a systematic review

ObjectivesTo assess the comparative accuracy of risk assessment models (RAMs) to identify women during pregnancy and the early postnatal period who are at increased risk of venous thromboembolism (VTE).DesignSystematic review...

Cardiovascular disease risk prediction models in the Chinese population- a systematic review and meta-analysis

BackgroundThere is an increasing prevalence of cardiovascular disease (CVD) in China, which represents the leading cause of mortality. Precise CVD risk identification is the fundamental prevention component. This study sought to systematically review the CVD risk prediction models derived and/or validated in the Chinese population to promote primary CVD prevention.MethodsReports were included if they derived or validated one or more CVD risk prediction models in the Chinese population. PubMed, Embase, CINAHL, Web of Science, Scopus, China National Knowledge Infrastructure (CNKI), VIP database, etc., were searched. The risk of bias was assessed with the Prediction Model Risk of Bias Assessment Tool (PROBAST). Meta-analysis was performed in R using the package metamisc.ResultsFrom 55,183 records, 22 studies were included. Twelve studies derived 18 CVD risk prediction models, of which seven models were derived based on a multicentre cohort including more than two provinces of mainland China, and one was a model developed based on a New Zealand cohort including Chinese individuals. The number of predictors ranged from 6 to 22. The definitions of predicted outcomes showed considerable heterogeneity. Fourteen articles described 29 validations of 8 models. The Framingham model and pooled cohort equations (PCEs) are the most frequently validated foreign tools. Discrimination was acceptable and similar for men and women among models (0.60–0.83). The calibration estimates changed substantially from one population to another. Prediction for atherosclerotic cardiovascular disease Risk in China (China-PAR) showed good calibration [observed/expected events ratio = 0.99, 95% PI (0.57,1.70)] and female sex [1.10, 95% PI (0.23,5.16)].ConclusionsSeveral models have been developed or validated in the Chinese population. The usefulness of most of the models remains unclear due to incomplete external validation and head-to-head comparison. Future research should focus on externally validating or tailoring these models to local settings.Trail registrationThis systematic review was registered at PROSPERO (International Prospective Register of Systematic Reviews, CRD42021277453).

Machine learning models for predicting acute kidney injury: a systematic review and critical appraisal.

The number of studies applying machine learning (ML) to predict acute kidney injury (AKI) has grown steadily over the past decade. We assess and critically appraise the state of the art in ML models for AKI prediction, considering performance, methodological soundness, and applicability. We searched PubMed and ArXiv, extracted data, and critically appraised studies based on the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD), Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), and Prediction Model Risk of Bias Assessment Tool (PROBAST) guidelines. Forty-six studies from 3166 titles were included. Thirty-eight studies developed a model, five developed and externally validated one, and three studies externally validated one. Flexible ML methods were used more often than deep learning, although the latter was common with temporal variables and text as predictors. Predictive performance showed an area under receiver operating curves ranging from 0.49 to 0.99. Our critical appraisal identified a high risk of bias in 39 studies. Some studies lacked internal validation, whereas external validation and interpretability of results were rarely considered. Fifteen studies focused on AKI prediction in the intensive care setting, and the US-derived Medical Information Mart for Intensive Care (MIMIC) data set was commonly used. Reproducibility was limited as data and code were usually unavailable. Flexible ML methods are popular for the prediction of AKI, although more complex models based on deep learning are emerging. Our critical appraisal identified a high risk of bias in most models: Studies should use calibration measures and external validation more often, improve model interpretability, and share data and code to improve reproducibility.

Risk Prediction Models of Natural Menopause Onset: A Systematic Review.

Predicting the onset of menopause is important for family planning and to ensure prompt intervention in women at risk of developing menopause-related diseases. We aimed to summarize risk prediction models of natural menopause onset and their performance. Five bibliographic databases were searched up to March 2022. We included prospective studies on perimenopausal women or women in menopausal transition that reported either a univariable or multivariable model for risk prediction of natural menopause onset. Two authors independently extracted data according to the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist. Risk of bias was assessed using a prediction model risk of bias assessment tool (PROBAST). Of 8132 references identified, we included 14 articles based on 8 unique studies comprising 9588 women (mainly Caucasian) and 3289 natural menopause events. All included studies used onset of natural menopause (ONM) as outcome, while 4 studies also predicted early ONM. Overall, there were 180 risk prediction models investigated, with age, anti-Müllerian hormone, and follicle-stimulating hormone being the most investigated predictors. Estimated C-statistic for the prediction models ranged from 0.62 to 0.95. Although all studies were rated at high risk of bias mainly due to the methodological concerns related to the statistical analysis, their applicability was satisfactory. Predictive performance and generalizability of current prediction models on ONM is limited given that these models were generated from studies at high risk of bias and from specific populations/ethnicities. Although in certain settings such models may be useful, efforts to improve their performance are needed as use becomes more widespread.

Comprehensive comparison of stroke risk score performance: a systematic review and meta-analysis among 6 267 728 patients with atrial fibrillation.

Multiple risk scores to predict ischaemic stroke (IS) in patients with atrial fibrillation (AF) have been developed. This study aims to systematically review these scores, their validations and updates, assess their methodological quality, and calculate pooled estimates of the predictive performance. We searched PubMed and Web of Science for studies developing, validating, or updating risk scores for IS in AF patients. Methodological quality was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). To assess discrimination, pooled c-statistics were calculated using random-effects meta-analysis. We identified 19 scores, which were validated and updated once or more in 70 and 40 studies, respectively, including 329 validations and 76 updates-nearly all on the CHA2DS2-VASc and CHADS2. Pooled c-statistics were calculated among 6 267 728 patients and 359 373 events of IS. For the CHA2DS2-VASc and CHADS2, pooled c-statistics were 0.644 [95% confidence interval (CI) 0.635-0.653] and 0.658 (0.644-0.672), respectively. Better discriminatory abilities were found in the newer risk scores, with the modified-CHADS2 demonstrating the best discrimination [c-statistic 0.715 (0.674-0.754)]. Updates were found for the CHA2DS2-VASc and CHADS2 only, showing improved discrimination. Calibration was reasonable but available for only 17 studies. The PROBAST indicated a risk of methodological bias in all studies. Nineteen risk scores and 76 updates are available to predict IS in patients with AF. The guideline-endorsed CHA2DS2-VASc shows inferior discriminative abilities compared with newer scores. Additional external validations and data on calibration are required before considering the newer scores in clinical practice. ID CRD4202161247 (PROSPERO).

Prediction Models for Prognosis of Hypoglycemia in Patients with Diabetes: A Systematic Review and Meta-Analysis.

To systematically summarize the reported prediction models for hypoglycemia in patients with diabetes, compare their performance, and evaluate their applicability in clinical practice. We selected studies according to the PRISMA, appraised studies according to the Prediction model Risk of Bias Assessment Tool (PROBAST), and extracted and synthesized the data according to the CHARMS. The databases of PubMed, Web of Science, Embase, and Cochrane Library were searched from inception to 31 October 2021 using a systematic review approach to capture all eligible studies developing and/or validating a prognostic prediction model for hypoglycemia in patients with diabetes. The risk bias and clinical applicability were assessed using the PROBAST. The meta-analysis of the performance of the prediction models were also conducted. The protocol of this study was recorded in PROSPERO (CRD42022309852). Sixteen studies with 22 models met the eligible criteria. The predictors with the high frequency of occurrence among all models were age, HbA1c, history of hypoglycemia, and insulin use. A meta-analysis of C-statistic was performed for 21 prediction models, and the summary C-statistic and its 95% confidence interval and prediction interval were 0.7699 (0.7299-0.8098), 0.7699 (0.5862-0.9536), respectively. Heterogeneity exists between different hypoglycemia prediction models (τ2 was 0.00734≠0). The existing predictive models are not recommended for widespread clinical use. A high-quality hypoglycemia screening tool should be developed in future studies.

Risk of bias of prognostic models developed using machine learning: a systematic review in oncology

BackgroundPrognostic models are used widely in the oncology domain to guide medical decision-making. Little is known about the risk of bias of prognostic models developed using machine learning and the barriers to their clinical uptake in the oncology domain.MethodsWe conducted a systematic review and searched MEDLINE and EMBASE databases for oncology-related studies developing a prognostic model using machine learning methods published between 01/01/2019 and 05/09/2019. The primary outcome was risk of bias, judged using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We described risk of bias overall and for each domain, by development and validation analyses separately.ResultsWe included 62 publications (48 development-only; 14 development with validation). 152 models were developed across all publications and 37 models were validated. 84% (95% CI: 77 to 89) of developed models and 51% (95% CI: 35 to 67) of validated models were at overall high risk of bias. Bias introduced in the analysis was the largest contributor to the overall risk of bias judgement for model development and validation. 123 (81%, 95% CI: 73.8 to 86.4) developed models and 19 (51%, 95% CI: 35.1 to 67.3) validated models were at high risk of bias due to their analysis, mostly due to shortcomings in the analysis including insufficient sample size and split-sample internal validation.ConclusionsThe quality of machine learning based prognostic models in the oncology domain is poor and most models have a high risk of bias, contraindicating their use in clinical practice. Adherence to better standards is urgently needed, with a focus on sample size estimation and analysis methods, to improve the quality of these models.