Threshold Of Specificity Research Articles

P-233 Deep ensembles-based AI as a tool to support embryo grading and clinical pregnancy prediction

Abstract Study question Can deep learning accurately evaluate embryo grades and predict clinical pregnancy while providing relevant clinical evidence, not just results from a black box? Summary answer The sophisticated ensemble method can improve the predictive performance for embryo grades and clinical pregnancy, while providing clinically relevant evidence. What is known already Previous studies have shown that AI can predict the IVF outcomes by analyzing the images of embryos. In many literature, AI outperformed human because AI could identify features human eyes could not easily detect. However, clinicians have been cautious to adopt the AI technology due to the black box nature of AI algorithms. In this study, we increased the predictive power of AI as well as providing evidence of the prediction by using deep ensembles and Grad-CAM images. Study design, size, duration We performed a retrospective study of single static images of 727 Day 5 blastocysts from 270 patients who underwent single embryo transfer at a single in vitro fertilization (IVF) clinic between January 2015 and March 2021. The images were collected from standard optical light microscopes and matched with metadata such as embryo grades and pregnancy outcomes. Participants/materials, setting, methods Two different models were designed: an automatic embryo grading model and a pregnancy prediction model. Embryologists labeled a day 5 embryo “GEM,” a good embryo if 4AA/AB or above in the Gardner system and pregnancy was defined as the presence of a fetal heartbeat (FHB). Deep ensembles were applied by training four convolutional neural networks (CNNs) and Grad-CAM images were extracted from the last layer and reviewed by experts. Main results and the role of chance Under several single CNNs, the highest AUROCs of the embryo grading model and the pregnancy prediction model were 0.80 and 0.67, respectively. After applying deep ensembles, the AUROCs of the two models increased to 0.84 and 0.72, respectively. When the F1-score for the positive cases were maximized by adjusting the threshold of ensembles, accuracy, sensitivity and specificity of the embryo grading model were 88.1%, 92.9% and 62.5% respectively. For the pregnancy prediction model, accuracy, sensitivity and specificity were 66.3%, 77.1% and 55.6% respectively. The accuracy of GEM predicting pregnancy for the embryologists and the embryo grading AI model was 47.3% and 59.2%, respectively. It is noteworthy that the AI pregnancy prediction model outperformed the embryologists while successfully auto-grading embryos, a strong evidence that AI considered more features for prediction than what was used for grading. It was also noted from the review of the Grad-CAM images that the both AI models were focusing on the ICM, TE and hatching. Although their area of focus was the same, the pregnancy prediction model was able to make better predictions than the embryologists and the embryo grading model. Limitations, reasons for caution This study has limitations as it is a retrospective study performed on embryo images from a single IVF center. In addition, including other variables such as clinical data may enhance the models. Wider implications of the findings We showed that deep learning can automatically grade embryos and more accurately predict pregnancy than embryologists. Furthermore, the embryologists confirmed the model was looking at key features like ICM, TE and hatching. Sharing such evidence with clinicians can be a necessary step for AI to be adopted for clinical practice. Trial registration number not applicable

ExhauFS: exhaustive search-based feature selection for classification and survival regression.

Feature selection is one of the main techniques used to prevent overfitting in machine learning applications. The most straightforward approach for feature selection is an exhaustive search: one can go over all possible feature combinations and pick up the model with the highest accuracy. This method together with its optimizations were actively used in biomedical research, however, publicly available implementation is missing. We present ExhauFS—the user-friendly command-line implementation of the exhaustive search approach for classification and survival regression. Aside from tool description, we included three application examples in the manuscript to comprehensively review the implemented functionality. First, we executed ExhauFS on a toy cervical cancer dataset to illustrate basic concepts. Then, multi-cohort microarray breast cancer datasets were used to construct gene signatures for 5-year recurrence classification. The vast majority of signatures constructed by ExhauFS passed 0.65 threshold of sensitivity and specificity on all datasets, including the validation one. Moreover, a number of gene signatures demonstrated reliable performance on independent RNA-seq dataset without any coefficient re-tuning, i.e., turned out to be cross-platform. Finally, Cox survival regression models were used to fit isomiR signatures for overall survival prediction for patients with colorectal cancer. Similarly to the previous example, the major part of models passed the pre-defined concordance index threshold 0.65 on all datasets. In both real-world scenarios (breast and colorectal cancer datasets), ExhauFS was benchmarked against state-of-the-art feature selection models, including L1-regularized sparse models. In case of breast cancer, we were unable to construct reliable cross-platform classifiers using alternative feature selection approaches. In case of colorectal cancer not a single model passed the same 0.65 threshold. Source codes and documentation of ExhauFS are available on GitHub: https://github.com/s-a-nersisyan/ExhauFS.

Translated article] Triggered electromyography (tEMG) in the lumbar spine it's reliable? Correlation of tEMG with postoperative CT images in lumbar instrumentation

IntroductionInstrumentation with pedicle screws (PS) can compromise the adjacent neural structures. Triggered electromyography (tEMG) is a modality of intraoperative neuromonitoring, used to assist in the placement of these. The objective of the present study is to evaluate the reliability of this tool. MethodRetrospective review of patients underwent posterior lumbar fusions from January 2017 to December 2019, correlating postoperative CT images (postop CT) with tEMG results for each PS with a threshold of 10mA, establishing sensitivity and specificity of this tool. Diagnostic test and receiver operating characteristic curve were performed to evaluate the area under the curve. ResultA total of 275 PS were evaluated between L1 to S1; 5 PS showed concordance for an inadequate trajectory, while 10 PS, with tEMG<10mA, were correctly positioned. The postoperative CT identified 17 pedicle gaps not noticed by the tEMG. Sensitivity and specificity were 23% (95% CI: 8–45) and 96% (95% CI: 93–98), respectively, with a PPV of 33.3% and a NPV of 93.6%. The area under the curve was 0.74 (95% CI: 0.62–0.86) with a cut-off point of 24mA, showing a sensitivity of 77% (95% CI: 0.55–0.92) and specificity of 69% (95% CI: 0.63–0.75). ConclusionGiven its low sensitivity, we do not recommend tEMG as the sole test in the verification of PS. We recommend using a cut-off point ≤8mA as it demonstrated the highest levels of sensitivity and specificity. We do not recommend using high thresholds to increase sensitivity.

Electromiografía evocada (tEMG) en la columna lumbar. ¿Es confiable? Correlación de tEMG con imágenes de TC posoperatorias en la instrumentación lumbar

IntroductionInstrumentation with pedicle screws (PS) can compromise the adjacent neural structures. Triggered electromyography (tEMG) is a modality of intraoperative neuromonitoring, used to assist in the placement of these. The objective of the present study is to evaluate the reliability of this tool. MethodRetrospective review of patients underwent posterior lumbar fusions from January 2017 to December 2019, correlating postoperative CT images (postop CT) with tEMG results for each PS with a threshold of 10mA, establishing sensitivity and specificity of this tool. Diagnostic test and receiver operating characteristic curve were performed to evaluate the area under the curve. ResultA total of 275 PS were evaluated between L1 to S1; 5 PS showed concordance for an inadequate trajectory, while 10PS, with tEMG <10mA, were correctly positioned. The postoperative CT identified 17 pedicle gaps not noticed by the tEMG. Sensitivity and specificity were 23% (95% CI: 8-45) and 96% (95% CI: 93-98), respectively, with a PPV of 33.3% and a NPV of 93.6%. The area under the curve was 0.74 (95% CI: 0.62-0.86) with a cut-off point of 24mA, showing a sensitivity of 77% (95% CI: 0.55-0.92) and specificity of 69% (95% CI: 0.63-0.75). ConclusionGiven its low sensitivity, we do not recommend tEMG as the sole test in the verification of PS. We recommend using a cut-off point ≤8mA as it demonstrated the highest levels of sensitivity and specificity. We do not recommend using high thresholds to increase sensitivity.

The clinical value of screening for diabetic retinopathy with a new mydriasis-free, full-feld ficker electrophysiological recording device

Objective To observe the preliminary clinical application value of the handheld non-mydriatic visual electrophysiological diagnostic system RETeval in screening for diabetic retinopathy (DR). Methods Retrospective clinical study. Fifty-eight patients with type 2 diabetes mellitus and 16 normal subjects who were admitted to Wuhan General Hospital of the PLA from November 2017 to May 2018 were enrolled in this study. All patients had not received any ophthalmologic treatment. All patients were examined by the default assessment protocol model of the RETeval device, and the score were measured by the system. The FFA results were used as the gold standard, and the DR was graded according to the international DR grading standard established in 2002. Patients were divided into vision threatening DR (VTDR) positive group and VTDR (−) group, DR (+) group and DR (−) group. Two independent sample t tests was used to compare the implicit time, amplitude, and pupil area ratio between eyes of different groups. Spearman correlation analysis was used to analyze the relationship between score and DR severity. The receiver operating characteristic area under the curve (AUC) assesses the sensitivity and specificity of RETeval in detecting DR and VTDR. The threshold of sensitivity and specificity was determined by using the maximum Youden index as a standard. Results The AUC of DR was 0.936, the sensitivity was 81%, the specificity was 92%; the AUC of VTDR was 0.976, the sensitivity was 96% and the specificity was 70%. Compared with DR (−) group, the implicit time of DR (+) group was delayed and the amplitude and pupil area were decreased (t=-13.43, 5.49, 6.09; P=0.000, 0.000, 0.000). Compared with VTDR (−) group, the implicit time of VTDR (+) group was delayed and the amplitude and pupil area were decreased (t=-11.05, 7.46, 5.73; P=0.000, 0.000, 0.000). The score was significantly correlated with the severity of DR (r=0.89, P<0.05). Conclusions The score measured by the RETeval instrument has a high degree of specificity and sensitivity in the diagnosis of DR and VTDR. It is highly correlated with the severity of DR. Key words: Diabetic retinopathy/diagnosis; Electroretinography

Machine learning classifiers can predict Gleason pattern 4 prostate cancer with greater accuracy than experienced radiologists

ObjectiveThe purpose of this study was: To test whether machine learning classifiers for transition zone (TZ) and peripheral zone (PZ) can correctly classify prostate tumors into those with/without a Gleason 4 component, and to compare the performance of the best performing classifiers against the opinion of three board-certified radiologists.MethodsA retrospective analysis of prospectively acquired data was performed at a single center between 2012 and 2015. Inclusion criteria were (i) 3-T mp-MRI compliant with international guidelines, (ii) Likert ≥ 3/5 lesion, (iii) transperineal template ± targeted index lesion biopsy confirming cancer ≥ Gleason 3 + 3. Index lesions from 164 men were analyzed (119 PZ, 45 TZ). Quantitative MRI and clinical features were used and zone-specific machine learning classifiers were constructed. Models were validated using a fivefold cross-validation and a temporally separated patient cohort. Classifier performance was compared against the opinion of three board-certified radiologists.ResultsThe best PZ classifier trained with prostate-specific antigen density, apparent diffusion coefficient (ADC), and maximum enhancement (ME) on DCE-MRI obtained a ROC area under the curve (AUC) of 0.83 following fivefold cross-validation. Diagnostic sensitivity at 50% threshold of specificity was higher for the best PZ model (0.93) when compared with the mean sensitivity of the three radiologists (0.72). The best TZ model used ADC and ME to obtain an AUC of 0.75 following fivefold cross-validation. This achieved higher diagnostic sensitivity at 50% threshold of specificity (0.88) than the mean sensitivity of the three radiologists (0.82).ConclusionsMachine learning classifiers predict Gleason pattern 4 in prostate tumors better than radiologists.Key Points• Predictive models developed from quantitative multiparametric magnetic resonance imaging regarding the characterization of prostate cancer grade should be zone-specific.• Classifiers trained differently for peripheral and transition zone can predict a Gleason 4 component with a higher performance than the subjective opinion of experienced radiologists.• Classifiers would be particularly useful in the context of active surveillance, whereby decisions regarding whether to biopsy are necessitated.

Correlation between serum CA724 and gastric cancer: multiple analyses based on Chinese population

Serum tumor biomarker carbohydrate antigen 724 (CA724) is noticeable for gastric cancer. Correlation between CA724 and gastric cancer was investigated based on Chinese population. Chinese Biomedical Database, Chinese Journal Full-text Database and PubMed were searched. Gastric cancer patients were proven by biopsy, and control included health volunteers or benign gastric diseases. Participants received at least one test of CA724, CA125, CA153, CA199, CA242 or CEA. Meta-analysis, summary ROC (SROC) and post hoc analysis were performed by RevMan 5.0 and SPSS 11.5. Totally, 33 eligible studies were analyzed. Meta-analysis showed CA724 had the highest odds ratio 32.86 compared to control, orderly followed by CA242, CA199, CEA, CA125 and CA153. Accumulated accuracy rate of CA724 was 77 %, superior to others. In SROC analysis, specificity of all studies was above 0.70, but sensitivity of few studies was above 0.70; CA724 was selected as the preferable single test, followed by CA242, CA199, CEA, CA125 and CA153. If threshold of both specificity and sensitivity up to 0.70, CA153 was unacceptable; if up to 0.80, only CA724 and CA242 were considerable. In CA724-combined patterns, CA724+CEA+CA199 combination performed best by increasing sensitivity to 0.74 without impairing specificity, while CA724 + CA199 pattern was not a proper combination. CA724 was the most correlative serum tumor biomarker for gastric cancer in Chinese population. Sensitivity of serum CA724 is limited, but CA724+CEA+CA199 combination is considerable to improve sensitivity without impairing specificity.

Differentiation of axillary inflammatory and metastatic lymph nodes with diffusion-weighted imaging in animal modes

To evaluate the value of diffusion-weighted imaging (DWI) in the differentiation of axillary inflammatory hyperplastic and metastatic lymph nodes. Forty female New Zealand white rabbits were divided randomly into 2 groups (n = 20 each). And the animal models of axillary inflammatory hyperplastic and metastatic lymph nodes were established. All successfully implanted models received conventional magnetic resonance imaging (MRI) and diffusion-weighted imaging (DWI) examinations. The features of signal intensity and shapes of lymph nodes were observed in two groups. The sizes of lymph nodes were measured on the selected axial T(2)WI. The signal intensity and appearance diffusion coefficient (ADC) value of lymph nodes and dorsal muscle at the same slice were measured on the selected T(2)WI, DWI and ADC map respectively. The relative signal intensity (rSIT(2)WI, rSIDWI) and relative ADC (rADC) value of lymph nodes to dorsal muscle were calculated and compared. The diagnostic efficacy of differentiating benign and malignant lymph nodes was analyzed with rADC value through the receiver operating characteristic curve. The correlation between rADC value and cell density was assessed with pathological findings as reference standard. The differences of size, rSIT(2)WI and rSIDWI were not statistically significant between two groups. The rADC value of inflammatory lymph nodes was greater than that of metastatic lymph nodes (0.91 ± 0.14 vs 0.64 ± 0.18). Significant difference existed (t = 3.879, P = 0.03). But there was a little overlap between two groups. With 0.78 as the diagnostic threshold of rADC value, the sensibility and specificity was 86.2% and 74.4% respectively. The correlation between rADC value and cell density of inflammatory hyperplastic and metastatic lymph nodes was significantly inverse (r = -0.53, P = 0.003). As compared with the routine MRI sequence, rADC value has a higher diagnostic efficacy in the differentiation of benign and malignant lymph nodes. Cell density may be the most important influencing factor for the differences of rADC value between two groups of lymph nodes.

Nationwide Acute Care Physical Therapist Practice Analysis Identifies Knowledge, Skills, and Behaviors That Reflect Acute Care Practice

Acute care physical therapy is a rapidly evolving practice area, but little is known about the skills, knowledge, and behaviors necessary for a clinician to be most effective in this area. The objective of this study was to perform the first nationwide survey of acute care physical therapists to validate the knowledge, skill, and behavior sets that reflect practice parameters specific for acute care physical therapy. A survey format was used. The Acute Care Physical Therapy Practice Analysis Survey was created on the basis of current literature and consensus of a 9-member Subject Matter Expert (SME) Group. The survey sections addressed knowledge areas, professional behaviors, and patient/client management approaches reflecting practice parameters specific for acute care physical therapy. Additionally, respondent demographic information was collected. Pilot testing necessitated minor changes in the survey. After revision, the survey questionnaire was sent to a sample of experienced acute care physical therapists throughout the United States. A convenience sample of 522 physical therapists who identified themselves as having extensive experience in acute care practice was used. Of these, 254 completed the survey, for a response rate of 48.7%. Through the use of predetermined decision rules, 34 items were eliminated on the basis of a descriptive analysis of survey results as well as a failure of the items to meet the threshold of specificity for acute care practice, as determined through SME Group consensus. The potential for self-selection bias, a sample weighted heavily with American Physical Therapy Association members and Acute Care Section members, and a small proportion of clinicians (<3%) with less than 1 year of acute care experience may limit generalizability of these results. The results of this practice analysis describe distinct knowledge, skills, and behaviors specific for acute care physical therapy. The outcomes of the survey might assist in the development of professional (entry-level) acute care competencies, a petition for the recognition of specialization in acute care physical therapy, or both. In addition, the findings of this practice analysis could serve as the foundation for the development of residencies or fellowships in acute care practice.

Identification of advanced coronary artery disease with exercise myocardial perfusion imaging: the clinical value of a novel approach for assessing lung thallium-201 uptake

The precise clinical utility of lung 201Tl uptake in exercise SPECT myocardial perfusion imaging remains open to research. This study validates an optimal index for lung 201Tl uptake measurement and assesses its value in the prediction of higher-risk coronary artery disease (CAD). Three hundred and ninety-eight patients underwent exercise SPECT myocardial perfusion imaging. They were separated into derivation (n = 217) and validation (n = 186) groups, both including sub-populations of lower- and higher-risk CAD, according to coronary angiography. Another 56 individuals with a low probability of CAD comprised the control group. From a planar, anterior, post-exercise acquisition, the lung (L) to heart (H) maximal (L/H(max)), total (L/H(mean)) and background-subtracted total (L/H(net)) ratios were calculated. These were also adjusted for confounding variables, as assessed from the control group. ROC analysis was used to compare the L/H ratios and define thresholds of abnormality. The performance of the optimal index was assessed in the derivation group and was then tested in the validation population. Subsequently, it was compared with other scintigraphic, exercise electrocardiography and clinical variables. In the derivation group L/H(net) was a better discriminator for higher-risk CAD than both L/H(max) and L/H(mean). Similarly, the adjusted L/H(net) was a better discriminator than both the adjusted L/H(max) and the adjusted L/H(mean). No significant difference was attained between L/H(net) and the adjusted L/H(net). At the upper defined threshold of abnormality, sensitivity and specificity of L/H(net) in the detection of higher-risk CAD in the derivation and the validation cohorts were 52% and 92% versus 47% and 94%, respectively (p = ns). The results were similar at other defined thresholds. Moreover, L/H(net) was found to be a significant predictor of higher-risk CAD, superior to myocardial perfusion images, transient ventricular dilation measurements, and clinical and exercise testing variables (ROC analysis and logistic regression). By raising the threshold of abnormality of L/H(net), specificity and positive predictive value increased, whereas sensitivity and negative predictive value decreased. Lung 201Tl assessment assists substantially in the identification of higher-risk CAD in exercise SPECT myocardial perfusion imaging and this is best achieved by L/H(net). This index is a significant predictor of higher-risk CAD, superior to myocardial perfusion images, and its value is associated with the probability of a disease state.

A comparison of the rates of clot lysis in a plasma milieu induced by tissue plasminogen activator (t-PA) and rec-pro-urokinase: Evidence that t-PA has a more restricted mode of action

Tissue plasminogen activator (t-PA) and pro-urokinase (pro-UK) were previously found to have distinct and complementary mechanisms of action in a plasma milieu. The t-PA activated primarily plasminogen bound to intact fibrin (internal lysine-bound), whereas pro-UK activated plasminogen bound to partially degraded fibrin (C-terminal lysine-bound). These observations therefore suggested that each of these activators was restricted in its fibrin-dependent lytic effect. At least this is true at activator concentrations where activation of plasminogen is strictly fibrin dependent; at some high concentration all plasminogen is activated including free plasminogen in the ambient plasma. However, since some pro-UK is converted to UK at the plasmin rich fibrin surface during the course of lysis and since UK is a non-selective plasminogen activator, the pro-UK/UK system should not be similarly restricted. The present study was therefore designed to examine whether this is the case by determining the highest rates of lysis achievable at the threshold of specificity for t-PA and for pro-UK/UK respectively. Maximal rates of clot lysis by t-PA (0.1–5 μg/ml) and by rec-pro-UK (0.5–5 μg/ml) were determined and correlated with the degree of preservation of fibrinogen. The highest rate of clot lysis induced by a fibrin-specific (<10% fibrinogen degradation) dose of rec-pro-UK was 80%/h. A similar rate could not be induced by t-PA without causing >80% fibrinogen degradation. The highest fibrin-specific rate of lysis achievable with t-PA was only 42%/h. When a small amount of t-PA was added at intervals during clot lysis induced by rec-pro-UK, the t-PA significantly foreshortened the lag phase, but it did not enhance the maximal rate of clot lysis. By contrast, when rec-pro-UK was added during the course of clot lysis induced by a large, but fibrin-specific amount of t-PA, rec-pro-UK accelerated the rate of lysis thereby demonstrating the presence of some fibrin-bound plasminogen not activated by t-PA. The study indicates that in a plasma milieu t-PA is more restricted in its action and activates only about half the fibrin-bound plasminogen which is activated by rec-pro-UK/UK.