Over-all Agreement Research Articles

A multicenter study to evaluate the analytical precision by pathologists using the Aperio GT 450 DX

BackgroundDigital pathology systems (DPS) are emerging as capable technologies for clinical practice. Studies have analyzed pathologists' diagnostic concordance by comparing reviews of whole slide images (WSIs) to glass slides (e.g., accuracy). This observational study evaluated the reproducibility of pathologists' diagnostic reviews using the Aperio GT 450 DX under slightly different conditions (precision). MethodDiagnostic precision was tested in three conditions: intra-system (within systems), inter-system/site (between systems/sites), and intra- and inter-pathologist (within and between pathologists). A total of five study/reading pathologists (one pathologist each for intra-system, inter-system/site, and three for intra-pathologist/inter-pathologist analyses) were assigned to the respective sub-studies.A panel of 69 glass slides with 23 unique histological features was used to evaluate the WSI system's precision. Each glass slide was scanned to generate a unique WSI. From each WSI, the field of view (FOV) was generated (at least 2 FOVs/WSI), which included the selected features (1–3 features/FOV). Each pathologist reviewed the digital slides and identified which morphological features, if any, were present in each defined FOV. To minimize recall bias, an additional 12 wild card slides from different organ types were used for which FOVs were extracted. The pathologists also read these wild card slides FOVs; however, the corresponding feature identification was not included in the final data analysis. ResultsEach measured endpoint met the pre-defined acceptance criteria of the lower bound of the 95% confidence interval (CI) overall agreement (OA) rate being ≥85% for each sub-study. The lower bound of the 95% CI for the intra-system OA rate was 95.8%; for inter-system analysis, it was 94.9%; for intra-pathologist analysis, 92.4%, whereas for inter-pathologist analyses, the lower bound of the 95% CI of the OA was 90.6%. ConclusionThe study results indicate that pathologists using the Aperio GT 450 DX WSI system can precisely identify histological features that may be required for accurately diagnosing anatomic pathology cases.

Comparing psychometric characteristics of a computerized cognitive test (BrainCheck Assess) against the Montreal cognitive assessment.

Previous validation studies demonstrated that BrainCheck Assess (BC-Assess), a computerized cognitive test battery, can reliably and sensitively distinguish individuals with different levels of cognitive impairment (i.e., normal cognition (NC), mild cognitive impairment (MCI), and dementia). Compared with other traditional paper-based cognitive screening instruments commonly used in clinical practice, the Montreal Cognitive Assessment (MoCA) is generally accepted to be among the most comprehensive and robust screening tools, with high sensitivity/specificity in distinguishing MCI from NC and dementia. In this study, we examined: (1) the linear relationship between BC-Assess and MoCA and their equivalent cut-off scores, and (2) the extent to which they agree on their impressions of an individual's cognitive status. A subset of participants (N = 55; age range 54-94, mean/SD = 80/9.5) from two previous studies who took both the MoCA and BC-Assess were included in this analysis. Linear regression was used to calculate equivalent cut-off scores for BC-Assess based on those originally recommended for the MoCA to differentiate MCI from NC (cut-off = 26), and dementia from MCI (cut-off = 19). Impression agreement between the two instruments were measured through overall agreement (OA), positive percent agreement (PPA), and negative percent agreement (NPA). A high Pearson correlation coefficient of 0.77 (CI = 0.63-0.86) was observed between the two scores. According to this relationship, MoCA cutoffs of 26 and 19 correspond to BC-Assess scores of 89.6 and 68.5, respectively. These scores are highly consistent with the currently recommended BC-Assess cutoffs (i.e., 85 and 70). The two instruments also show a high degree of agreement in their impressions based on their recommended cut-offs: (i) OA = 70.9%, PPA = 70.4%, NPA = 71.4% for differentiating dementia from MCI/NC; (ii) OA = 83.6%, PPA = 84.1%, NPA = 81.8% for differentiating dementia/MCI from NC. This study provides further validation of BC-Assess in a sample of older adults by showing its high correlation and agreement in impression with the widely used MoCA.

Abstract 3754: High staining reproducibility and scoring precision demonstrated in endometrial cancer stained with PD-L1 IHC 22C3 pharmDx used in conjunction with combined positive score

Abstract Background: PD-L1 IHC 22C3 pharmDx (SK006) is a qualitative immunohistochemical (IHC) assay using anti-PD-L1 clone 22C3 to detect PD-L1 in formalin-fixed, paraffin-embedded (FFPE) human endometrial cancer (ENDO) tissues. PD-L1 expression is determined using the Combined Positive Score (CPS) algorithm, which is the number of PD-L1 staining cells (tumor cells, lymphocytes, macrophages) divided by the total number of viable tumor cells, multiplied by 100. Validation testing evaluated the sensitivity and analytical performance of SK006 PD-L1 expression in FFPE ENDO specimens based on two PD-L1 expression cutoffs for all applicable studies: CPS ≥ 1 and CPS ≥ 10. Analytical performance studies included precision (combined precision, intra-run and intra-inter observer) and robustness (time, temperature, pH and re-use) testing. Methods: FFPE ENDO specimens were stained with SK006 and scored using CPS. Negative percent agreement (NPA), positive percent agreement (PPA), and overall agreement (OA) with two-sided 95% bootstrap confidence intervals (CIs) were used for data analysis, based on PD-L1 binary status (positive/negative) at the evaluated cutoffs. Acceptance Criteria (AC): the lower bound of the two-sided 95% CI computed on percent agreement (NPA, PPA, and OA) must meet or exceed 85%. Sensitivity determined if SK006 could detect a range of CPS expression levels in ENDO specimens. Combined precision measured inter-instrument, -operator, and -day variation nested within the same study. Intra-run measured repeatability within the same run and Inter/Intra-Observer measured scoring reproducibility by 3 pathologists over 3 non-consecutive days. Robustness evaluated the staining performance of SK006 under varying Target Retrieval Solution (TRS) laboratory conditions: time (18-22min), temperature (95-99°C), pH (5.9-6.3) and re-use (up to 3 uses). Results: The sensitivity data showed that SK006 could detect a range of CPS expression levels in ENDO specimens. Analyses for each analytical validation study, subject to AC, showed NPA, PPA, OA point estimates (PE) of ≥ 95.5% and CI lower bounds (CILB) of ≥ 87.1%. Conclusion: PD-L1 IHC 22C3 pharmDx (SK006) used in conjunction with CPS provides high staining reproducibility and scoring precision for ENDO specimens at the CPS ≥ 1 and CPS ≥ 10 cutoffs, under standard day-to-day laboratory testing and scoring conditions. Citation Format: Julia Hand, Amber Morones, Deanna Moquin, Kelly Martyniuk, Megan Kalpakoff, Siena Tabuena-Frolli, Jay Milo, Stephanie Hund. High staining reproducibility and scoring precision demonstrated in endometrial cancer stained with PD-L1 IHC 22C3 pharmDx used in conjunction with combined positive score [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3754.

A simple, clinically usable whole-body MRI system of joint assessment in adolescents and young people with juvenile idiopathic arthritis.

To introduce and evaluate a simple method for assessing joint inflammation and structural damage on whole-body MRI (WBMRI) in juvenile idiopathic arthritis (JIA), which is usable in clinical practice. The proposed system utilizes post-contrast Dixon WBMRI scans. Joints are assessed for synovitis (grade 0-2) and structural damage (present/absent) at 81 sites. The synovitis grading is based on features including above-normal intensity synovial enhancement, synovial hypertrophy, joint effusion, subarticular bone marrow oedema and peri-articular soft tissue oedema.This system was evaluated in a prospective study of 60 young people (47 patients with JIA and 13 controls with non-inflammatory musculoskeletal pain) who underwent a WBMRI. Three readers (blinded to diagnosis) independently reviewed all images and re-reviewed 20 individual scans. The intra- and inter-reader overall agreement (OA) and the intra- and inter-reader Gwet's agreement coefficients 2 (GAC2) were measured for the detection of a) participants with ≥1 joint with inflammation or structural damage and b) joint inflammation or structural damage for each joint. The inter-reader OA for detecting patients with ≥1 joint with inflammation, defined as grade 2 synovitis (G2), and ≥1 joint with structural damage were 80% and 73%, respectively. The intra-reader OA for readers 1-3 was 80-90% and 75-90%, respectively. The inter-reader OA and GAC2 for joint inflammation (G2) at each joint were both ≥85% for all joints but were lower if grade 1 synovitis was included as positive. The intra- and inter-reader agreements of this WBMRI assessment system are adequate for assessing objective joint inflammation and damage in JIA.

Correlation between Magee score and recurrence score in Indian patients with breast cancer.

e12555 Background: Recurrence score (RS) from Oncotype Dx (ODx) test predict chemotherapy benefit in estrogen receptor (ER) positive, and HER2-negative patients with breast cancer. ODx is an expensive test in an Indian practice setting - cost of adjuvant chemotherapy is one-sixth of the cost of the test. Magee equations use histopathological variables, and semiquantitative results from ER, PR and Ki-67, to calculate Magee score (MS) and provide a cost-effective surrogate mechanism for predicting RS. We aimed to assess the correlation between MS and RS in an Indian setting. Methods: Pathology reports were obtained from oncologists in India. Immunohistochemistry for ER and PR were reported using the Allred system. Therefore, two methods were identified to estimate H-scores - using a calculated method (A), and using an expert-approximated (B) method. MS were calculated by a single investigator blinded to the RS. MS and RS were then assessed for correlation via the Pearson correlation method. MS were arbitrarily stratified into ≤ 15, 15 - 30, and ≥ 30, and their overall agreement (OA) or concordance with RS risk categories, ≤ 15, 15 - 30, and ≥ 30, were assessed. The OA was assessed by determining the proportion of patients in similar risk categories across low, intermediate and high-risk RS and MS. Results: Among 34 patients included in the study, 38% (13) had an RS ≤ 15, 38% (13) had an RS between 15 and 30, and 23% (8) had an RS ≥ 30. Correlation coefficient between MS and RS was 0.78 for both methods of estimation (p<0.0001). The OA between MS-A and RS, and MS-B and RS were 59% (20/34) and 47% (16/34) respectively. The concordance between low-risk MS-A and RS is 78% (n=9) and between low-risk MS-B and RS is 50% (n=20). The concordance between high-risk MS-A and RS is 100% (n=2) and high-risk MS-B and RS is 100% (n=2). Conclusions: Although there is good correlation between RS and MS, the concordance between the two in similar risk categories is not optimal. Of the two methods to estimate H-score from Allred score, the calculated method seems to be more optimal. Our pilot study shows good correlation between MS and RS. We recommend that pathologists incorporate H-score as part of the reporting of ER and PR. A larger prospective study will be useful to test our hypothesis for the use of MS as a predictive tool for adjuvant chemotherapy in breast cancer.

Prognostic value of the end of induction PET in patients with follicular lymphoma: results from the analysis of FOLL 12 trial

Aim: To assess the prognostic value of the PET metabolic response using the Deauville Score at the end of induction (EoI) therapy. Methods: FOLL12 trial (NCT02063685) is a multicenter, phase III, randomized study with the primary objective to evaluate whether a FDG-PET (PET) and MRD (molecular residual disease) response-based maintenance therapy (experimental arm) is more effective in terms of Progression-Free Survival (PFS) than a standard maintenance therapy with Rituximab (standard arm) in patients with follicular lymphoma. Adult patients with untreated grade 1–3a FL and stage II-IV were randomized to receive standard immunochemotherapy followed by rituximab maintenance versus standard immunochemotherapy followed by a response adapted post-induction management (i.e. non-maintenance for patients in CMR and with MRD negative, short rituximab therapy for CMR (complete metabolic response) and MRD +ve cases and radioimmunotherapy followed by rituximab maintenance for non-CMR patients). EoI-PET was mandatory. All PET scans were centralized on WIDEN® platform and classified according to Deauville score (DS) in a blind independent central review. DS1-3 was considered negative (CMR), whereas DS4-5 was considered positive (not CMR). The primary endpoint was PFS. Main secondary endpoint was Overall Survival (OS) Results: Overall, 729 follicular lymphoma patients, 52% women, 89% stage III-IV disease and 40% with a high-risk FLIPI-2 score, were included in the analysis. DS at the EoI-PET resulted 1,2,3,4, and 5 in 361(49.5%), 168(23.1%), 112(15.4%), 49(6.7%) and 39(5.3%) patients, respectively. EoI-PET resulted positive in 88/729 (12%). Among the reviewers, the overall agreement (OA) on PET pos/neg result was 0.92, while agreement on positive and negative cases were 0.77 and 0.94, respectively. In the whole population of 729 patients, 5 yrs PFS (95% CI) for DS1-3 and DS4-5 was 69%(65–73) and 35%(25–46) with HR3.32(2.45–4.50), p < 0.001, respectively. The 5 yrs OS (95% CI) in DS1-3 and DS4-5 was 94%(92–96) and 81%(71–88), HR4.72(2.69–8.32), p < 0.001. The prognostic role of CMR was confirmed both in the standard and in the experimental arm of the study. In the whole population, the 5 yrs PFS (95% CI) in DS1-2, DS 3 ad DS4-5 was 74% (69–78), 55% (44–65; HR1.90 (1.36–2.66) p < 0.001) and 37%(26–48; HR 4.01 (2.90–5.54) p < 0.001); DS3 patients had an intermediate 5 yrs PFS rates between those of DS1-2 and DS4-5 patients. OS in DS3 patients had similar values to those with DS1-2. Applying a different definition of CMR (i.e., DS1-2 only) the rate of positive EoI-PET rose to 27%. Conclusion: In FOLL12 trial, EoI-PET according DS criteria was a reliable tool to stratify PFS and OS in advanced follicular lymphoma patients treated with immunochemotherapy; in addition, these data showed DS3 patients as a new category with an intermediate prognosis in terms of PFS. Keywords: Diagnostic and Prognostic Biomarkers, PET-CT No conflicts of interests pertinent to the abstract.

Technical Validation and Clinical Implications of Ultrasensitive PCR Approaches for EGFR-Thr790Met Mutation Detection in Pretreatment FFPE Samples and in Liquid Biopsies from Non-Small Cell Lung Cancer Patients.

In pretreatment tumor samples of EGFR-mutated non-small cell lung cancer (NSCLC) patients, EGFR-Thr790Met mutation has been detected in a variable prevalence by different ultrasensitive assays with controversial prognostic value. Furthermore, its detection in liquid biopsy (LB) samples remains challenging, being hampered by the shortage of circulating tumor DNA (ctDNA). Here, we describe the technical validation and clinical implications of a real-time PCR with peptide nucleic acid (PNA-Clamp) and digital droplet PCR (ddPCR) for EGFR-Thr790Met detection in diagnosis FFPE samples and in LB. Limit of blank (LOB) and limit of detection (LOD) were established by analyzing negative and low variant allele frequency (VAF) FFPE and LB specimens. In a cohort of 78 FFPE samples, both techniques showed an overall agreement (OA) of 94.20%. EGFR-Thr790Met was detected in 26.47% of cases and was associated with better progression-free survival (PFS) (16.83 ± 7.76 vs. 11.47 ± 1.83 months; p = 0.047). In LB, ddPCR was implemented in routine diagnostics under UNE-EN ISO 15189:2013 accreditation, increasing the detection rate of 32.43% by conventional methods up to 45.95%. During follow-up, ddPCR detected EGFR-Thr790Met up to 7 months before radiological progression. Extensively validated ultrasensitive assays might decipher the utility of pretreatment EGFR-Thr790Met and improve its detection rate in LB studies, even anticipating radiological progression.

Design and Multicenter Clinical Validation of a 3-Dimensionally Printed Nasopharyngeal Swab for SARS-CoV-2 Testing

Three-dimensionally printed nasopharyngeal swabs (3DP swabs) have been used to mitigate swab shortages during the coronavirus disease 2019 (COVID-19) pandemic. Clinical validation for diagnostic accuracy and consistency, as well as patient acceptability, is crucial to evaluate the swab's performance. To determine the accuracy and acceptability of the 3DP swab for identifying severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A diagnostic study was conducted from May to July 2020 at 2 tertiary care centers in Singapore with different reference swabs (FLOQSwab [COPAN Diagnostics] or Dacron swab [Deltalab]) and swab processing techniques (wet or dry) to evaluate the performance of the 3DP swab compared with traditional, standard-of-care nasopharyngeal swabs used in health care institutions. The participants were patients with COVID-19 in the first 2 weeks of illness and controls with acute respiratory illness with negative test results for SARS-CoV-2. Paired nasopharyngeal swabs were obtained from the same nostril and tested for SARS-CoV-2 by reverse-transcriptase polymerase chain reaction. The sequence of swabs was randomized based on odd and even participant numbers. Primary outcome measures were overall agreement (OA), positive percentage agreement (PPA), and negative percentage agreement of the 3DP swab compared with reference swabs. Secondary outcome measures were the correlation of cycle threshold (Ct) values of both swabs. The mean (SD) age of participants was 45.4 (13.1) years, and most participants were men (87 of 89 [97.8%]), in keeping with the epidemiology of the COVID-19 pandemic in Singapore. A total of 79 patients with COVID-19 and 10 controls were recruited. Among the patients with COVID-19, the overall agreement and PPA of the 3DP swab was 91.1% and 93.5%, respectively, compared with reference swabs. The PPA was 100% for patients with COVID-19 who were tested within the first week of illness. All controls tested negative. The reverse-transcriptase polymerase chain reaction Ct values for the ORF1ab and E-gene targets showed a strong correlation (intraclass correlations coefficient, 0.869-0.920) between the 3DP and reference swab on independent testing at each institution despite differences in sample processing. Discordant results for both gene targets were observed only at high Ct values. In this diagnostic study of 79 patients with COVID-19 and 10 controls, the 3DP swab performed accurately and consistently across health care institutions and could help mitigate strained resources in the escalating COVID-19 pandemic.

Validation of self-collected buccal swab and saliva as a diagnostic tool for COVID-19

BackgroundEffective management of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) requires large-scale testing to identify and isolate infectious carriers. Self-administered buccal swab and saliva collection are convenient, painless, and safe alternatives to the current healthcare worker (HCW)-collected nasopharyngeal swab (NPS). MethodsA cross-sectional single-centre study was conducted on 42 participants who had tested positive for SARS-CoV-2 via an NPS within the past 7 days. Real-time polymerase chain reaction (RT-PCR) was performed and cycle threshold (Ct) values were obtained for each test. The positive percent agreement (PPA), negative percent agreement (NPA), and overall agreement (OA) were calculated for the saliva samples and buccal swabs, and compared with NPS. ResultsAmong the 42 participants, 73.8% (31/42) tested positive by any one of the three tests. With reference to NPS, the saliva test had PPA 66.7%, NPA 91.7%, and OA 69.0%; the buccal swab had PPA 56.7%, NPA 100%, and OA 73.8%. ConclusionSelf-collected saliva tests and buccal swabs showed only moderate agreement with HCW-collected NPS. Primary screening for SARS-CoV-2 may be performed with a saliva test or buccal swab, with a negative test warranting a confirmatory NPS to avoid false-negatives, minimize discomfort, and reduce the risk of spread to the community and HCWs.

Reproducibility of the classification of early onset scoliosis (C-EOS).

Reproducibility study. Assess the agreement and reliability of the classification of early onset scoliosis (C-EOS). C-EOS is a promising tool for patients with early onset scoliosis (EOS). However, the reliability has only been examined without measuring radiographs and not including the annual progression rate (APR) modifier. We included a single-center consecutive cohort of patients diagnosed with EOS seen in our outpatient clinic. Patients had no previous spine surgery. Four raters rated 60 cases. Two anterior-posterior full-spine radiographs, taken minimum 6months apart, and one sagittal radiograph were measured twice by all raters in a blinded test-retest setup. Results were assessed using crude frequency of overall agreement (OA), intra- and inter-rater Fleiss kappa (κ) statistics, and intraclass correlation coefficient (ICC). We calculated the 95% limits of agreement (LOA) for major curve angle (MCA), kyphosis, and APR using a linear mixed-effects model. Inter- and intra-rater LOA were analyzed for each etiology separately. Mean age was 8.7 ± 3.4years and the etiology were congenital/structural (n = 20), idiopathic (n = 19), neuromuscular (n = 13), or syndromic (n = 8). For etiology, OA was 75.8% and κ = 0.80. For major curve angle, OA was 84.2%, κ = 0.86, ICC = 0.97, and LOA = 12.8°. For kyphosis, OA was 55.8%, κ = 0.52, ICC = 0.87, and LOA = 20.6°. For APR, OA was 76.7%, κ = 0.61, ICC = 0.77, and LOA = 17.4°/year. Inter- and intra-rater LOA were generally largest for neuromuscular and smallest for idiopathic patients. We found substantial agreement for etiology, however, with disagreement in certain cases. The reliability of MCA was excellent; however, somewhat lower for kyphosis and APR with less accuracy. The measurement errors of MCA, kyphosis, and APR depended largely on the etiology. Regarding APR, LOA exceeded the 10°/year increments proposed in the C-EOS, suggesting a revision of this optional modifier. Diagnostic study level 1.

P4343Reconciling acute coronary syndrome diagnoses between linked administrative data and hospital medical records in medical research

Abstract Background/Introduction Administrative data incorporating the International Classification of Diseases 10th Revision (ICD-10) is commonly used in cardiac research. Using patient records, diagnoses are systematically coded by trained coders who have limited/no clinical experience. Therefore, it is important to understand how systematically coded cardiac diagnoses compare with clinically assessed diagnoses to better analyse and interpret studies that have used linked administrative data to adjudicate patient's diagnosis. Purpose To assess the agreement between the acute coronary syndrome (ACS) diagnoses according to linked data compared to those extracted from hospital medical records by clinicians participating in a national registry and determine the factors associated with diagnoses disagreement. Methods The rate of ST-elevation myocardial infarction (STEMI), non-STEMI (NSTEMI) and unstable angina (UA) obtained from the medical records, from admission to discharge, for the nationwide SNAPSHOT ACS audit in 2012 were compared to the corresponding ICD-10 Australian Modification (ICD-10-AM) codes using linked data from 6 jurisdictions covering all Australian states (6) and territories (2). The proportions of the overall agreement (OA), the positive agreement (PA) and the Cohen's weighted kappa and the 95% confidence interval (CI) were derived using both data sources for STEMI, NSTEMI and UA individually, where kappa≥0.8 confers strong agreement and 0.6≤kappa&lt;0.8 moderate agreement. The factors associated with the diagnostic disagreement were explored using multilevel multivariable logistic regression model (backward selection method), accounting for the hospital clustering effect. Results Overall, 3130 patients had both medical records and linked data available for comparison. The degree of agreement was greatest for STEMI and lowest for UA (STEMI: OA=97%, PA=85%, kappa (95% CI)=0.84 (0.81, 0.87); NSTEMI: OA=91%, PA=81%, kappa (95% CI)=0.76 (0.73,0.79); UA: OA = 81%, PA=53%, kappa (95% CI)=0.41 (0.38, 0.45)). Further, the independent factors associated with the disagreement between the medical records and the linked data were the diagnosis of UA (UA vs. STEMI (odds ratio (95% CI)): 6.85 (4.12, 11.40)), not receiving revascularisation (2.27 (1.69, 3.03)), and the state where the ICD-10-AM was coded (p=0.007) (see Figure). Figure 1 Conclusion This study suggests that the agreement between the systematically coded diagnoses from linked administrative data and the diagnosis from the clinical assessment is greater in patients who received revascularisation and worse in those with UA. Also, the degree of agreement varies between states. As the linked data and the ICD codes are being used more often in research to support the evidence-based policies and practice, more attention is needed in testing and improving the accuracy of the ICD-10 codes as well as the ICD-11 codes that are soon to be introduced. Acknowledgement/Funding KH is funded by Heart Foundation Postdoctoral Fellowship. SNAPSHOT data linkage project was funded by the NSW Heart Foundation CVRN Project Grant

Her2 assessment using quantitative reverse transcriptase polymerase chain reaction reliably identifies Her2 overexpression without amplification in breast cancer cases

BackgroundImmunohistochemistry (IHC) and fluorescent-in situ hybridization (FISH) are standard methods to assess human epidermal growth factor receptor 2 (HER2) status in breast cancer (BC) patients. Real-time quantitative polymerase-chain-reaction (qRT-PCR) is able to detect HER2 overexpression. Here we compared FISH, IHC, quantitative PCR (qPCR), and qRT-PCR to determine the concordance rates and evaluate their relative roles in HER2 determination.Patients and methodsWe determined HER2 status in 153 BC patients, using IHC, FISH, Q-PCR and qRT-PCR. In discordant cases, we directly measured HER2 protein levels using Western blotting.ResultsThe overall agreement (OA) between FISH and Q-PCR was 94.1, with a k value of 0.87. Assuming FISH as the standard reference, Q-PCR showed an 86.1% sensitivity and a 99.0% specificity with a global accuracy of 91.6%. OA between FISH and qRT-PCR was 90.8% with a k value of 0.81. Of interest, the disagreement between FISH and qRT-PCR was mostly restricted to equivocal cases. HER2 protein analysis suggested that qRT-PCR correlates better than FISH with HER2 protein levels, particularly where FISH fails to provide conclusive results.SignificanceqRT-PCR may outperform FISH in identifying patients overexpressing HER2 protein. Q-PCR cannot be used for HER2 status assessment, due to its suboptimal level of agreement with FISH. Both FISH and Q-PCR may be less accurate than qRT-PCR as surrogates of HER2 protein determination.

Clinical investigational studies for validation of a next-generation sequencing in vitro diagnostic device for cystic fibrosis testing

Purpose: Clinical investigational studies were conducted to demonstrate the accuracy and reproducibility of the Illumina MiSeqDx CF System, a next-generation sequencing (NGS) in vitro diagnostic device for cystic fibrosis testing. Methods: Two NGS assays – a Clinical Sequencing Assay (Sequencing Assay) and a 139-Variant Assay (Variant Assay) – were evaluated in both an Accuracy Study and a Reproducibility Study, with comparison to bi-directional Sanger sequencing and PCR as reference methods. For each study, positive agreement (PA), negative agreement (NA), and overall agreement (OA) were evaluated. Results: In the Accuracy Study, the Sequencing Assay achieved PA of 99.7% including the polyTG/polyT region and PA of 100% excluding the region. The Variant Assay achieved PA of 100%. NA and OA were >99.99% for both Assays. In the Reproducibility Study, the Sequencing Assay achieved PA of 99.2%; NA and OA were both 99.7%. The Variant Assay achieved PA of 99.8%; NA and OA were both 99.9%. Sample pass rates were 99.7% in both studies for both assays. Conclusion: This is the first systematic evaluation of a NGS platform for broad clinical use as an in vitro diagnostic, including accuracy validation with multiple reference methods and reproducibility validation at multiple clinical sites. These NGS-based Assays had accurate and reproducible results which were comparable to or better than other methods currently in clinical use for clinical genetic testing of cystic fibrosis.

Observer Variation in Drug-Induced Sleep Endoscopy: Experienced Versus Nonexperienced Ear, Nose, and Throat Surgeons

To determine variations in interobserver and intraobserver agreement of drug-induced sleep endoscopy (DISE) in a cohort of experienced versus nonexperienced ear, nose, and throat (ENT) surgeons. Prospective, blinded agreement study. Ninety-seven ENT surgeons (90 nonexperienced with DISE; seven experienced) observed six different DISE videos and were asked to score the upper airway (UA) level (palate, oropharynx, tongue base, hypopharynx, epiglottis), direction (anteroposterior, concentric, lateral), and degree of collapse (none; partial or complete collapse). Findings were collected and analyzed, determining interobserver and intraobserver agreement [overall agreement (OA), specific agreement (SA)] and kappa values per UA level. In the nonexperienced group, overall interobserver agreement on presence of tongue base collapse (OA = 0.63; kappa = 0.33) was followed by the agreement on epiglottis (OA = 0.57; kappa = 0.23) and oropharynx collapse (OA = 0.45; kappa = 0.09). Low overall interobserver agreement in this group was found for hypopharyngeal collapse (OA = 0.33; kappa = 0.08). A similar ranking was found for degree of collapse. For direction of collapse, high interobserver agreement was found for the palate (OA = 0.57; kappa = 0.16). Among the experienced observers, overall interobserver agreement was highest for presence of tongue base collapse (OA = 0.93; kappa = 0.71), followed by collapse of the palate (OA = 0.80; kappa = 0.51). In this group, lowest agreement was also found for hypopharyngeal collapse (OA = 0.47; kappa = 0.03). Interob-server agreement on direction of collapse was highest for epiglottis collapse (OA = 0.97; kappa = 0.97). Concerning the degree of collapse, highest agreement was found for degree of oropharyngeal collapse (OA = 0.82; kappa = 0.66). Among the experienced observers a statistically significant higher interobserver agreement was obtained for presence, direction, and degree of oropharyngeal collapse, as well as for presence of tongue base collapse and degree of epiglottis collapse. Among the nonexperienced observers, high intraobserver agreement was found in particular for tongue base and epiglottis collapse. Among the experienced observers, high agreement was found for all levels but to a lesser extent for hypopharyngeal collapse. Intraobserver agreement was statistically significantly higher in the experienced group, for all UA levels expect for the hypopharynx. This study indicates that both interobserver and intraobserver agreement was higher in experienced versus nonexperienced ENT surgeons. Agreement ranged from poor to excellent in both groups. The current results suggest that experience in performing DISE is necessary to obtain reliable observations.

AN AUTOMATED CHEMILUMINESCENCE IMMUNOASSAY MAY DETECT MOSTLY RELEVANT IgG ANTICARDIOLIPIN ANTIBODIES ACCORDING TO REVISED SYDNEY CRITERIA

Background: Detection of anticardiolipin antibodies (ACA) is an independent laboratory criterion for diagnosis of antiphospholipid syndrome (APS). Alternative methods to ELISA were recently developed such as automated chemiluminescence immunoassay (CLIA).Patients and methods: We compared a CLIA to an ELISA kit for the detection of IgG isotype of ACA. 87 routine samples from 75 patients suspected of having APS were tested using each method. Cut-off values were calculated in our laboratory for each test using 99th percentile of 50 normal controls.Results: Cut-off values were > 20 GPL for ELISA and > 2 GPL for CLIA. Overall agreement (OA), agreement for positive (AP) and agreement for negative (AN) cases were 56.3%, 49.2% and 77.2% respectively. Most discrepant results were positive with ELISA and negative with CLIA. However, OA, AP and AN increased to 82.1%, 84.6% and 80% respectively when CLIA was compared to the repeated ELISA performed at least 12 weeks later. When correlated with APS-related clinical background, CLIA showed lower sensitivity, higher specificity and higher likelihood ratio (LR) as compared to first ELISA whereas these parameters were similar to those of the repeated ELISA. No association was found between any test results and APS-related clinical background of the patients. Using our own cut-off value (> 2GPL), sensitivity, specificity and LR of CLIA to identify patients with APS were respectively 100%, 72.3% and 3.6. A ROC curve showed that at 7.5 GPL cut-off value, specificity and LR improved to 91.1% and 11.25 respectively, without affecting sensitivity. A strong correlation was observed between CLIA results and APS (χ2 = 12.25; p < 0.001).Conclusion: The performance of CLIA is as good as a repeated ELISA test to detect IgG ACA in suspected APS patients. It is fully automated, which represents several advantages over semi-manual ELISA techniques for its implementation in a routine laboratory.

Automated high throughput DNA isolation for detection of human papillomavirus in oral rinse samples

Background Oral HPV infection elevates risk of oropharyngeal cancer, but its natural history is unknown. Natural history studies necessitate validation of an automated, high-throughput method for HPV genomic DNA detection in oral rinse samples (ORS). Objectives To compare agreement of oral HPV detection in ORS processed by a magnetic-bead based automated platform to a previous gold-standard, manual protein-precipitation method. Agreement was compared to that of repeat sampling and repeat HPV testing. Study design HIV-infected individuals ( n = 100) provided two ORS collected 15 min apart. DNA was isolated from equal aliquots by either a protein-precipitation based (Puregene, Qiagen) or magnetic bead-based (QIAsymphony™ SP, Qiagen) method. HPV DNA was detected and type-specified by consensus primer PCR and reverse line blot hybridization. The kappa statistic was used to assess overall agreement (OA) and agreement on a positive test (Ps+). Results The DNA purification methods had very high agreement for categorizing an individual as HPV infected (OA = 0.95; Ps+ = 0.94) as well as for detection of HPV type-specific infection (OA = 0.99; Ps+ = 0.88) in ORS. Agreement for detection of HPV type-specific infection was greater than that observed with repeat oral rinse sampling (OA = 0.99, Ps+ = 0.76) but comparable to inter-assay agreement (OA = 1.00, Ps+ = 0.90). Conclusions HPV detection in ORS processed with a magnetic-bead based automated platform will facilitate large natural history studies of oral HPV infection necessary to evaluate the potential use of oral HPV detection in oral cancer screening.

Reliability Exercise for the Polymyalgia Rheumatica Classification Criteria Study: The Oranjewoud Ultrasound Substudy

Objective. A study supported by the EULAR and the ACR being conducted to establish classification criteria for polymyalgia rheumatica (PMR) will include ultrasound examination of the shoulders and hips. Ultrasound (US) depicts glenohumeral joint effusion, biceps tenosynovitis, subdeltoid bursitis, hip joint synovitis, and trochanteric bursitis in PMR. These findings may aid in distinguishing PMR from other diseases. The purpose of this study was to assess standards and US interreader agreement of participants in the PMR classification criteria study. Methods. Sixteen physicians in four groups examined shoulders and hips of 4 patients and 4 healthy adults with ultrasound. Overall agreement and interobserver agreement were calculated. Results. The overall agreement (OA) between groups was 87%. The OA for healthy shoulders was 88.8%, for healthy hips 100%, for shoulders with pathology 85.2%, and 74.3% for hips with pathology, respectively. Conclusion. There was a high degree of agreement found for the examination of healthy shoulders and pathologic hips. Agreement was moderate for pathologic shoulders and perfect for healthy hips. US of shoulder and hips performed by different examiners is a reliable and feasible tool for assessment of PMR related disease pathology and can be incorporated into a classification criteria study.