Overall Percent Agreement Research Articles

Analytic performance studies and clinical reproducibility of a real-time PCRassay for the detection of epidermal growth factor receptor gene mutations informalin-fixed paraffin-embedded tissue specimens of non-small cell lungcancer

BackgroundEpidermal growth factor receptor (EGFR) gene mutations identify patients with non-small cell lung cancer (NSCLC) who have a high likelihood of benefiting from treatment with anti-EGFR tyrosine kinase inhibitors. Sanger sequencing is widely used for mutation detection but can be technically challenging, resulting in longer turn-around-time, with limited sensitivity for low levels of mutations. This manuscript details the technical performance verification studies and external clinical reproducibility studies of the cobas EGFR Mutation Test, a rapid multiplex real-time PCR assay designed to detect 41 mutations in exons 18, 19, 20 and 21.MethodsThe assay’s limit of detection was determined using 25 formalin-fixed paraffin-embedded tissue (FFPET)-derived and plasmid DNA blends. Assay performance for a panel of 201 specimens was compared against Sanger sequencing with resolution of discordant specimens by quantitative massively parallel pyrosequencing (MPP). Internal and external reproducibility was assessed using specimens tested in duplicate by different operators, using different reagent lots, instruments and at different sites. The effects on the performance of the cobas EGFR test of endogenous substances and nine therapeutic drugs were evaluated in ten FFPET specimens. Other tests included an evaluation of the effects of necrosis, micro-organisms and homologous DNA sequences on assay performance, and the inclusivity of the assay for less frequent mutations.ResultsA >95% hit rate was obtained in blends with >5% mutant alleles, as determined by MPP analysis, at a total DNA input of 150 ng. The overall percent agreement between Sanger sequencing and the cobas test was 96.7% (negative percent agreement 97.5%; positive percent agreement 95.8%). Assay repeatability was 98% when tested with two operators, instruments, and reagent lots. In the external reproducibility study, the agreement was > 99% across all sites, all operators and all reagent lots for 11/12 tumors tested. Test performance was not compromised by endogenous substances, therapeutic drugs, necrosis up to 85%, and common micro-organisms. All of the assessed less common mutations except one (exon 19 deletion mutation 2236_2248 > AGAC) were detected at a similar DNA input level as that for the corresponding predominant mutation.ConclusionThe cobas EGFR Mutation Test is a sensitive, accurate, rapid, and reproducible assay.

Abstract 1707: A real-time PCR assay for detecting EGFR mutations in formalin-fixed paraffin-embedded tissue (FFPET) specimens of non-small cell lung cancer (NSCLC)

Abstract EGFR mutations identify patients with NSCLC who have a high likelihood of benefiting from first-line treatment with anti-EGFR tyrosine kinase inhibitors. Sanger sequencing is widely used for mutation detection but can be time-consuming, and has limited sensitivity for low levels of mutations. We describe the analytic performance of the cobas EGFR Mutation Test, a multiplex, 3-reaction real-time PCR assay designed to detect 41 mutations in exons 18, 19, 20 and 21. The mutations detected and the mutation report calls are listed below: The amount of DNA required for the assay (150 ng) can typically be isolated from a single 5-micron FFPET section, and the test can be performed in < 8 hours. Analytic sensitivity was assessed using DNA blends from NSCLC FFPET tumor sections. A >95% hit rate was obtained in blends with >5% mutant alleles for L858R and exon 19 deletions as determined by 454 sequencing (quantitative massively parallel pyrosequencing) at a total DNA input of 150ng, or 50 ng per PCR amplification. The cobas test was compared to 2x bidirectional Sanger sequencing using a set of 152 NSCLC FFPET specimens. The overall percent agreement (OPA) between the 2 methods was 96.7% (negative agreement - NPA 97.5%; positive agreement - PPA 95.8%). Specimens with discordant cobas EGFR Test and Sanger results were analyzed by 454 and a revised agreement analysis was performed based on the composite results of the 3 assays. The revised OPA was 98.7% (NPA 98.8%; PPA 98.6%). The call repeatability of a panel of NSCLC FFPET specimens was 98% when tested with two operators, instruments, and reagent lots. Necrotic tissue, hemoglobin, triglycerides, and common respiratory organisms did not interfere with the assay. The test showed no cross reactivity with the corresponding exon sequences from the HER2, HER3, and HER4 genes. These analytic studies demonstrate that the cobas EGFR Mutation Test is a sensitive, accurate and reproducible assay. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 1707. doi:1538-7445.AM2012-1707

Evaluation of Four Qualitative Third-Generation HIV Antibody Assays and the Fourth-Generation Abbott HIV Ag/Ab Combo Test

Objective: Compare the agreement between four third-generation rapid or automated human immunodeficiency virus (HIV) tests vs the fourth-generation Abbott ARCHITECT HIV Ag/Ab Combo assay (Combo Test) (Abbott Laboratories, Abbott Park, IL) in 7 clinical laboratories located throughout the Dallas-Fort Worth, TX, metroplex area. Methods: We tested a total of 220 specimens from the general population of patients being screened for HIV antibody status, including Western blot (WB) and HIV-1 RNA viral load by NAAT in patients with initially positive HIV screening tests. We performed a method precision study of the Combo Test and evaluated the agreement between the 4 comparative methods (CMs) against the Combo Test method (TM) using a 2 × 2 contingency table to determine values for positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) according to the guideline (EP12-A2) for the evaluation of qualitative assays published by the Clinical Laboratory Standards Institute (CLSI). Results: The Combo Test yielded consistent (reproducible) results when the HIV antibody (or p24 antigen) concentration was 20% away or more from C50. In addition, values for PPA, NPA, and OPA for CM vs TM results were all 100%, or not significantly different from 100%. Conclusion: The high detection rate of the Combo Test, when coupled with its high diagnostic sensitivity and specificity, rapid TAT (<30 minutes) of test results using a fully automated instrument (ARCHITECT), and excellent agreement with HIV-1/2 antibody results by several commercially available HIV-1/2 screening tests and with WB and NAAT results, indicate that the Abbott HIV-1/2 Ab/Ag Test is a useful screening test in the identification of HIV-infected individuals. * HIV : human immunodeficiency virus CM : comparative method TM : test method PPA : positive percent agreement NPA : negative percent agreement OPA : overall percent agreement CLSI : Clinical Laboratory Standards Institute AIDS : acquired immunodeficiency syndrome UNAIDS : United Nations Programme on HIV/acquired immunodeficiency syndrome CDC : Centers for Disease Control gag : group-specific antigen EIAs : enzyme immunoassays CAP : College of American Pathologists WB : Western blot NAAT : nucleic acid amplification test AMH : Arlington Memorial Hospital BAS : Baylor All-Saints BUMC : Baylor University Medical Center THD : Texas Health Presbyterian Hospital, Dallas THFW : Texas Health Harris Methodist Hospital THP : Texas Health Presbyterian Hospital THSW : Texas Health Harris Methodist Hospital Southwest R : reactive NR : nonreactive EDTA : ethylenediaminetetraacetic acid LL : lower limits UP : upper limits S/CO : signal-to-cutoff %pos : percentage of positive %neg : percentage of negative FP : false positive FN : false negative POC : point-of-care TAT : turnaround time

Pattern recognition of EEG-EOG as a technique for all-night sleep stage scoring

This study reports an approach to using a pattern recognition system, simulated on a digital computer, for automatic, all-night sleep stage scoring. A detailed comparison of percent agreement for the computer and three experienced human judges is presented. The results indicate that sleep scorers agreed among themselves approximately 89% of the time. The over-all percent agreement between the computerized recognition system and the consensus formed by a majority vote of the three human judges was 7% less than the average pairwise agreement among human scorers. The consensus scoring showed slightly better agreement with the automatic scoring system than the individual scorer vs. computer comparison. An argument can be made for eliminating the discrimination of stage 3 as a separate sleep stage since average agreement among human scorers for this stage was only 60%. When stages 3 and 4 were combined, performance increased for both humans and computer. The sleep scorers agreed among themselves approximately 91% of the time. The computer's average percent agreement was only 6% lower than the human agreement. From the standpoint of accuracy, the computer scoring system has achieved a satisfactory performance level and may be considered as a useful alternative to visual sleep stage scoring. Performance is expected to increase with refinement of the K complex detector and provision of a more reliable means of identifying awake periods. In addition, the system design can be readily modified to accomodate specific subjects with the attendant increase in classification accuracy.