Nationwide External Quality Assessment Research Articles

Diagnostic performance of mpox virus (MPXV) real-time PCR assays: multicenter assessment and extended sensitivity analysis.

To comprehensively investigate the diagnostic performance of routinely used assays in MPXV testing, the National Center of Clinical Laboratories in China conducted a nationwide external quality assessment (EQA) scheme and an evaluated nine assays used by ≥ 5 laboratories in the EQA. MPXV virus-like particles with 2700, 900 and 300 copies/mL were distributed to 195 EQA laboratories. For extended analysis, triple-diluted samples from 9000 to 4.12 copies/mL were repeated 20 times using the assays employed by ≥ 5 laboratories. The diagnostic performance was assessed by analyzing EQA data and calculating the limits of detection (LODs). The performance was competent in 87.69% (171/195) of the participants and 87.94% (175/199) of the datasets. The positive percentage agreements (PPAs) were greater than 99% for samples at 2700 and 900 copies/mL, and 95.60% (761/796) for samples at 300 copies/mL. The calculated LODs for the two clades ranged from 228.44 to 924.31 copies/mL and were greater than the LODs specified by the respective kits. EasyDiagnosis had the lowest calculated LODs and showed superior performance in EQA, whereas BioGerm and Sansure, with higher calculated LODs, did not perform well in EQA. This study provides valuable information from the EQA data and evaluation of the diagnostic performance of MPXV detection assays. It also provided insights into reagent optimization and enabled prompt public health interventions for the outbreak.

National external quality assessment for molecular detection of severe acute respiratory syndrome coronavirus 2 Delta variant

Objective: To clarify the impact of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Delta variant on the performance of existing molecular diagnostic assays, and investigate the detection ability of clinical laboratories across China. Methods: The first nationwide external quality assessment (EQA) for molecular detection of Delta variant was carried out based on the non-infectious phage virus-like particles samples, which were prepared by genetic engineering methods and distributed to 8 488 laboratories nationwide. The EQA panel was composed of three Delta variant samples (7.5×102, 1.5×103 and 6.0×103 copies/ml), one non-variant weak positive sample and one negative sample. The percentage of agreement (PA) of Delta variant samples with different concentration, the PA of Delta variant and non-variant samples with 7.5×102 copies/ml, the PA of assays used by more than 100 laboratories for Delta variant samples with different concentration and the PA of Delta variant and non-variant samples with 7.5×102 copies/ml were calculated and analyzed. Results: The data from 8 127 laboratories were available for evaluation. The testing capability of 98.77% (8 027/8 127) of the participating laboratories was found to be competent in reporting correct results for all samples. The overall percentage of agreement (OPA), negative percentage of agreement (NPA) and positive percentage of agreement (PPA) of the samples were 99.64% (40 490/40 635), 99.73% (8 105/8 127), 99.62% (32 385/32 508), respectively. With the decrease of the concentration of the samples, the PPA of Delta variant samples decreased. The PPAs were 99.41% and 99.51% for Delta variant and non-variant samples with 7.5×102 copies/ml, respectively, with no statistical difference (P=0.392). The OPA, NPA and PPA of the assays used by more than 100 laboratories were all greater than 98%, and no statistical difference of the PPAs was identified between Delta variant and non-variant samples with 7.5×102 copies/ml (P>0.05). Conclusions: Delta variant fails to impair the performance of current molecular diagnostic assays in China. The clinical laboratories have the same detection capabilities for Delta variant and non-variant samples. However, in certain laboratories, further improvement is required to ensure the accurate detection of weak positive samples.

Nationwide external quality assessment of SARS-CoV-2 nucleic acid amplification tests in Japan

ObjectivesWe conducted a nationwide external quality assessment (EQA) study of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleic acid amplification testing in Japan. MethodsA total of 563 public health and private sector laboratories participated. The EQA samples comprised 6 RNA and full-process controls. ResultsThe overall agreements were 99.3% and 97.9% for the RNA and full-process controls, respectively. A total of 530/563 (94.1%) laboratories reported correct results; public health laboratories had the highest accuracy. Thirty-three laboratories reported at least one incorrect result (26 laboratories of medical facilities, 5 commercial laboratories, 1 public health laboratory, and 1 other). Sixteen laboratories of medical facilities that used a fully automated assay system failed to detect the presence of the full-process control, due to inherent insufficiency in the limit of detection (LOD). Other causes of incorrect results included failure to ensure the LOD (n = 13), error in result judging or reporting (n = 3), and error in sample handling (n = 1). ConclusionsPerformance was mostly dependent on the laboratory category and assay evaluation, particularly the LOD. Guidance should be developed based on these results, particularly in the phase of new entry into laboratory services for SARS-CoV-2.

Laboratory practice of organic acid analysis based on gas chromatography–mass spectrometry in China

Abstract Objectives The aim of this study is to investigate the status of laboratory practice of organic acid (OA) analysis using gas chromatography–mass spectrometry in China. Methods A survey, investigating details of laboratory practice of OA analysis, was issued on the website of the National Center for Clinical Laboratories of China. Nationwide external quality assessment participating laboratories of OA assay were informed to participate in this survey. Results A total of 36 laboratories completed this survey. Most laboratories started OA analysis during 2016–2020. Most (100%) labs reported semi-quantitative results, in which 79.4% of labs adopted the form of the ratio of peak area of OA and quantitative internal standard. Rare labs reported quantitative results. Few labs released reports in three days, most in 5–7 days. The source of control materials varied, 64.5% of labs adapted self-made materials. A total of 43.8% of laboratories directly used reference intervals (RIs) from published literature, 43.8% of laboratories established RIs themselves, but 21.2% of laboratories reported they didn’t verify RIs. Conclusions Appropriate supervision for the organic acid assay is needed in the aspect of the turnaround time of reporting results, the establishment validation and verification of reference ranges, and the quantification of results.

Nationwide External Quality Assessment of SARS-CoV-2 Molecular Testing, South Korea.

External quality assessment (EQA) is essential for ensuring reliable test results, especially when laboratories are using assays authorized for emergency use for newly emerging pathogens. We developed an EQA panel to assess the quality of real-time reverse transcription PCR assays being used in South Korea to detect severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). With the participation of 23 public health organization laboratories and 95 nongovernmental laboratories involved in SARS-CoV-2 testing, we conducted qualitative and semiquantitative performance assessments by using pooled respiratory samples containing different viral loads of SARS-CoV-2 or human coronavirus OC43. A total of 110 (93.2%) laboratories reported correct results for all qualitative tests; 29 (24.6%) laboratories had >1 outliers according to cycle threshold values. Our EQA panel identified the potential weaknesses of currently available commercial reagent kits. The methodology we used can provide practical experience for those planning to conduct evaluations for testing of SARS-CoV-2 and other emerging pathogens in the future.

Trueness verification survey for blood lead concentration measurement in Chinese clinical laboratories

The results of trueness verification for blood lead measurement were analyzed in a 2016 nationwide external quality assessment (EQA) survey, in order to evaluate current approaches of blood lead measurement in Chinese Laboratories and their traceability to certified reference materials (CRMs). A panel of 2 frozen whole blood CRMs with values assigned by an absolute quantitative method was provided to 12 laboratories with satisfactory performances from the 2015 blood lead EQA scheme. Participants were required to measure the samples 5 times each day for 3 consecutive days and report all 15 data, which were then calculated for means and standard deviations. Data were statistically analyzed by adopted method groups, before comparing with the assigned values to assess the trueness of blood lead measurement using different approaches. Eleven laboratories reported on schedule, rendering a reporting rate of 91.7 %. The target values for the two batches were (37.6 ± 2.7) μg/L and (124.6 ± 4.5) μg/L, respectively. By the evaluation criterion of target value ± 20 µg/L, the pass rates were 72.7 % for the lower concentration and 54.5 % for the higher concentration, while 5 out of 11 laboratories delivered satisfactory results for both CRMs. The overall performance of the few participating laboratories in this EQA survey was basically acceptable by CLSI standard on the whole, yet barely desirable for the higher concentration. Therefore, trueness verification of blood lead measurement with CRMs is potentially necessary for national EQA schemes, which can be imperative as part of the validation procedures of analytical methods in clinical settings.

118P - Nationwide external quality assessment (EQA) of EGFR testing in circulating tumor DNA: The French experience

118P - Nationwide external quality assessment (EQA) of EGFR testing in circulating tumor DNA: The French experience

External quality assessment of medical laboratories in Croatia: preliminary evaluation of post-analytical laboratory testing

IntroductionProper standardization of laboratory testing requires assessment of performance after the tests are performed, known as the post-analytical phase. A nationwide external quality assessment (EQA) scheme implemented in Croatia in 2014 includes a questionnaire on post-analytical practices, and the present study examined laboratory responses in order to identify current post-analytical phase practices and identify areas for improvement.Materials and methodsIn four EQA exercises between September 2014 and December 2015, 145-174 medical laboratories across Croatia were surveyed using the Module 11 questionnaire on the post-analytical phase of testing. Based on their responses, the laboratories were evaluated on four quality indicators: turnaround time (TAT), critical values, interpretative comments and procedures in the event of abnormal results. Results were presented as absolute numbers and percentages.ResultsJust over half of laboratories (56.3%) monitored TAT. Laboratories varied substantially in how they dealt with critical values. Most laboratories (65-97%) issued interpretative comments with test results. One third of medical laboratories (30.6-33.3%) issued abnormal test results without confirming them in additional testing.ConclusionOur results suggest that the nationwide post-analytical EQA scheme launched in 2014 in Croatia has yet to be implemented to the full. To close the gaps between existing recommendations and laboratory practice, laboratory professionals should focus on ensuring that TAT is monitored and lists of critical values are established within laboratories. Professional bodies/institutions should focus on clarify and harmonized rules to standardized practices and applied for adding interpretative comments to laboratory test results and for dealing with abnormal test results.

Performance of Microscopy for the Diagnosis of Malaria and Human African Trypanosomiasis by Diagnostic Laboratories in the Democratic Republic of the Congo: Results of a Nation-Wide External Quality Assessment.

The present External Quality Assessment (EQA) assessed microscopy of blood parasites among diagnostic laboratories in the Democratic Republic of the Congo. The EQA addressed 445 participants in 10/11 provinces (October 2013–April 2014). Participants were sent a panel of five slides and asked to return a routinely stained slide which was assessed for quality of preparation and staining. Response rate was 89.9% (400/445). For slide 1 (no parasites), 30.6% participants reported malaria, mostly Plasmodium falciparum. Only 11.0% participants reported slide 2 (Plasmodium malariae) correctly, 71.0% reported “malaria” or “Plasmodium falciparum” (considered acceptable). Slide 3 contained Plasmodium falciparum (109/μl) and Trypanosoma brucei brucei trypomastigotes: they were each reported by 32.5% and 16.5% participants respectively, 6.0% reported both. Slide 4 (Trypanosoma) was recognised by 44.9% participants. Slide 5 (Plasmodium ovale) was correctly reported by 6.2% participants, another 68.8% replied “malaria” or “Plasmodium falciparum” (considered acceptable). Only 13.6% of routine slides returned were correctly prepared and stained. The proportion of correct/acceptable scores for at least 4/5 slides was higher among EQA-experienced participants compared to first time participants (40.9% versus 22.4%, p = 0.001) and higher among those being trained < 2 years ago compared to those who were not (42.9% versus 26.3%, p = 0.01). Among diagnostic laboratories in Democratic Republic of the Congo, performance of blood parasite microscopy including non-falciparum species and Trypanosoma was poor. Recent training and previous EQA participation were associated with a better performance.

Status of External Quality Assessment on Tumor Markers in China.

The nationwide external quality assessment (EQA) of tumor markers in China has been launched for years. The quality of the performance of Chinese clinical laboratories on tumor markers is partly reflected through analysis of EQA results. This report presents an 8-year EQA result of the six most common tumor markers from 2006 to 2013. Ten freeze-dried EQA samples were distributed to participants every year. Satisfactory performance was defined as scores of more than 80% of acceptable responses with the evaluation criterion of ± 25%. The robust coefficient of variability (CV) between laboratories and percentage difference against the target value of each sample were also calculated by year. A total number of 1154 laboratories submitted results in 2013, which was more than threefold of 2006. The proportion of laboratories with satisfactory performance showed an overall rising trend over the years and was up to 95% for the second survey in 2013. The overall decrease of robust CV was observed for all analytes including alpha fetoprotein (AFP), carcinoembryonic antigen (CEA), total prostate specific antigen (t-PSA), cancer antigen 125 (CA 125), cancer antigen 15-3 (CA 15-3), and cancer antigen 19-9 (CA 19-9) except for CEA, which exhibited a rise followed by a flat trend. The percentage difference narrowed gradually and was less than 2% in 2013. The 8-year EQA results showed a significant enhancement of degree of harmonization of tumor markers in China. However, standardization among various testing systems and improvement of harmonization has yet to be achieved.

Improvement of the quality of BRAF testing in melanomas with nationwide external quality assessment, for the BRAF EQA group

BackgroundKnowledge about tumour gene mutation status is essential for the treatment of increasing numbers of cancer patients, and testing quality has a major impact on treatment response and cost. In 2012, 4,629 tests for BRAF p.V600 were performed in France, in patients with melanomas.MethodsTwo batches of unstained melanoma sections were sent, in May and November 2012, to the 46 laboratories supported by the French National Institute of Cancer (INCa). An external quality assessment (EQA) evaluated mutation status, response times and compliance with INCa recommendations.ResultsAll the French laboratories involved in testing participated in the EQA. Fourteen different methods were used to detect BRAF mutations, most consisting of combinations of in-house techniques. False responses were noted in 25/520 cases (4.8%), 11 of which concerned confusion between p.V600E and p.V600K. Thus, 2.7% of responses would have led to inappropriate treatment. Within six months, mean response times decreased from 22 to 12 days (P<0.001), and the percentage of samples evaluated by a pathologist for tumour cell content increased, from 75.2% to 96.9% (P<0.001).ConclusionDespite the use of non-certified methods, the false response rate was low. Nationwide EQA can improve the quality of molecular pathology tests on tumours.

Performance and Cost Efficiency of KRAS Mutation Testing for Metastatic Colorectal Cancer in Routine Diagnosis: The MOKAECM Study, a Nationwide Experience

PurposeRapid advances in the understanding of cancer biology have transformed drug development thus leading to the approval of targeted therapies and to the development of molecular tests to select patients that will respond to treatments. KRAS status has emerged as a negative predictor of clinical benefit from anti-EGFR antibodies in colorectal cancer, and anti-EGFR antibodies use was limited to KRAS wild type tumors. In order to ensure wide access to tumor molecular profiling, the French National Cancer Institute (INCa) has set up a national network of 28 regional molecular genetics centers. Concurrently, a nationwide external quality assessment for KRAS testing (MOKAECM) was granted to analyze reproducibility and costs.Methods96 cell-line DNAs and 24 DNA samples from paraffin embedded tumor tissues were sent to 40 French laboratories. A total of 5448 KRAS results were collected and analyzed and a micro-costing study was performed on sites for 5 common methods by an independent team of health economists.ResultsThis work provided a baseline picture of the accuracy and reliability of KRAS analysis in routine testing conditions at a nationwide level. Inter-laboratory Kappa values were >0.8 for KRAS results despite differences detection methods and the use of in-house technologies. Specificity was excellent with only one false positive in 1128 FFPE data, and sensitivity was higher for targeted techniques as compared to Sanger sequencing based methods that were dependent upon local expertise. Estimated reagent costs per patient ranged from €5.5 to €19.0.ConclusionThe INCa has set-up a network of public laboratories dedicated to molecular oncology tests. Our results showed almost perfect agreements in KRAS testing at a nationwide level despite different testing methods ensuring a cost-effective equal access to personalized colorectal cancer treatment.

Tests for the identification and isolation of Neis seria gonorrhoeae: a nationwide external quality assessment in 203 medical and healthcare facilities providing sexually transmitted disease services in 2011

Objective To make a nationwide external quality assessment of tests for isolation and identification of N.gonorrhoeae in medical and healthcare facilities at different levels,and to analyze current problems.Methods Lyophilized quality control samples were uniformly delivered to 252 medical and healthcare facilities providing sexually transmitted disease (STD) services at different levels.Test results were analyzed by the National Center for STD Control,China Center for Disease Control and Prevention,and evaluation results were fed back to participating laboratories.Results Finally,test results were received from 203 (80.56％) facilities.The comprehensive score averaged at 87.14,and facilities achieving a comprehensive score of 80 or greater amounted to 80.30％ (163/203).The coincidence rate was 53.69％ (109/203) for all of the 5 quality control samples,82.76％ (168/203) and 87.68％ (178/203) respectively for two quality control samples containing only N.gonorrhoeae,86.21％ (175/203) and 96.06％ (195/203) respectively for a sample containing Neisseria sicca and a sample containing Enterococcus faecalis,69.46％ (141/203)for a sample containing different species of Neisseria.Conclusion The external quality assessment reveals a disparity in the capability to isolate and identify Neisseria among medical and healthcare facilities providing STD services at different levels.

The application of quality specifications derived from biological variation in routine chemistry external quality assessment and internal quality control

Objective To discuss the application of allowable total error (TEa) and allowable imprecision derived from biological variation in routine chemistry external quality assessment (EQA) and internal quality control (IQC) and set up quality specifications of routine chemistry in our country. Methods Data of test items including K，Na，CI，Ca，P，Glu，Urea，UA，Cre，Alb，TP，TC，TG，AST，ALT，TBil，ALP，AMY，CK，LDH，Fe，Mg，Cu，Zn and GGT was collected and evaluated by a nationwide EQA. At the same time the coefficients of variation (CVs) of these test items during the month were acquired from the IQC reports of each laboratory and then the results were analyzed. Results Percent of pass was different in these test items based on TEa derived from biological variation in EQA results.Except for items of CI，Mg，Cu and Zn, about 80% of participant laboratories could achieve the minimum performance of biological variation.About 80% of participant laboratories could achieve the desired performance of biological variation for K，P，Glu，Urea，UA，Cre，TC，TG，ALT，AST，TBil，AMY，CK，LDH，Fe and GGT.About 80% of participant laboratories could achieve the optimum performance of biological variation for Urea，UA，TC，TG，ALT，AST，TBil，CK，and GGT.And the IQC results showed that acceptable percents of different items based on three allowable imprecision were different. More than 80% of participant laboratories could achieve the minimum allowable imprecision for K，P，Glu，Urea，UA，TC，TG，ALT，AST，TBil，AMY，CK，LDH，DBil，Fe，GGT , the desirable imprecision for P，Urea，UA，TG，ALT，AST，TBil，CK，DBil，Fe，GGT and the optimum imprecision for TG，ALT，CK，DBil，Fe. Conclusions The quality specifications derived from biological variation can be as evaluation criteria for EQA and IQC in order to know the detection ability of each laboratory more completely and objectively, set up quality specifications derived from allowable total error and allowable imprecision in routine chemistry and to provide basis for mutual recognition of routine chemistry test results.（Chin J Lab Med，2012,35：531-537） Key words: Clinical chemistry tests; Reference values; Quality control

Nationwide external quality assessment for drug susceptibility testing of Neisseria gonorrhoeae from 2007 to 2009

Objective To make a nationwide external quality assessment for drug sensitivity testing of Neisseria gonorrhoeae, analyze the problems in and factors associated with the drug sensitivity testing, and to enhance the quality of drug sensitivity testing of N. gonorrhoeae at different monitoring sites. Methods Samples were uniformly delivered to monitoring sites by express mail service. Test results were analyzed in the National Center for STD Control, and the evaluation results were fed back to these monitoring sites. Results A total of 105 quality control samples were delivered from 2007 to 2009, with a response rate of 88.57% (93/105). Thirteen monitoring sites were enrolled in the external quality assessment, including 9 laboratories in 2007, 9 in 2008 and 13 in 2009. The total percentage amounted to 77.42% (24/31) for qualified laboratories during the 3 years, including 6 laboratories in 2007, 7 in 2008 and 11 in 2009. The coincidence rate increased for the detection of penicillinase-producing N. gonorrhoeae (PPNG), N. gonorrhoeae with chromosome-mediated ciprofloxacin resistance, and N. gonorrhoeae with chromosome-mediated spectinomycin resistance, and declined for the detection of N. gonorrhoeae with plasmid-mediated high level tetracycline-resistance (TRNG) and N. gonorrhoeae with chromosome-mediated ceftriaxone resistance. Conclusions The 3-year external quality assessment reveals an improvement in the overall quality of drug sensitivity testing of N. gonorrhoeae at national monitoring sites; the accuracy is improved markedly for the detection of PPNG, N. gonorrhoeae with resistance to spectinomycin and ciprofloxacin, but is needed to increase for the detection of ceftriaxone-resis- tant N. gonorrhoeae and TRNG. Key words: Neisseria gonorrhoeae; Microbial sensitivity tests; Quality assurance

Current Status of External Quality Assessment of Fecal Occult Blood Test

Nationwide external quality assessment (EQA) of the fecal occult blood test (FOBT) in Korea was first introduced in 2007-2009. The EQA results were analyzed to assess the current status of FOBT and to plan the continuation of the EQA program. The surveys included 40 hospitals in the preliminary survey conducted in 2007, 249 general hospitals in 2008, and 389 hospitals in 2009. In the surveys, the participating hospitals provided the results of the distributed materials and replies to the questionnaire on the FOBT test procedures and quality controls. In the surveys conducted between 2007 and 2009, a total of 650 institutes submitted 653 test system results; 3 institutes used 2 kinds of methods. All of the institutes used immunologic methods; 107 institutes (16.5%) used quantitative equipments and 546 institutes (84.0%) used qualitative kits. Most quantitative tests yielded consistent positive or negative results; however, their cut-off and measured values differed according to the equipments used. A low-level material tested in 2007 was negative in the quantitative methods but positive in some qualitative methods because of lower detection limits. The discordance rates among quantitative tests were 3.2% in 2007, 4.4% in 2008, and 0% in 2009 and the rates among qualitative tests were 13.8% in 2008 and 2.6% in 2009. Semi-solid EQA materials showed the ability to evaluate the overall test procedures with acceptable stability. In the first Korean FOBT EQA, commercially available EQA materials were proven to be stable. Continuation of the EQA program and further education of laboratory personnel are needed to reduce inconsistency in results. Further, the test kit, procedures, and result reports must be standardized.