Interlaboratory Performance Research Articles

Report from the HarmoSter study: different LC-MS/MS androstenedione, DHEAS and testosterone methods compare well; however, unifying calibration is a double-edged sword.

Current liquid chromatography-tandem mass spectrometry (LC-MS/MS) applications for circulating androgen measurements are technically diverse. Previously, variable results have been reported for testosterone. Data are scarce for androstenedione and absent for dehydroepiandrosterone sulfate (DHEAS). We assessed the agreement of androstenedione, DHEAS and testosterone LC-MS/MS measurements among nine European centers and explored benefits of calibration system unification. Androgens were measured twice by laboratory-specific procedures in 78 patient samples and in EQA materials. Results were obtained by in-house and external calibration. Intra- and inter-laboratory performances were valued. Intra-laboratory CVs ranged between 4.2-13.2 % for androstenedione, 1.6-10.8 % for DHEAS, and 4.3-8.7 % and 2.6-7.1 % for female and male testosterone, respectively. Bias and trueness in EQA materials were within ±20 %. Median inter-laboratory CV with in-house vs. external calibration were 12.0 vs. 9.6 % for androstenedione (p<0.001), 7.2 vs. 4.9 % for DHEAS (p<0.001), 6.4 vs. 7.6 % for female testosterone (p<0.001) and 6.8 and 7.4 % for male testosterone (p=0.111). Median bias vs. all laboratory median with in-house and external calibration were-13.3 to 20.5 % and-4.9 to 18.7 % for androstenedione,-10.9 to 4.8 % and-3.4 to 3.5 % for DHEAS,-2.7 to 6.5 % and-11.3 to 6.6 % for testosterone in females, and-7.0 to 8.5 % and-7.5 to 11.8 % for testosterone in males, respectively. Methods showed high intra-laboratory precision but variable bias and trueness. Inter-laboratory agreement was remarkably good. Calibration system unification improved agreement in androstenedione and DHEAS, but not in testosterone measurements. Multiple components, such as commutability of calibrators and EQA materials andinternal standard choices, likely contribute to inter-laboratory variability.

Analysis of Clinical Laboratory Detecting Challenging Variants from Exome Sequencing Using Simulated Patient–Parent Trio Sample: Pilot Study for Neurodevelopmental Disorder de Novo Variants

To date, there has been no systematic analysis for the clinical laboratory in detecting technically challenging variants using the trio-based exome sequencing (ES) approach. Here, we present an interlaboratory pilot proficiency testing study that used synthetic patient-parent specimens to assess the detection of challenging variants with de novo dominant inheritance modes for neurodevelopmental disorders using various trio-based ES. In total, 27 clinical laboratories that performed diagnostic exome analyses participated in the survey. One of the 26 challenging variants was identified by all laboratories, whereas all 26 variants were identified by only nine laboratories. The lack of identification of mosaic variants was often due to the bioinformatics analysis that excluded the variant. For missing intended heterozygous variants, probable root causes were related to the technical bioinformatics pipeline and variant interpretation and reporting. For each missing variant, there may be more than one probable reason from the different laboratories. There was considerable variation in interlaboratory performance for detecting challenging variants using trio-based ES. This finding may have important implications for the design and validation of tests for different variant types in clinical laboratories, especially for technically challenging variants, and necessary workflow modification can potentially improve trio-based ES performance.

Forensic interlaboratory evaluations of a systematic method for examining, documenting, and interpreting duct tape physical fits

This paper describes two interlaboratory studies of a novel method for duct tape physical fit examinations, through the collaboration of 38 forensic practitioners from 23 laboratories. Each participant analyzes seven duct tape pairs, representing casework-like fits and non-fits. The ground truth is maintained blind to participants, who are instructed to follow a pilot method to examine the tapes, document bin-by-bin observations, estimate the edge similarity score (ESS) as a metric for the quality of the fit, and report a conclusion. The results from 266 examinations are compared across participants and to a consensus ESS established prior to administering the studies by an independent panel. The participant’s feedback from the first study is applied to refine the instructions, training, and reporting tools for the second exercise. Both interlaboratory studies evaluate the inter-examiner variability using performance rates based on ESS results and conclusions provided by the participants, showing an overall accuracy of 95% and 99% for the first and second exercises, respectively. When the data is compared to the consensus ESS from the independent panel, the error rate improves from 9.3% to 5.5% in the second study. When assessing the ESS data using z-scores, most participants' results were satisfactory, with eight cautionary and two insufficient results in the first study, and seven cautionary and no insufficient results in the second trial. Overall, the inter-laboratory performances indicate that the proposed qualitative and quantitative criteria assist practitioners in arriving at reproducible assessments of the quality of the fit and demonstrable conclusions.

Report from the HarmoSter study: inter-laboratory comparison of LC-MS/MS measurements of corticosterone, 11-deoxycortisol and cortisone.

Liquid chromatography-tandem mass spectrometry (LC-MS/MS) panels that include glucocorticoid-related steroids are increasingly used to characterize anddiagnose adrenal cortical diseases. Limited information is currently available about reproducibility of these measurements among laboratories. The aim of the study was to compare LC-MS/MS measurements of corticosterone, 11-deoxycortisol and cortisone at eight European centers and assess the performance after unification of calibration. Seventy-eight patient samples and commercial calibrators were measured twice by laboratory-specific procedures. Results were obtained according to in-house and external calibration. We evaluated intra-laboratory and inter-laboratory imprecision, regression and agreement against performance specifications derived from 11-deoxycortisol biological variation. Intra-laboratory CVs ranged between 3.3 and 7.7%, 3.3 and 11.8% and 2.7 and 12.8% for corticosterone, 11-deoxycortisol and cortisone, with 1, 4 and 3 laboratories often exceeding the maximum allowable imprecision (MAI), respectively. Median inter-laboratory CVs were 10.0, 10.7 and 6.2%, with 38.5, 50.7 and 2.6% cases exceeding the MAI for corticosterone, 11-deoxycortisol and cortisone, respectively. Median laboratory bias vs. all laboratory-medians ranged from-5.6 to 12.3% for corticosterone,-14.6 to 12.4% for 11-deoxycortisol and-4.0 to 6.5%for cortisone, with few cases exceeding the total allowable error. Modest deviations were found in regression equations among most laboratories. External calibration did not improve 11-deoxycortisol and worsened corticosterone and cortisone inter-laboratory comparability. Method imprecision was variable. Inter-laboratory performance was reasonably good. However, cases with imprecision and total error above the acceptable limits were apparent for corticosterone and 11-deoxycortisol. Variability did not depend on calibration but apparently onimprecision, accuracy and specificity of individual methods. Tools for improving selectivity and accuracy are required to improve harmonization.

Interlaboratory performance and quantitative PCR data acceptance metrics for NIST SRM® 2917

Surface water quality quantitative polymerase chain reaction (qPCR) technologies are expanding from a subject of research to routine environmental and public health laboratory testing. Readily available, reliable reference material is needed to interpret qPCR measurements, particularly across laboratories. Standard Reference Material® 2917 (NIST SRM® 2917) is a DNA plasmid construct that functions with multiple water quality qPCR assays allowing for estimation of total fecal pollution and identification of key fecal sources. This study investigates SRM 2917 interlaboratory performance based on repeated measures of 12 qPCR assays by 14 laboratories (n = 1008 instrument runs). Using a Bayesian approach, single-instrument run data are combined to generate assay-specific global calibration models allowing for characterization of within- and between-lab variability. Comparable data sets generated by two additional laboratories are used to assess new SRM 2917 data acceptance metrics. SRM 2917 allows for reproducible single-instrument run calibration models across laboratories, regardless of qPCR assay. In addition, global models offer multiple data acceptance metric options that future users can employ to minimize variability, improve comparability of data across laboratories, and increase confidence in qPCR measurements.

Comparability of Steroid Collision Cross Sections Using Three Different IM-HRMS Technologies: An Interplatform Study

Steroids play key roles in various biological processes and are characterized by many isomeric variants, which makes their unambiguous identification challenging. Ion mobility-mass spectrometry (IM-MS) has been proposed as a suitable platform for this application, particularly using collision cross section (CCS) databases obtained from different commercial IM-MS instruments. CCS is seen as an ideal additional identification parameter for steroids as long-term repeatability and interlaboratory reproducibility of this measurand are excellent and matrix effects are negligible. While excellent results were demonstrated for individual IM-MS technologies, a systematic comparison of CCS derived from all major commercial IM-MS technologies has not been performed. To address this gap, a comprehensive interlaboratory comparison of 142 CCS values derived from drift tube (DTIM-MS), traveling wave (TWIM-MS), and trapped ion mobility (TIM-MS) platforms using a set of 87 steroids was undertaken. Besides delivering three instrument-specific CCS databases, systematic comparisons revealed excellent interlaboratory performance for 95% of the ions with CCS biases within ±1% for TIM-MS and within ±2% for TWIM-MS with respect to DTIM-MS values. However, a small fraction of ions (<1.5%) showed larger biases of up to 7% indicating that differences in the ion conformation sampled on different instrument types need to be further investigated. Systematic differences between CCS derived from different IM-MS analyzers and implications on the applicability for nontargeted analysis are critically discussed. To the best of our knowledge, this is the most comprehensive interlaboratory study comparing CCS from three different IM-MS technologies for analysis of steroids and small molecules in general.

Interlaboratory Performance of a Real-Time PCR Method for Detection of Ceratocystisplatani, the Agent of Canker Stain of Platanus spp.

Ceratocystis platani (CP), an ascomycetous fungus, is the agent of canker stain, a lethal vascular disease of Platanus species. Ceratocystis platani has been listed as a quarantine pest (EPPO A2 list) due to extensive damage caused in Southern Europe and the Mediterranean region. As traditional diagnostic assays are ineffective, a Real-Time PCR detection method based on EvaGreen, SYBR Green, and Taqman assays was previously developed, validated in-house, and included in the official EPPO standard PM7/14 (2). Here, we describe the results of a test performance study performed by nine European laboratories for the purpose of an interlaboratory validation. Verification of the DNA extracted from biological samples guaranteed the high quality of preparations, and the stability and the homogeneity of the aliquots intended for the laboratories. All of the laboratories reproduced nearly identical standard curves with efficiencies close to 100%. Testing of blind-coded DNA extracted from wood samples revealed that all performance parameters—diagnostic sensitivity, diagnostic specificity, accuracy and reproducibility—were best fit in most cases both at the laboratory and at the assay level. The previously established limit of detection, 3 fg per PCR reaction, was also validated with similar excellent results. The high interlaboratory performance of this Real-Time PCR method confirms its value as a primary tool to safeguard C. platani-free countries by way of an accurate monitoring, and to investigate the resistance level of potentially canker stain-resistant Platanus genotypes.

Report from the HarmoSter study: impact of calibration on comparability of LC-MS/MS measurement of circulating cortisol, 17OH-progesterone and aldosterone.

Liquid chromatography-tandem mass spectrometry (LC-MS/MS) is recommended for measuring circulating steroids. However, assays display technical heterogeneity. So far, reproducibility of corticosteroid LC-MS/MS measurements has received scant attention. The aim of the study was to compare LC-MS/MS measurements of cortisol, 17OH-progesterone and aldosterone from nine European centers and assess performance according to external quality assessment (EQA) materials and calibration. Seventy-eight patient samples, EQA materials and two commercial calibration sets were measured twice by laboratory-specific procedures. Results were obtained by in-house (CAL1) and external calibrations (CAL2 and CAL3). We evaluated intra and inter-laboratory imprecision, correlation and agreement in patient samples, and trueness, bias and commutability in EQA materials. Using CAL1, intra-laboratory CVs ranged between 2.8-7.4%, 4.4-18.0% and 5.2-22.2%, for cortisol, 17OH-progesterone and aldosterone, respectively. Trueness and bias in EQA materials were mostly acceptable, however, inappropriate commutability and target value assignment were highlighted in some cases. CAL2 showed suboptimal accuracy. Median inter-laboratory CVs for cortisol, 17OH-progesterone and aldosterone were 4.9, 11.8 and 13.8% with CAL1 and 3.6, 10.3 and 8.6% with CAL3 (all p<0.001), respectively. Using CAL1, median bias vs. all laboratory-medians ranged from-6.6 to 6.9%,-17.2 to 7.8% and-12.0 to 16.8% for cortisol, 17OH-progesterone and aldosterone, respectively. Regression lines significantly deviated from the best fit for most laboratories. Using CAL3 improved cortisol and 17OH-progesterone between-method bias and correlation. Intra-laboratory imprecision and performance with EQA materials were variable. Inter-laboratory performance was mostly within specifications. Although residual variability persists, adopting common traceable calibrators and RMP-determined EQA materials is beneficial for standardization of LC-MS/MS steroid measurements.

An Interlaboratory Comparison on the Characterization of a Sub-micrometer Polydisperse Particle Dispersion

The measurement of polydisperse protein aggregates and particles in biotherapeutics remains a challenge, especially for particles with diameters of ≈ 1 µm and below (sub-micrometer). This paper describes an interlaboratory comparison with the goal of assessing the measurement variability for the characterization of a sub-micrometer polydisperse particle dispersion composed of five sub-populations of poly(methyl methacrylate) (PMMA) and silica beads. The study included 20 participating laboratories from industry, academia, and government, and a variety of state-of-the-art particle-counting instruments. The received datasets were organized by instrument class to enable comparison of intralaboratory and interlaboratory performance. The main findings included high variability between datasets from different laboratories, with coefficients of variation from 13 % to 189 %. Intralaboratory variability was, on average, 37 % of the interlaboratory variability for an instrument class and particle sub-population. Drop-offs at either end of the size range and poor agreement on maximum counts of particle sub-populations were noted. The mean distributions from an instrument class, however, showed the size-coverage range for that class. The study shows that a polydisperse sample can be used to assess performance capabilities of an instrument set-up (including hardware, software, and user settings) and provides guidance for the development of polydisperse reference materials.

Inter-laboratory performance of ICE histopathology scoring to identify UN GHS Category 1 surfactants and non-extreme pH detergents

The inter-laboratory performance of Isolated Chicken Eye (ICE) histopathology scoring was assessed for predicting EU CLP/UN GHS Cat. 1 surfactants. Furthermore, the predictive capacity of ICE histopathology was evaluated for the combined dataset of surfactants and existing data for non-extreme pH (2 < pH < 11.5) detergents. Use of ICE histopathology led to increased sensitivity compared to the ICE test method alone for surfactants. When combined with the existing dataset of detergents, use of histopathology in addition to the standard ICE test method decreased the false negative rates from 64% (14/22) to 27% (6/22); increased accuracy from 53% (16/30) to 77% (23/30); and led to acceptable level of false positives (from 0/8 to 1/8 (12.5%). Moreover, good reproducibility of ICE histopathology predictions conducted on the same slides was found between pathologists and peer-reviewers from three independent laboratories (10/12 or 83%) and over time. Use of ICE histopathology was therefore found suitable to predict EU CLP/UN GHS Cat. 1 surfactants and non-extreme pH detergents. In addition, appropriate reproducibility of ICE histopathology was found, provided that i) an internal peer-review system was in place; ii) original slides were assessed to enable evaluation of three dimensional effects; and iii) appropriate training and proficiency appraisal were conducted.

P–105 Clinical validation of mojo AISA, an artificial intelligence robotic CASA system

Abstract Study question Can a CASA system based on Artificial Intelligence perform as well as manual semen assessment, within the WHO error margins? Summary answer The AI-based CASA systems that mimic high quality assessments show great potential for reducing clinical workloads while increasing treatment efficacy. What is known already The field of male-factor fertility investigation is still lacking an automated semen analysis system that can be widely clinically adopted. By leveraging state-of-the-art robotics and Artificial Intelligence (AI), it was possible to build mojo AISA which is an AI and robotic platform designed according to WHO recommendation for semen analysis. This system is based on AI software with a unique convolutional neural network (CNN) that detects and measures sperm concentration and motility while ruling out unwanted cells and debris in raw samples. Study design, size, duration This study presents and validates the mojo AISA device. A total of 60 patient samples at ANOVA Karolinska University Hospital were collected and results from manual assessment were compared to mojo AISA for concentration and motility. Semen samples were assessed manually (WHO 2010) and concurrently with Mojo AISA. Manual measurements ranged from 1–206M/ml. This study lasted from May 2020 to December 2020 following informed consent and ethics committee practices of ANOVA. Participants/materials, setting, methods Sample preparation protocol for mojo AISA consisted of placing two 10ul drops and covering with two 22x22mm coverslip. Manual assessment followed ANOVA EQA procedures akin to the WHO. A CNN was trained using videos captured with mojo AISA as input data. Images were annotated to form a validation set by which the AI was trained. To account for sampling error, videos of Hamilton Thorne Accubeads+ were captured using mojo AISA and the mojo counting chambers. Main results and the role of chance Comparing the concentration measured by mojo AISA with the known value for each microbead, results are in agreement of 86%, within the confidence interval of the microbeads. The mean relative error was 6.7% and maximum error was 11%. Therefore, Accubeads+ validation proved no observational error regarding the use of mojo AISA microscope. As for comparing mojo AISA to manual assessment for concentration, Pearson (Spearman) correlation was 0.95 (0.97). The mean relative error was 24.8% and maximum relative error was 71.1%, where 90% of samples were below 50% error. By looking at the concentration range between 10 and 20 M/ml, mojo AISA displayed a mean error of 18.5%. For motility, as comparing mojo AISA to manual assessment, a result of 35.4% mean relative error was obtained. To conclude, mojo’s robotic solution shows promise for clinical practice as the AI continues to improve. In 6 months, sperm concentration correlation improved by 3-fold. Next, the AI will be further clinically trained for low concentration. Limitations, reasons for caution mojo AISA requires further development, especially for very low concentration ranges, below 5M/ml, due to high sensibility to false positive detections. The same applies to post-vasectomy samples. Additionally, the necessity to compute the motility of each sperm scales poorly with high concentration generating a poor experience for high volume clinics. Wider implications of the findings: Automation is crucial in several industries. It enables fertility clinics &amp; andrologists to standardize male factor infertility measurements (if paired with widespread standardization of protocols for automation) while enabling them to put more focus on demanding activities of their profession and removes human biases of inter-laboratory performance. Trial registration number Not applicable

Interlaboratory Performance in Measurement of Dabigatran and Rivaroxaban.

Assessing direct oral anticoagulant (DOAC) drug levels by reliable laboratory assays is necessary in a number of clinical scenarios. To evaluate the performance of DOAC-specific assays for various concentrations of dabigatran and rivaroxaban, assess the interlaboratory variability in measurement of these DOACs, and investigate the responsiveness of the routine clotting assays to various concentrations of these oral anticoagulants. College of American Pathologists proficiency testing survey data from 2013 to 2016 were summarized and analyzed. For dabigatran, the interlaboratory coefficient of variation (CV) of ecarin chromogenic assay was broad (ranging from 7.5% to 29.1%, 6.3% to 15.5%, and 6.8% to 9.0% for 100-ng/mL, 200-ng/mL, and 400-ng/mL targeted drug concentrations, respectively). The CV for diluted thrombin time for dabigatran was better overall (ranging from 11.6% to 17.2%, 9.3% to 12.3, and 7.1% to 11.2% for 100 ng/mL, 200 ng/mL, and 400 ng/mL, respectively). The rivaroxaban-calibrated anti-Xa assay CVs also showed variability (ranging from 11.5% to 22.2%, 7.2% to 10.9%, and 6.4% to 8.1% for 50-ng/mL, 200-ng/mL, and 400-ng/mL targeted drug concentrations, respectively). The prothrombin time (PT) and activated partial thromboplastin time (aPTT) showed variable dose- and reagent-dependent responsiveness to DOACs: PT was more responsive to rivaroxaban and aPTT to dabigatran. The undiluted thrombin time showed maximum prolongation across all 3 dabigatran concentrations, making it too sensitive for drug-level monitoring, but supporting its use as a qualitative screening assay. DOAC-specific assays performed reasonably well. While PT and aPTT cannot be used safely to determine DOAC degree of anticoagulation, a normal thrombin time excludes the presence of dabigatran.

Evaluation of the GENE-UP® EHEC Detection Method for the Detection of Enterohemorrhagic E. coli in Select Foods: Collaborative Study: First Action Method 2020.06.

The GENE-UP® EHEC assay (Performance Tested MethodSM 121806) is a real-time PCR molecular detection method that utilizes fluorescence resonance energy transfer proprietary hybridization probes for the rapid detection of enterohemorrhagic Escherichia coli (EHEC) in select foods. The purpose of this validation was to evaluate the method's interlaboratory performance and submit the results to AOAC INTERNATIONAL for adoption as a First Action Official Method of AnalysisSM for the detection of EHEC in select foods. The GENE-UP method was evaluated in a multi-laboratory study as part of the MicroVal VALIDATION certification process using unpaired test portions for one food matrix, raw ground beef (85% lean). Collaborators evaluated the candidate method using either an automated or manual lysis procedure. The candidate method was compared to the ISO/TS 13136:2012 method. Data from 17 participants from 15 laboratories throughout the European Union were evaluated. Three levels of contamination were evaluated: a non-inoculated control level (0 CFU/test portion), a low contamination level (∼1 CFU/test portion), and a high contamination level (∼10 CFU/test portion). Data from the study were analyzed according to the probability of detection (POD) statistical model. The difference in laboratory probability of detection (dLPODC) values with 95% confidence interval between the candidate and reference method results were -0.01 (-0.04, 0.02), 0.23 (0.07, 0.39), and 0.06 (0.01, 0.12) for the non-inoculated, low, and high contamination levels, respectively. For the candidate method, values obtained for repeatability and reproducibility were similar to the reference method and indicated minimal variation between samples or between laboratories. No discrepant results (false positive or false negative) were observed for each contamination. A statistical difference was calculated between the candidate and reference method at the low and high inoculation levels, with the candidate method detecting a higher number of positive samples indicating a higher sensitivity than the reference method. No differences in the recovery of the target analyte were observed between the manual and automated lysis procedures. The GENE-UP EHEC Detection Method provides end users a rapid, easy-to-use workflow for the detection of EHEC in food matrixes.

Evaluation of the GENE-UP®Salmonella Method for the Detection of Salmonella Species in a Broad Range of Foods and Select Environmental Surfaces: Collaborative Study, First Action 2020.02.

The GENE-UP®Salmonella assay (Performance Tested MethodSM [PTM] 121802) is a PCR detection method that utilizes fluorescence resonance energy transfer (FRET) hybridization probes for the rapid detection of Salmonella species in foods and on environmental surfaces. The purpose of this validation was to evaluate the method's interlaboratory performance for submission to AOAC INTERNATIONAL for adoption as First Action Official Method of AnalysisSM. The GENE-UP method was evaluated in a multi-laboratory study using unpaired test portions for one food matrix, raw ground beef (80% lean). The candidate method was compared to the US Department of Agriculture (USDA), Food Safety Inspection Service, Microbiology Laboratory Guidebook (MLG) 4.10 reference method. An alternative confirmation procedure, using ASAP™ and CHROMID®Salmonella chromogenic agars, was included in the validation study. Fifteen collaborators from 14 laboratories participated. Three levels of contamination were evaluated: a non-inoculated control level (0 CFU/test portion), a low contamination level (∼0.7 CFU/test portion), and a high contamination level (∼2 CFU/test portion). Data were analyzed using the probability of detection (POD) statistical model. The dLPODC values with 95% confidence interval for the GENE-UP Salmonella method, with either alternative or traditional confirmation, were: 0.00 (-0.03, 0.03), -0.02 (-0.15, 0.12), and 0.02 (-0.03, 0.09) for the non-inoculated, low, and high contamination levels respectively. The dLPODC results demonstrate no difference in performance between the candidate method and reference method for the matrix evaluated. The GENE-UP Salmonella method, with ASAP and CHROMID Salmonella, provides industry with a simplified, rapid, and accurate workflow for the detection of Salmonella in a broad range of foods and select environmental surfaces.

Interlaboratory evaluation of plasma N-glycan antennary fucosylation as a clinical biomarker for HNF1A-MODY using liquid chromatography methods

Antennary fucosylation alterations in plasma glycoproteins have been previously proposed and tested as a biomarker for differentiation of maturity onset diabetes of the young (MODY) patients carrying a functional mutation in the HNF1A gene. Here, we developed a novel LC-based workflow to analyze blood plasma N-glycan fucosylation in 320 diabetes cases with clinical features matching those at risk of HNF1A-MODY. Fucosylation levels measured in two independent research centers by using similar LC-based methods were correlated to evaluate the interlaboratory performance of the biomarker. The interlaboratory study showed good correlation between fucosylation levels measured for the 320 cases in the two centers with the correlation coefficient (r) of up to 0.88 for a single trait A3FG3S2. The improved chromatographic separation allowed the identification of six single glycan traits and a derived antennary fucosylation trait that were able to differentiate individuals carrying pathogenic mutations from benign or no HNF1A mutation cases, as determined by the area under the curve (AUC) of up to 0.94. The excellent (r = 0.88) interlaboratory performance of the glycan biomarker for HNF1A-MODY further supports the development of a clinically relevant diagnostic test measuring antennary fucosylation levels.

Interlaboratory Agreement of Anti-Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Serologic Assays in the Expedited College of American Pathologists Proficiency Testing Program.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a recently emerged, currently pandemic virus, and the etiologic agent of coronavirus disease 2019 (COVID-19). Clinical testing for antibodies to SARS-CoV-2 has rapidly become widespread, but data regarding the interlaboratory performance of these serologic assays are limited. To describe the development and initial results of the College of American Pathologists (CAP) SARS-CoV-2 Serology Survey. Members from the CAP Microbiology and Diagnostic Immunology and Flow Cytometry Committees formed a working group to support development of a new proficiency testing survey for anti-SARS-CoV-2 antibody assays. Supplemental questions in the survey assessed the state of SARS-CoV-2 serologic testing among participating laboratories as of July 2020. Results were analyzed for agreement by immunoglobulin (Ig) isotype tested, assay manufacturer, and methodology. A total of 4125 qualitative results were received from 1110 laboratories participating in the first survey. Qualitative agreement for assays measuring anti-SARS-CoV-2 total antibodies or IgG was greater than 90% for all 3 samples in the survey. Qualitative agreement for IgM and IgA for the negative sample was greater than 95%, but lacked consensus for the other 2 samples. These initial data suggest overall excellent agreement and comparable performance for most qualitative anti-SARS-CoV-2 IgG and total antibody assays across all participating clinical laboratories, regardless of specific target antigen or assay methodology.

The FIRMS Network’s PT scheme: What can be learned about inter-laboratory performance?

Demonstrating comparability of isotope-delta values between different laboratories by participation in inter-laboratory exercises (ILEs) is a vital component of method validation. Since its inception, the Forensic Isotope Ratio Mass Spectrometry (FIRMS) Network has provided non-accredited ILEs to its members. Provision of an accredited proficiency testing (PT) scheme to ISO/IEC 17043:2010 was not possible until the administration of the scheme was outsourced to an accredited PT provider—in this case LGC Standards. LGC’s FIRMS PT scheme has been operating since 2013 and in 2015 it was included in their scope of accreditation to ISO/IEC 17043:2010 and it remains the only such accredited PT for stable isotopic analyses.Each round of the PT scheme contains two materials that are of interest to forensic scientists. Participants report up to four isotope delta values for each material and the results of the PT scheme are interpreted using z- or z′-scores calculated using a standard deviation of performance assessment (SDPA) that was initially derived from the submitted data, but now have fixed values.This paper presents and discusses results for the FIRMS PT scheme, including inter-laboratory comparability over time not just for individual laboratories but also for particular matrices that have been analysed in multiple rounds. The FIRMS PT scheme has also distributed the same physical material as earlier FIRMS ILEs providing a longer-term view. A number of interesting findings are presented highlighting the value of PT participation together with a discussion of the status of forensic bio-element stable isotope ratio analysis in the light of these outcomes.

Measurement harmonization and traceability for trace element analyses across the Children's Health Exposure Analysis Resource laboratory network

Harmonization and traceability are related metrological principles that are indispensable to assuring measurement comparability across different biomonitoring studies. The Children's Health Exposure Analysis Resource (CHEAR) was established in 2015 with six laboratories providing environmental exposure measurements on biospecimens. To ensure harmonization across studies for trace elements, CHEAR used a multi-faceted approach that included: 1) an initial interlaboratory validation exercise based on the analysis of certified blood and urine reference materials; 2) frequent participation in an established interlaboratory proficiency program for trace elements; and 3) analysis of a common pool of well-characterized biological reference materials with each analytical batch. Method accuracy and precision were established for each laboratory via analysis of NIST SRM 955c Toxic Elements in Caprine Blood, SRM 2668 Toxic Elements in Frozen Human Urine and SRM 3668 Mercury, Perchlorate, and Iodide in Frozen Human Urine. The differences among the six laboratories for As, Cd, Hg, and Mn in urine and Cd, Hg, and Pb in whole blood were judged to be fit-for-purpose. Interlaboratory performance over a 5-year period demonstrated an improvement in performance, such that for 2018–2019, >99% of challenges for urine As, Cd, Hg, and Mn, and 95% for whole blood Cd, Hg, Pb, and Mn, were found to be satisfactory. The CHEAR common reference materials were analyzed by at least 5 laboratories for 22 elements in urine and 13–14 elements in whole blood, thus providing a rich source of data to assess intra- and inter-run performance. The suite of trace elements with assigned values in both blood and urine matrices are more comprehensive than similar reference materials from other sources, and is reflective of the concentrations necessary to support biomonitoring studies. While some areas for future improvement were identified, significant progress was made to improve harmonization of trace element measurements in biological matrices among the CHEAR network labs.

Evaluation of the GENE-UP® Listeria spp. Method for the Detection of Listeria Species in a Variety of Foods and Select Environmental Surfaces: Collaborative Study, First Action 2019.10.

The GENE-UP® Listeria spp. 2 (LIS 2) assay (Performance Tested MethodSM 121803) is a real-time PCR molecular detection method for the rapid detection of Listeria species (Listeria monocytogenes, L. innocua, L. ivanovii, L. seeligeri, and L. welshimeri) in a variety of foods and environmental surfaces. The purpose of this validation was to evaluate the method's interlaboratory performance and submit the results to AOAC INTERNATIONAL for adoption as First Action Official MethodSM for the detection of Listeria species in a variety of foods and select environmental surfaces. The GENE-UP method was evaluated in a multi-laboratory study as part of the AFNOR NF VALIDATION certification process using unpaired test portions for one food matrix, full-cream goat milk cottage cheese (8.4% fat). The candidate method was compared to the ISO 11290-1/Amd.1 reference method. Sixteen participants from 15 laboratories throughout the European Union participated. Three levels of contamination were evaluated: a non-inoculated control level (0 CFU/test portion), a low contamination level (∼2 CFU/test portion), and a high contamination level (∼10 CFU/test portion). Data from that study were analyzed according to the probability of detection (POD) statistical model. The dLPODC values with 95% confidence intervals between the candidate and reference method results were -0.02 (-0.07, 0.03), -0.08 (-0.31, 0.16), and 0.00 (-0.03, 0.03) for the non-inoculated, low, and high contamination levels, respectively. The dLPODC results demonstrate no difference in performance between the candidate method and reference method for the matrix evaluated. Data from a singular collaborative study was used to achieve adoption as an AOAC First Action Official Method for the detection of Listeria species in a variety of foods and select environmental surfaces.

Evaluation of the GENE-UP® Listeria monocytogenes Method for the Detection of Listeria monocytogenes in a Variety of Foods and Select Environmental Surfaces: Collaborative Study, First Action 2019.11.

The GENE-UP® Listeria monocytogenes 2 (LMO 2) assay (Performance Tested MethodSM 121804) uses real-time PCR technology and a proprietary detection platform, the GENE-UP Thermocycler, to detect Listeria monocytogenes in a variety of foods and environmental surfaces. The purpose of this validation was to evaluate the method's interlaboratory performance and submit the result to AOAC INTERNATIONAL for adoption as First Action Official MethodSM for the detection of Listeria monocytogenes in a variety of foods and select environmental surfaces. The GENE-UP method was evaluated in a multi-laboratory study as part of the AFNOR NF VALIDATION certification process using unpaired test portions for one food matrix, full-cream goat milk cottage cheese (8.4% fat). The candidate method was compared to the ISO 11290-1/Amd.1:2004 reference method. Sixteen participants from 15 laboratories throughout the European Union participated. Three levels of contamination were evaluated: a non-inoculated control level (0 CFU/test portion), a low inoculum level (∼2 CFU/test portion), and a high inoculum level (∼10 CFU/test portion). Data from the study were analyzed according to the Probability of Detection (POD) statistical model as presented in the AOAC validation guidelines. The dLPODC values with 95% confidence interval for each comparison were; -0.02 (-0.07, 0.03), -0.08 (-0.31, 0.16), and 0.00 (-0.03, 0.03) for the non-inoculated, low, and high contamination levels, respectively. The dLPODC results demonstrate no difference in performance between the candidate method and reference method for the matrix evaluated. The GENE-UP LMO method demonstrated accuracy and precision in detecting and discerning L. monocytogenes from other Listeria species.