Proficiency Testing Data Research Articles

Protein C Assay Performance

To determine the performance and frequency of protein C reagents currently used by clinical laboratories, we analyzed North American Specialized Coagulation Laboratory Association (NASCOLA) protein C proficiency testing data from 6 surveys conducted in 2009 and 2010 (2009-1 to 2009-3 and 2010-1 to 2010-3). Interlaboratory coefficients of variation (CV) for commonly used reagents on a survey with normal protein C ranged from 8% to 12% for antigenic assays, from 4% to 7% for chromogenic activity assays, and from 7% to 22% for clot-based activity assays. CVs for commonly used reagents on specimens with abnormal protein C ranged from 15% to 24% for antigenic, 4% to 11% for chromogenic, and 10% to 17% for clot-based assays (averaged across 3 surveys). Some reagents were used by relatively few laboratories and therefore additional study may be needed for those reagents. For all commonly used reagents, biases were usually small and often not statistically significant. All assessed reagents were clinically accurate, and were considered acceptable options for a specialized coagulation laboratory.

Recommendations for Appropriate Activated Partial Thromboplastin Time Reagent Selection and Utilization

The activated partial thromboplastin time (aPTT) is widely used as a screening coagulation test and for monitoring unfractionated heparin therapy. Various commercial reagents are available, with different performance characteristics, particularly responsiveness to the lupus anticoagulant (LA). Because aPTT reagent selection significantly affects the interpretation of results, we reviewed College of American Pathologists proficiency testing data involving approximately 4,000 coagulation laboratories, and conducted a survey of coagulation laboratories (n = 93) using The Fritsma Factor hemostasis Web site to determine the basis for aPTT reagent selection. The data demonstrate that for routine aPTT testing, most laboratories use reagents with high/moderate responsiveness to LA. Significant misunderstanding was apparent regarding the use of appropriate aPTT reagent for routine testing and LA identification. We recommend aPTT reagents with low LA responsiveness to screen for coagulation factor deficiencies and heparin monitoring, and suggest continued education of laboratory professionals and reagent manufacturers about appropriate aPTT reagent use.

Analysis of the Accuracy and Precision of the Axis-Shield Afinion Hemoglobin A1c Measurement Device

Point-of-care (POC) hemoglobin A1c measurement is now used by many physicians to make more timely decisions on therapy changes. A few studies have highlighted the drawbacks of some POC methods, e.g., poor precision and lot-to-lot variability. Evaluating performance in the clinical setting is difficult because there is minimal proficiency testing data on POC methods. In this issue of Journal of Diabetes Science and Technology, Wood and colleagues describe their experience with the Afinion method in a pediatric clinic network, comparing these results to another POC method as well as to a laboratory high-performance liquid chromatography method. Although they conclude that the Afinion exhibits adequate performance, they do not evaluate lot-to-lot variability. As with laboratory methods, potential assay interferences must also be considered.

Use of characteristic functions derived from proficiency testing data to evaluate measurement uncertainties

Interlaboratory comparisons show that reproducibility standard deviations are dependent on the concentration of the analyte. Many attempts have been made to model this. In this paper, ‘characteristic’ functions are used for modelling the concentration dependence of the reproducibility standard deviations based on data from proficiency tests for water analysis. The characteristics of the resulting functions can be used for the estimation of measurement uncertainties at different concentration levels. These functions are especially useful to determine the concentration levels below which absolute uncertainties tend to be constant and above which the relative uncertainties are more constant. By comparing the characteristic functions of different analytical procedures for the determination of the same analyte, the performance of these procedures under routine application can be compared. Finally, these functions may be used to get an indication on the average quality of analytical result in a specific field to be used by regulators in order to formulate requirements in the legislation that are in accordance with current measurement quality.

Evaluating Frequency, Diagnostic Quality, and Cost of Lyme Borreliosis Testing in Germany: A Retrospective Model Analysis

Background. Data on the economic impact of Lyme borreliosis (LB) on European health care systems is scarce. This project focused on the epidemiology and costs for laboratory testing in LB patients in Germany. Materials and Methods. We performed a sentinel analysis of epidemiological and medicoeconomic data for 2007 and 2008. Data was provided by a German statutory health insurance (DAK) company covering approx. 6.04 million members. In addition, the quality of diagnostic testing for LB in Germany was studied. Results. In 2007 and 2008, the incident diagnosis LB was coded on average for 15,742 out of 6.04 million insured members (0.26%). 20,986 EIAs and 12,558 immunoblots were ordered annually for these patients. For all insured members in the outpatient sector, a total of 174,820 EIAs and 52,280 immunoblots were reimbursed annually to health care providers (cost: 2,600,850€). For Germany, the overall expected cost is estimated at 51,215,105€. However, proficiency testing data questioned test quality and standardization of diagnostic assays used. Conclusion. Findings from this study suggest ongoing issues related to care for LB and may help to improve future LB disease management.

An approach to integrated data assessment in a proficiency test on the enumeration of Escherichia coli

To develop appropriate statistical approaches to plan and evaluate proficiency tests for the enumeration of Escherichia coli, addressing, in particular, a possible but frequently unavoidable lack of test sample homogeneity. Each of 50 laboratories analysed two samples of a stabilized suspension of E. coli in duplicate, using various media, inoculation methods, and incubation times and conditions. In parallel, the E. coli suspension was tested by the organiser for homogeneity and stability. Escherichia coli counts followed a log-normal distribution. After eliminating, by Youden analysis, two data sets that were considered outliers and eight data sets for underperformance of the laboratories (substantial lack of repeatability), the standard deviation of the mean was about 0·06 log(10) units. There was no evidence of bimodality of the data. Lack of homogeneity of distribution of bacteria had a strong effect on measurement uncertainty, in addition to laboratory bias and method repeatability. The homogeneity decreases during storage of the individual test vials; this effect could be modelled by the known kinetics of inactivation of micro-organisms. The results were confirmed by Monte Carlo simulations. By a tailored analysis of proficiency testing data, it is possible to distinguish the effect of lack of homogeneity, laboratory bias and method repeatability, on the measurement uncertainty. A statistic tool is provided to solve problems related to lack of stability of microbiological test material and to separate the effects of sample inhomogeneity from the performance of the individual laboratory.

Proficiency Testing Performance in US Laboratories: Results Reported to the Centers for Medicare & Medicaid Services, 1994 Through 2006

Beginning in 1994, clinical laboratories performing nonwaived testing were required, under the regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA), to enroll and participate in a proficiency testing (PT) program approved by the Centers for Medicare & Medicaid Services. Successful PT performance is a requirement for maintaining CLIA certification to perform testing in certain specialties and subspecialties and for specific analytes. To evaluate the PT performance from 1994 through 2006 of hospital and independent laboratories (HI) compared with all other testing sites (AOT) for selected commonly performed tests and analytes. Proficiency testing data, from 1994 through 2006, were electronically reported to the Centers for Medicare & Medicaid Services by approved PT programs as required by CLIA regulations. Approximately 16 million PT event scores from 36 000 unique testing sites were sorted into 2 groups based on the type of testing facility: HI or AOT. The PT performance scores for 15 of the most commonly performed tests demonstrated a decline in failure rates for both HI and AOT laboratory groups during 1994 through 2006 (analyte/test values reported in this article include alanine aminotransferase, amylase, bilirubin, cholesterol, digoxin, glucose, hemoglobin, leukocyte count, potassium, prothrombin time, theophylline, thyroxine, triglycerides, white blood cell differential, and uric acid). For most analytes, the difference in failure rates between HI and AOT was statistically significant. The AOT group started with higher failure rates, and remained higher for all analytes, during most years when compared with the HI group; although, over time, that difference diminished. The AOT group showed a greater decline in PT failure than the HI group. For all analytes, the AOT group performance improved during this period. The PT performance improved dramatically for the AOT group from 1994 through 2006 as measured by a decrease in the percentage of laboratories with unsatisfactory performance for 15 selected analytes. The PT performance in the HI group improved modestly for some analytes during this same period, whereas, for other analytes, the group showed no apparent improvement.

Proficiency Testing Performance in US Laboratories Results Reported to the Centers for Medicare & Medicaid Services, 1994 Through 2006

Abstract Context.—Beginning in 1994, clinical laboratories performing nonwaived testing were required, under the regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA), to enroll and participate in a proficiency testing (PT) program approved by the Centers for Medicare & Medicaid Services. Successful PT performance is a requirement for maintaining CLIA certification to perform testing in certain specialties and subspecialties and for specific analytes. Objective.—To evaluate the PT performance from 1994 through 2006 of hospital and independent laboratories (HI) compared with all other testing sites (AOT) for selected commonly performed tests and analytes. Design.—Proficiency testing data, from 1994 through 2006, were electronically reported to the Centers for Medicare & Medicaid Services by approved PT programs as required by CLIA regulations. Approximately 16 million PT event scores from 36 000 unique testing sites were sorted into 2 groups based on the type of testing fac...

PATHOGEN DETECTION IN FOOD MICROBIOLOGY LABORATORIES: AN ANALYSIS OF QUALITATIVE PROFICIENCY TEST DATA, 1999–2007

ABSTRACT The objective of this study was to assess laboratories' ability to detect or rule out the presence of four common food pathogens: Escherichia coli O157:H7, Salmonella spp., Listeria monocytogenes and Campylobacter spp. To do this, qualitative proficiency test data provided by one proficiency test provider from 1999 to 2007 were examined. The annual and cumulative 9‐year percentages of false‐negative and false‐positive responses were calculated. The cumulative 9‐year false‐negative rates were 7.8% for E. coli O157:H7, 5.9% for Salmonella spp., 7.2% for L. monocytogenes and 13.6% for Campylobacter spp. Atypical strains and low concentrations of bacteria were more likely to be missed, and the data showed no trend of improving performance over time. Percentages of false‐positive results were below 5.0% for all four pathogens.PRACTICAL APPLICATIONSThe results imply that food testing laboratories often fail to detect the presence of these four food pathogens in real food specimens. To improve pathogen detection, supervisors should ensure that testing personnel are adequately trained, that recommended procedures are followed correctly, that samples are properly prepared, that proper conditions (temperature, atmosphere and incubation time) are maintained for good bacterial growth and that recommended quality control procedures are followed. Supervisors should also always investigate reasons for unsatisfactory proficiency test results and take corrective action. Finally, more research is needed into testing practices and proficiency test performance in food testing laboratories.

Validation of an environmental friendly segmented flow method for the determination of phenol index in waters as alternative to the conventional one

Phenolic compounds are a sort of common pollutants in water. Phenol index becomes an expedite indicator for the evaluation of the contamination level of water samples, in spite of the knowledge of the individual phenol and its derivatives are also important.In this work, an environmental friendly method for the determination of phenols, using a segmented flow system based on the conventional method's reactions without the liquid–liquid extraction step, was validated.Three linear dynamic ranges using C6H5OH: 1–10μgl−1, 10–200μgl−1 and 0.2–2.5mgl−1, with a coefficient of variation lower than 2%, were obtained. Several method's performance parameters were determined: limits of detection, limits of quantification, precision through duplicate analysis and trueness using the reference materials purchased from LGC Promochem, RTC no. QCI-043-2 Lot:P1. Measurement uncertainty was evaluated using an interlaboratory approach based on proficiency testing data. Relative combined uncertainty for phenols in water samples, ucrel(γsample), of 0.054 were obtained, in according to those imposed by the Portuguese Legislation: target ucrel(γsample)=0.58 for 1μgl−1 of phenol (surface waters) and target ucrel(γsample)=0.06 for 500μgl−1 of phenol (wastewaters).A high efficiency reduction and elimination of reagents and wastes, reduction of analysis time and exposition of the analyst were also obtained.

Local Laboratory Reference Intervals in Clinical Trials

Laboratory data are of major importance indrug safety assessments since they are consid-ered objective tools to supplement clinical judg-ments and the transfer of results across studypopulations. Moreover, the spectrum of avail-able biomarkers to assess drug safety and effica-cy is continuously being developed and refined.Although central laboratories are widely usedin clinical trials, local laboratories continue toplay an important role for trials where the use ofsuch central laboratories is not practical or fea-sible. Local laboratories are primarily perform-ing safety testing. The collection and analysis ofthese data remain a challenge in view of differ-ences in reference intervals, analytical methods,and reporting units. These challenges are wide-ly recognized. Other factors that may accountfor differences between local and central labora-tories are differences in population characteris-tics, such as ethnicity or nutrition. While suchdifferences should be reflected by differencesbetween local and central laboratory referenceintervals, analysis of central laboratory data inglobal clinical studies has shown these differ-ences to be insignificant (see next section). Al-though units are easily standardized by mathe-matical conversion, differences in methods andassociated reference intervals continue to raisequestions for statisticians. Chuang-Stein hasproposed a normalization procedure based onthe results of proficiency surveys by devising analgorithm to adjust results by comparing thembetween different laboratories and making theadjusted results comparable across laboratories(1). While such a procedure could indeedimprove result comparability, its feasibility islimited by, inter alia, the difficulties of gainingregular access to proficiency testing data, inter-pretation of the different schemes, especially inglobal studies, and the degree of participationin those schemes by the involved local laborato-ries. Thompson et al. have used the normal re-sult distribution of study populations as the basis for defining a commonly acceptable refer-ence interval (2). However, this proposal is limit-ed by the scope of the observations, which maynot necessarily cover existing differences inmethods in global studies, for example, the well-documented differences for commonly testedanalytes such as alkaline phosphatase or lactatedehydrogenase. Ruvuna et al. have proposed theapplication of generalized lab norms from a“phantom laboratory” by compiling a standard

Proficiency testing for measurement of radon (222Rn) in drinking water

The Finnish Environment Institute in collaboration with the Radiation and nuclear safety authority (STUK) carried out the proficiency test for measurement of radon (222Rn) in water. Samples were taken from two drilled wells in November 2007. STUK has supplied the regional laboratories with RADEK MKGB-01 equipment based on gamma spectrometry. Two samples for this PT were taken from two drilled wells. Ground water moves irregularly in the process and cracks of the bedrock which is why each participant received an individual sample. Each participant’s sample was measured also by STUK using liquid scintillation counter (LSC) as the reference method in this proficiency test. In estimating laboratory performance the results that deviated less than ±10% from the value measured by STUK using LSC were regarded satisfactory. In total, 73% of the results in the analysis of the sample R1 and 82% in the analysis of the sample R2 deviated less than 10% from the values measured by STUK. The results reported by the participants were generally smaller than the results measured by STUK. The deviation between each participant’s result and the result measured by STUK with the LSC was −7.4% (Sample R1) and −6.2% (Sample R2). Due to the lack of certified reference materials and a reliable proficiency testing data, it is impossible to check the traceability of radon measurements by using the reference method (LSC) at this moment.

Quality Control Validation, Application of Sigma Metrics, and Performance Comparison between Two Biochemistry Analyzers in a Commercial Veterinary Laboratory

A review of the literature pertinent to interpretation of biochemistry data and quality control (QC) and proficiency testing data from 2 biochemistry analyzers was used to determine clinical quality requirements for biochemistry assays, characterize the performance of and calculate sigma metrics for the analytes run on the 2 analyzers, and perform QC validation in order to determine the needs for statistical QC for each analyzer. Quality requirements suitable for the analytes based on the needs of the authors' laboratory are presented. These requirements may or may not be appropriate for other laboratories, depending on the needs of the clients, species, and equipment performance capability. The majority of the analytes were easily controlled using the 1(3s) control rule, with a sigma metric approaching or exceeding 6 and with a high probability of error detection and a low probability of false rejection. Some analytes could not be controlled using the 1(3s) rule, and additional control rules with a greater number of control data points were required. There were differences between performances of the 2 analyzers. The findings in the present study emphasize the need for QC specific for the analyte and the clinical decision level and the need for separate QC validation on every instrument.

Trends in sperm morphology as assessed by strict or world health organization (WHO) 3rd edition criteria over eleven years of american association of Bioanalysts proficiency testing data

OBJECTIVE: The aim of this analysis was to evaluate and interpret laboratory variability and trends in strict compared to WHO 3rd morphology over time. DESIGN: Retrospective analysis of multi-center testing data. MATERIALS AND METHODS: The American Association of Bioanalysts (AAB) provides proficiency testing (PT) services to ensure quality control in andrology laboratories throughout the United States. In their PT program, the AAB provided two unstained slides per year with instructions to perform sperm morphology using routine methodology. Laboratories reported percent normal morphology, methodology used for evaluation, and type of stain for quality assessment. Analysis was performed on results collected from participating laboratories between 1997 and 2007. Of the participants, 66.5% and 20.9% reported using strict and WHO 3rd morphology, respectively, with Diff-Quik® utilized as the most common staining method (46.0%). Inclusion criteria for this study were a minimum of 10 or more proficiency testing events per lab and Diff-Quik® staining. Nine hundred-and-one laboratories with a total of 11455 PT challenges fulfilled these criteria. Results were analyzed using generalized linear mixed models with a gamma distribution, a log link graph, and random laboratory effects to determine if morphology analysis varied significantly over time and among testing centers. RESULTS: Morphology for both methods decreased significantly over time. There was a more significant decrease in predicted morphology over time for strict compared to WHO 3rd morphology (p < 0.0001).Table 1Changes in Morphology Assessment Between 1997 and 2007Strict Predicted Median (IQR)WHO 3rd Predicted Median (IQR)199710.2 (9.3, 11.4)33.2 (30.2, 34.3)20077.5 (6.7, 8.6)29.0 (26.3, 34.8) Open table in a new tab In addition to the overall trend of a decrease in strict values, variation in labs was significant. 25% and 19% of labs reported values that were significantly lower than the group average for strict and WHO 3rd morphology, respectively. CONCLUSIONS: Strict morphology is an important tool utilized to determine application of ICSI when infertile couples are undergoing IVF. For this reason, insuring accurate results of strict morphology is essential. These results indicate strict morphology analysis is low for at least a quarter of the laboratories participating in the AAB PT and that morphology has decreased more in laboratories performing strict assessment than those performing WHO 3rd.

Role of respiratory pathogens in infants hospitalized for a first episode of wheezing and their impact on recurrences

In order to evaluate the infectious agents associated with the first episode of severe acute wheezing in otherwise healthy infants and to define the role of each of them in recurrences, 85 patients in Italy, aged <12 months, hospitalized because of a first acute episode of wheezing, were prospectively enrolled between 1 October 2005 and 31 March 2006. Upon enrolment, nasopharyngeal swabs were collected for the real‐time PCR detection of respiratory syncytial virus (RSV) types A and B, influenza virus types A and B, adenovirus, parainfluenza viruses types 1, 2, 3 and 4, rhinovirus, human metapneumovirus, human coronavirus types 229E, OC43, NL63, and HKU1, bocavirus, enterovirus, and paraechovirus; nasopharyngeal aspirates were also obtained to detect atypical bacteria. At least one infectious agent was identified in 76 children (89.4%). RSV was the most frequently detected pathogen and its prevalence was significantly higher than that of the other pathogens in both age groups, and significantly higher in the children aged 3–12 months than in those aged <3 months. Only the children with RSV infection experienced recurrent wheezing. Viral load was significantly higher in children with than in those without recurrent wheezing. This study shows that RSV is the main reason for hospitalization during the first wheezing episode in infants, and that it appears to be the only pathogen associated with a high frequency of recurrences. A high viral load seems to be strictly related to the likelihood of recurrence.

Comparison of different statistical methods for evaluation of proficiency test data

In analytical chemistry, proficiency testing usually consists in tests that laboratories conduct under routine conditions and report the result to the PT provider who then converts the result to a score which helps the participant to assess the accuracy of the result. The aim of this work is to show PT providers, accreditations bodies, and participating laboratories that different scoring results can be achieved depending on the evaluation system selected. The influence of different evaluation techniques on the results of an interlaboratory comparison for determination of gold in precious metals alloys was investigated. Results from 19 participating laboratories were evaluated by means of the three procedures: (1) classical statistical approach—outliers detection; (2) robust methods—(2A) robust procedure and (2B) ISO 13528; and (3) fitness for purpose. Evaluation of the same PT data revealed very interesting issues depending on the different scoring systems that were used and the robustness of the statistical methods used for detecting outliers. As a general rule, laboratories with scoring Z > 2 offered clearly poorer performance in robust approaches than classical ones. In order to support this first evidence, we evaluated a second data set with results from 24 laboratories (mercury from soil samples) by means of the four mentioned approaches. Selection and comparison of different scoring systems must be done very carefully, because sometimes they are not the best approach for studying the data population or the more appropriate one for evaluating the distribution of the data. Finally it should be taken into account that sometimes the robust scoring systems are not always suitable for evaluating the results of some PT schemes.

Performance of uncertainty evaluation strategies in a food proficiency scheme

A study of the performance of different uncertainty evaluation strategies among 163 voluntary respondents from food proficiency schemes is presented. Strategies included use of: single-laboratory validation data, quality control data, past proficiency testing data, reproducibility data, a measurement equation and the dispersion of replicate observations on the test material. Most performed reasonably well, but the dispersion of replicate observations underestimated uncertainty by a factor of approximately 3. Intended compliance with accreditation requirements was associated with significantly improved uncertainty evaluation performance, while intended compliance with the ISO “Guide to the expression of uncertainty in measurement” had no significant effect. Substituting estimates based on the Horwitz or Horwitz–Thompson models or on PT target standard deviation for the respondents’ own estimates of uncertainty led to a marked reduction in poor zeta scores and significant improvement in dispersion of zeta scores.

Approaching target uncertainty in proficiency testing schemes: experience in the field of water measurement

Good correspondence between evaluations of the proficiency testing data for water measurements according to different scoring schemes was found as a result of implementation of the measurement quality according to the target uncertainty defined in the Drinking Water Directive. The accuracy of the evaluation is determined by the uncertainty of the assigned value, and this uncertainty should be in correspondence with the target uncertainty. Modification of the zeta-score by the thorough application of the target uncertainty is presented. It is convenient for generalised presentation of the proficiency testing data. The allowed bias is included as a linear term on the measured value scale, as it is treated in uncertainty analysis. The importance of implementation of the target measurement uncertainty in other fields of the routine measurements is indicated.

Estimates of measurement uncertainty from proficiency testing schemes, internal laboratory quality monitoring and during routine enforcement examination of foods

To determine the levels of measurement uncertainty (MU) obtained in proficiency testing and routine microbiological analyses of foods and to compare these with estimates of MU obtained for results of analyses obtained in collaborative interlaboratory studies of microbiological methods. Raw data submitted by participants in the Food Examination Proficiency Assessment Scheme were obtained from the Central Science Laboratory (York). Internal quality monitoring data were obtained from Health Protection Agency (HPA) laboratories, together with data from routine food examinations undertaken in HPA laboratories. The data sets were analysed to determine the relative standard deviations of reproducibility (RSD(R)), based on log(10) colony count values, and thence the relative measures of expanded uncertainty. Analysis of proficiency test data showed extreme values of RSD(R) up to +/-30% depending upon the organism, the laboratory and the method of examination. RSD(R) values on routine samples averaged around +/-12% but ranged up to +/-41% in a few instances. Internal quality assessments for different organisms ranged up to +/-27%, depending upon the particular organism and examination procedure. The results show little difference in uncertainty for counts obtained using different plating systems (e.g. pour plates, spread plates or spiral plating) on the same dilutions of the same food samples. The data are compared with estimates of microbiological uncertainty derived in interlaboratory studies. The estimates of uncertainty ranged widely, both within and between laboratories, and appeared to bear little relationship to the foodstuff under examination. The extent of MU associated with many routine microbiological examinations is generally no worse than those produced in inter-laboratory trials, although notable exceptions were seen. Knowledge of the levels of MU may have wide impact on the establishment of international standard methods for microbiological examination of foods and the ability to set realistic microbiological criteria.

Implications of ABO Error Rates in Proficiency Testing for Solid Organ Transplantation

Implications of ABO Error Rates in Proficiency Testing for Solid Organ Transplantation