Proficiency Testing Performance Research Articles

Review of Actual Proficiency-Testing Performance Under CLIA '67 (March 14, 1990) Rules: Perspective from the First Year's Data

Under the Clinical Laboratory Improvement Act of 1967 the Health Care Financing Administration's proficiency-testing requirement applies to approximately 12,000 hospital, reference, and large-clinic laboratories in the United States. The Wisconsin State Laboratory of Hygiene is approved by the Health Care Financing Administration to provide proficiency testing in all specialties and subspecialties. The focus of the program is to provide highly specialized service and support to a limited number of participants in order to assess intralaboratory performance correctly. We report the findings over the four proficiency-testing events in 1991 for the subspecialty of routine chemistry, which serves approximately 470 participants. Failure rates for individual analytes on single proficiency testing events ranged from 0% to 13%. After four events or one year, if the mandated evaluation criteria and failure rules were strictly applied, as many as 11% of the laboratories could have found themselves involuntarily suspended from offering all routine chemistry testing.

1990 Medicare/CLIA final rules for proficiency testing: minimum intralaboratory performance characteristics (CV and bias) needed to pass

On March 14, 1990, the Centers for Disease Control and the Health Care Financing Administration published criteria for defining minimum performance in proficiency testing (PT). Using our previously described computer modeling technique, we determined the likelihood of passing PT under the new rules. The model relates combinations of intralaboratory CV and bias to PT performance criteria. For example, a laboratory with a bias of zero and an internal CV of 5% will pass a 10% fixed-limit PT criterion (i.e., the criterion for glucose analyses) 98% of the time when five samples are used. The model provides similar analyses for all PT criteria and all relevant combinations of CV and bias. The probability of passing PT decreases as the number of analytes tested increases, i.e., from 98% to 37% as the number of analytes increases from 1 to 20. A laboratory's internal CV has a greater effect on the outcome of PT than do the corresponding bias values. We conclude that a laboratory that operates with methods that have internal CVs less than or equal to 33% and biases less than or equal to 20% of the PT criteria will have a greater than 99% chance of passing PT.

Quality assurance in North America

Quality insurance (QA) in North America traditionally has been aimed at insuring accuracy and precision of the analytical process in clinical laboratory determinations, Almost all hospital laboratories in the United States and Canada subscribed to programs sponsored by the College of American Pathologists (COP) assessing analytical accuracy and precision, A recent change in these programs includes establishment of CRP programs for physician office laboratories. Proposed regulatory changes for 1990 will require laboratories to demonstrate adequate performance In proficiency testing (PT) programs. Laboratories falling PT will be required to participate in enhanced PT. In 1987 a major change occurred when the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) redefined QA in terms of patient outcomes, thereby establishing activities involving the analytical process as quality control (QC). During on-site inspections, JCAHO representatives require demonstration studies relating laboratory quality and patient care, institutions without these studies have had accreditation withheld. New programs have been established evaluating pathologists’ performance, and assessing the preanalytical and post-analytical aspects of laboratory testing, and linking these with patient outcomes. One of these programs of the CAP, Q-Probes, has over 800 participating institutions for its first subscription year in 1989. Data will be presented from Q Probe Pilot study in 36 hospitals, and subscription program studies already complete.

A regulatory model for clinical laboratories: an empirical evaluation.

Clinical laboratories in the United States are subject to various regulatory and accreditation programs, which mandate a broad range of requirements regarding personnel, quality-control systems, and analytical proficiency standards. Reported here, for a cross-section of U.S. laboratories, is the degree of compliance with these regulatory requirements, some other laboratory characteristics, and their relation to analytical proficiency. The results suggest that those laboratory characteristics that predict highest proficiency-test performance differ for each laboratory specialty. Regression models are presented that explain from 12% to 35% of the variation in analytical performance and suggest that factors outside of those specified in the regulatory model and other characteristics measured in this research are important. Indeed, the current regulatory approach may not ensure highest performance. Also discussed are the current status, limitations, and prospects for change of the clinical laboratory regulatory system.

Use of computer modeling to predict the magnitude of intralaboratory error tolerated by proposed CDC interlaboratory proficiency testing performance criteria.

In December 1987, the Centers for Disease Control (CDC) proposed to the Health Care Financing Administration revised criteria for evaluating participating laboratories' performance in proficiency-testing programs. If these criteria are accepted, they will become the minimum standard for all regulatory proficiency-testing programs. To evaluate a laboratory's performance in a clinical chemistry proficiency-testing program, the CDC proposed a combination of the use of fixed limits, multiples of the interlaboratory group standard deviations, and absolute values. In addition, laboratories would be required to meet 70% of the most recent proficiency-testing challenges. Because the purpose of regulatory proficiency testing is to identify poorly performing laboratories, it is essential that regulators be aware of the relationship between the regulatory criteria and the actual magnitude of intralaboratory error they tolerate. Through computer simulation, we determined for 21 chemistry analytes the amount of intralaboratory error tolerated by the CDC-proposed criteria. We evaluated the effectiveness of the proposed criteria by comparing the levels of total intralaboratory error permitted by a proficiency-testing program by using the CDC criteria with actual currently achievable levels of performance and defined medical usefulness needs. The proposed CDC criteria were too lenient for six analytes, about correct for seven, and too stringent on two; no medical usefulness limits were available for six.

Ability of the 1(2)s rule to detect substandard performance in interlaboratory proficiency testing.

The most common evaluation criterion used by regulatory and voluntary interlaboratory proficiency testing (PT) programs to judge the quality of chemistry results is based on the group mean +/- 2.0 standard deviations (SD), the 1(2)s rule. The mean and SD are those of the selected PT population. Through computer simulations we have quantified, for the first time, the ability of the 1(2)s criterion to detect "acceptable" as well as substandard ("unacceptable") intralaboratory performance by PT. We found that the 1(2)s rule correctly identifies "acceptable" performance--i.e., low intralaboratory coefficient of variation (CV) values and small bias values--greater than 90% of the time. However, this criterion fails to detect laboratories with "unacceptable" bias and (or) CV at least 23% of the time. The high failure rate of the 1(2)s rule raises serious questions about its appropriateness for use by PT programs.

Laboratory performance in proficiency testing.

Laboratory performance in proficiency testing.