Laboratory Performance Evaluation Research Articles

Implementation of a companion diagnostic in the clinical laboratory: The BRAF example in melanoma

A companion diagnostic test provides information that is essential for the safe and effective use of a corresponding therapeutic product as indicated in the drug instructions. The implementation of a companion diagnostic follows the rules of a molecular test for somatic mutations in a routine clinical laboratory environment and needs guidance on practical aspects, including the choice of the proper analytical method and the procedures for internal and external quality controls. Selection of the appropriate assay for detection of genetic alterations depends on several factors: the type of mutation under study, the sample to be assayed and its preparation procedure. In addition, the results of a molecular assay require a complex interpretation process of the analytical data as the patient's genotype, the translation of the identified variant into a predicted phenotype and knowledge on restrictions of the method used. In relation to these aspects herein we report an opinion paper of the Working Group Personalized Laboratory Medicine jointly constituted by the European Federation of Laboratory Medicine (EFLM) and by the European Society of Pharmacogenomics and Theranostics (ESPT) using, as an example, the BRAF genotype analysis in tumor tissue samples for identification of melanoma patients that can benefit treatment with BRAF inhibitors. The manuscript is focused on the following aspects: i) medical rationale, ii) methodologies of analysis, iii) laboratory performance evaluation and iv) the laboratory specific report for the clinicians. The critical evaluation of these aspects would be useful for the implementation of a companion diagnostic in the clinical laboratory.

Challenges of the Unified Health System: present status of public laboratory services in 31 cities of Minas Gerais, Brazil

Introduction: Modifications in the Brazilian Unified Health System (SUS) have led to a significant improvement in the national health indexes. However, some challenges still need to be faced, especially concerning SUS patients' access to high-quality laboratory support services. Objective: To evaluate the present status of laboratory services in SUS in 31 cities of Minas Gerais, Brazil, between 2008 and 2011. Material and method: This analysis was performed through data from the Information Technology Department of SUS (DATASUS) and through interviews with local public health managers with structured questionnaires. Results: Among all the studied cities, 21 had their own laboratory, 90.2% of which were in precarious conditions, not meeting the requirements established by the legislation in force, and employing inappropriate procedures and techniques, in addition to using obsolete equipment. The range of available laboratory tests was limited, what demanded the services of supporting laboratories. None of the evaluated laboratories developed any systematic activity on quality management, including control of analytical quality, maintenance of laboratory equipment, calibration and performance evaluation of critical equipment, continuing education programs, and safety and biosecurity. Conclusion: The effective role of laboratory test results in medical decision is unquestionably impaired, risking the safety of SUS patients. The present work reveals the deficiencies of public laboratory services in Minas Gerais, and proposes a new management model, which is able to associate operational quality, technological development and optimization of human and material resources with higher productivity.

First proficiency testing of second-line anti-tuberculosis drug susceptibility testing in 12 provinces of China

To evaluate the performance of drug susceptibility testing (DST) against the main second-line (SL) anti-tuberculosis drugs in tuberculosis (TB) laboratories in China. The supranational TB reference laboratory issued 30 Mycobacterium tuberculosis isolates to the participating laboratories. Each participating laboratory performed DST against kanamycin (KM), amikacin (AMK), capreomycin (CPM) and ofloxacin (OFX) using the proportion method in Löwenstein-Jensen medium per World Health Organization recommendations. Reported results were checked and compared with the judicial results. The main performance indicators for the four anti-tuberculosis drugs evaluated (KM, AMK, CPM, OFX) were as follows: accordance rates: 91.62%, 99.16%, 96.93% and 96.37%; reproducibility: 99.16%, 99.16%, 94.96% and 94.12%; specificity: 99.12%, 99.64%, 98.00% and 98.41%; sensitivity: 78.03%, 97.62%, 94.44% and 91.51%. The accordance rates and sensitivity values of the four drugs showed statistically significant differences, while specificities showed no significant differences. Eight (66.7%) participating laboratories met the set requirement criteria; however, DST in four (33.3%) laboratories requires greater attention. Of the four drugs tested, the results for KM were lower than those for the other drugs. External quality assessment can lead to effective evaluation of laboratory performance in SL DST.

A Study on Laboratory Testing and Performance Evaluation Method for the Development of an Anti-dew Road Sign

Condensation is a state change when the temperature drops below the dew point, and the moisture contained within the atmosphere forms dew on surfaces. This type of condensation caused problems on roads and road facilities. Especially, if vapors occur in the form of dew on road signs, problems in retro-reflectivity, visibility, and legibility of road signs may occur, resulting in difficulty in providing information. In order to make improvements for such problems that arise as condensation forms on road signs, dew-resistance films, illuminated road signs, and heating cables are currently being used. However there is no test method to analyze the effectiveness of anti-dew road sign method. Therefore in this research, an experiment method was developed to simulate real-world conditions and verified the performance of an anti-dew road sign using thermohygrostats in the laboratory. Comparisons were also conducted between the value of theoretic concepts and the value of the research.

Are Forensic Science Services Club Goods? An Analysis of the Optimal Forensic Science Service Delivery Model

Abstract Forensic science has been described as a public good by practitioners, legal professionals, and scholars, many of whom were suggesting that forensic science is simply something good for the public. It would indeed be difficult to argue otherwise. In an economic sense, the concept of a public good is defined differently from this colloquial meaning, however, leading to confusion in discussions between forensic scientists and business consultants concerning how to evaluate laboratory performance and ultimately consider strategic change from an economic or efficiency perspective. This article discusses what economists mean by a public or private good, with an application using the forensic science industry. Forensic science is likely neither a purely public or purely private good, but rather a club good that contains a degree of both the public and private. When calculated, the degree of publicness of this club good will aid in determining the appropriate institutional framework from which to provid...

Comparison of different approaches to evaluate External Quality Assessment Data

In EQA programs, Z-scores are used to evaluate laboratory performance. They should indicate poorly performing laboratories, regardless of the presence of outliers. For this, two different types of approaches exist. The first type are “outlier-based” approaches, which first exclude outlying values, calculate the average and standard deviation on the remaining data and obtain Z-scores for all values (e.g., Grubbs and Dixon). The second type includes the “robust” approaches (e.g., Tukey and Qn or the algorithm recommended by ISO). The different approaches were assessed by randomly generated samples from the Normal and Student t distributions. Part of the sample data were contaminated with outliers. The number of false and true outliers was recorded and subsequently, Positive and Negative Predictive Values were derived. Also, the sampling mean and variability were calculated for location and scale estimators. The various approaches performed similarly for sample sizes above 10 and when outliers were at good distance from the centre. For smaller sample sizes and closer outliers, however, the approaches performed quite differently. Tukey's method was characterised by a high true and a high false outlier rate, while the ISO and Qn approaches demonstrated weak performance. Grubbs test yielded overall the best results.

Critical evaluation of laboratory performance from IAEA-452 (fish scallop) inter-comparison exercise results

The International Atomic Energy Agency Marine Environmental Laboratory (IAEA-MEL) conducted an intercomparison exercise for the “Trace elements and methyl mercury in fish scallop: IAEA-452” in 2009. The Neutron Activation Analysis (NAA) laboratory at PINSTECH, Pakistan took part in this exercise. Comparison with the IAEA results showed that of the 18 elements reported to the IAEA 7 had unacceptable z-scores while the data for two elements was questionable (2 3.

Evaluation of laboratory performance using proficiency test exercise results

The IAEA conducted the IAEA-CU-2006-06 Proficiency Test (PT) on “The determination of major, minor and trace elements in ancient Chinese ceramic” in 2006. The results of this PT showed that of the 21 analytes reported by our laboratory 9 failed the precision criteria. Therefore the results reported by our laboratory along with the results of other laboratories which carried out analysis using neutron activation analysis (NAA) were studied. It was found that the major factor contributing towards data falling in the “Warning” category, failing the precision criteria was the high uncertainties cited in the certificates of the reference materials (RMs) used for quantification of data. In this regard, it is recommended that synthetic standards should be prepared and used on a routine basis especially for the measurement of the elements K, Eu, Lu, Ta, Tb and Yb.

Harmonized Testing for BCR-ABL Kinase Domain Mutations In CML: Results of a Survey and First Control Round within 28 National Reference Laboratories In Europe

AbstractAbstract 894Various techniques have been employed to detect BCR-ABL kinase domain mutations in patients with chronic myeloid leukemia (CML) resistant to imatinib or second generation tyrosine kinase inhibitors. This variation may at least partially explain the different frequencies of mutations that have been reported. Furthermore, the pattern of individual mutations reported seems to depend on the specific method used for mutation detection. Standardized techniques and protocols for the detection of BCR-ABL mutations will be necessary to obtain comparable mutation results within clinical studies. The first objective of this study conducted within the EUTOS (European Treatment and Outcome Study) for CML program was to record the mutation analysis techniques and protocols that are used for routine diagnostics by 28 national reference laboratories in 23 European countries, 9 of whom perform regular mutation analyses as a central laboratory for national or international clinical trials. The second objective was to evaluate the techniques by analysis of blinded samples containing various BCR-ABL kinase domain mutations. Initially a web-based survey was conducted with a total of 39 technical and PCR-specific questions. Most laboratories (n=17; 61%) perform 1–10 mutation tests per month (range <1 to >100) and use published protocols for nested RT-PCR from peripheral blood or bone marrow leukocytes. Failure to amplify BCR-ABL mRNA at low BCR-ABL transcript levels was reported to be the most common PCR related difficulty in 16 (57%) laboratories. Sanger sequencing is applied for routine BCR-ABL mutation analysis in 27 (96%) laboratories. Additional screening techniques (e.g. denaturing high-performance liquid chromatography, high-resolution melting) or more sensitive detection methods (e.g. allele-specific oligonucleotide PCR, pyrosequencing) are used routinely by eight laboratories each, respectively. The application of positive or negative mutation control samples was reported by 13 (46%) or 17 (61%) laboratories, respectively. Quantitative mutation analysis is performed routinely in nine (32%) laboratories. To evaluate laboratory performance we prepared a series of control samples that were distributed in a blinded fashion to testing laboratories. Seventeen Ba/F3BCR-ABL cell lines harboring various BCR-ABL kinase domain mutations were mixed with non-mutated Ba/F3BCR-ABL to produce dilutions ranging from 1% to 100% of mutant alleles. Three samples were non-mutated Ba/F3BCR-ABL only and were used as negative controls. Mutated and non-mutated Ba/F3BCR-ABL cell lines were diluted into HL60 cells to simulate a BCR-ABL level of 10% on the International Scale. Twenty blinded cDNA samples were sent out on dry ice to each participating laboratory (total of 560 samples). Twenty-three labs have reported their results so far (460 samples). The three non-mutated samples were identified correctly in 68/69 (99%) tests. For the 20 mutated samples we distinguished those with high (≥20%; n=11) and low (≤10%; n=9) levels of mutation. For the high level samples 218/253 (86%) tests were reported correctly. Of the incorrect results, 22 were scored as negative and 13 were false positives (T315I, n=7; D276G, K357E, D363Y, N374I, S386S, E505K, n=1 each). For the low level samples only 15/138 (11%) tests correctly identified the mutation. Most (122/138; 88%) were reported as undetectable and a false positive result (L323H) was reported in one case. We conclude that Sanger sequencing is the most frequently applied technique for routine analysis of BCR-ABL kinase domain mutations in CML in Europe. In general it reliably identifies mutations when the proportion of mutant alleles comprise 20% or more. Nevertheless, false negative and false positive results were reported in a substantial proportion of samples with ≥20% mutation level (35/253, 14%). For mutations that are present at 10% or less mutant alleles, routine methods mainly failed to identify mutations. This study provides a basis for further comparisons and standardization efforts comparable with the introduction of the international scale for quantification of BCR-ABL transcripts. Disclosures:No relevant conflicts of interest to declare.

Laboratory characterization and performance of the high-order adaptive optics system for the Large Binocular Telescope

We present the laboratory characterization and performance evaluation of the First Light Adaptive Optics (FLAO), the natural guide star adaptive optics system for the Large Binocular Telescope (LBT). The system uses an adaptive secondary mirror with 672 actuators and a pyramid wavefront sensor with adjustable sampling of the telescope pupil from 30×30 down to 7×7 subapertures. The system was fully assembled in the Arcetri Observatory laboratories and passed its acceptance test in December 2009. The performance measured at different star magnitudes during the acceptance test is shown to lie between the baseline and the goal specifications. In particular, FLAO obtained a 77% Strehl ratio (SR) in the bright end (8.5 magnitude star in the R band) using an H-band filter and correcting 495 modes with 30×30 subaperture sampling. In the faint end (16.4 magnitude), a 5% SR correcting 36 modes with 7×7 subapertures was measured. The seeing conditions for these tests were 0.8 arcsec (r0=0.14 m at 550 nm) and an average wind speed of 15 m/s. The results for other seeing conditions up to 1.5 arcsec are also presented. The system has been shipped to the LBT site in Arizona, and its commissioning started in March 2010.

Australian national residue survey - closing the loop on pesticide residue risk management for Australian grain.

Australia exports a major proportion of its agricultural production and is highly dependent on maintaining and developing access to, and competitiveness in, export markets. To preserve Australia’s status as a provider of high quality grain, the majority of Australian primary producers rely on pesticides to protect their crops from pests and diseases, particularly in post-harvest situations. The Australian Pesticides and Veterinary Medicines Authority (APVMA) supports Australian agriculture by registering and allowing the supply of safe and effective animal health and crop protection products. A residue risk management continuum is established when the effectiveness of chemical registration and control of chemical use regulations is assessed through residue monitoring programs. Programs assess good agricultural practice and provide traceback capacity to investigate areas of concern. Risk communication provides opportunities for continuous improvement. In the early 1960s, the Australian Government established a non-regulatory body, the national residue survey (NRS). In 2008-2009, random monitoring programs were conducted for over 50 commodities (21 grains, five horticultural commodities, 11 fish species, 12 animal species, honey and egg with over 20,000 samples collected for analytical testing. The NRS grain residue monitoring program is presented as a case study of the residue risk management continuum demonstrating to overseas markets the high level of residue integrity of Australian grain. Over 4,000 grain samples are collected and analysed per annum. Most of the samples are collected in the bulk export program where samples are collected from every hatch of every ship loaded at the seventeen Australian grain export terminals. The chemical screens have expanded beyond the multi-residue screen (MRS) insecticides, fungicides and herbicides, to include phosphine, additional herbicides (not included in the MRS), heavy metals and mycotoxins. In its current form, the NRS grains program provides 15 years of residue testing data which demonstrates a very high degree of conformance with Australian MRLs and the import tolerances of overseas trading partners. In addition, trends in residue testing data demonstrate a decline in the frequency of residue detections and the levels of residue detected. To be confident that residue testing results meet the requisite standards, the reliability of the Australian analyses must be assured. The NRS laboratory performance evaluation system has been developed to provide that assurance, using a range of proficiency tests and other techniques in the selection of laboratories for NRS work. Residue testing results are reported against both Australian MRLs and the international MRLs which apply in the relevant export market. NRS maintains databases of overseas MRLs and compares its residue testing results of exported commodities against those standards. Grain marketers receive certificates of analysis, in the form of NRS residue testing results, for each shipment prior to arrival at the overseas market. Keywords : Grain, Pesticide, residue, monitoring program, residue risk management

Evaluating Laboratory Performance With Quality Indicators

Objective: Quality indicators in the clinical laboratory provide a useful tool for continuous improvement of laboratory services. The goal of this study was to design and review quality indicators from time to time in an effort to improve the performance of the laboratory. Methods: The study included an evaluation of different quality indicators collected from the biochemistry laboratory of a tertiary care hospital in New Delhi over a period of 1 year. A total of 84250 samples were processed, and the quality indicators monitored were classified into the following 3 categories: preanalytical, analytical, and post-analytical. Results: Hemolysis (0.7%) was the most common anomaly observed during the assessment of pre-analytical indicators followed by improper samples for coagulation profile (0.34%). In the analytical phase, the frequency for fibrin clots was 1.2%. Conclusion: Working constantly to improve the outcome of these indicators by taking corrective measures over a period of time will definitely help to improve the quality of laboratory services and patient health care. * IOM : Institute of Medicine GBPH : Govind Ballabh Pant Hospital OPD : Out Patient Department QC : Quality Control TAT : turnaround time LIS : laboratory information system

Second interlaboratory exercise on non-steroidal anti-inflammatory drug analysis in environmental aqueous samples

Several interlaboratory exercises were organised within the framework of European FP6 project NORMAN. Among others, non-steroidal anti-inflammatory drugs were investigated in different aqueous samples in two sequential ring studies. The aim of both studies was to evaluate the state-of-art in Europe and to determine possible sources of variation, while also attempting to diminish them. In the present paper we discuss the results of the 2nd Interlaboratory study, while the results of 1st round were presented before. The main scope of the 1st exercise organised within NORMAN project was to assess the laboratory proficiency regardless of the analytical method applied, to evaluate the stability of the target compounds during sample storage, and to define possible sources of variation during sample shipment, storage and analysis. In the 2nd round we primarily aimed to diminish these sources of variation by applying two predetermined analytical protocols based on liquid chromatography-mass spectrometry or gas chromatography-mass spectrometry. The two analytical protocols were compared in terms of their ability to determine individual analytes in matrices of different complexity, i.e. tap water, river water and wastewater. Furthermore, the 2nd exercise addressed also the filtration and compared the influence of different filter material categories on the analysis of non-steroidal anti-inflammatory drugs. Results presented herein evaluate laboratory performance using z-score, bias, proximity and Youden plots. Overall, the laboratory performances were found to be satisfactory for determining NSAIDs in aqueous samples. The two analytical protocols, LC-MS and GC-MS, are assessed according to their sensitivity and measurement uncertainty, where the GC-MS proved superior for the analysis of Ibuprofen, Ketoprofen and Naproxen in matrices with higher complexity. Finally, neither the filtration itself, nor the filter materials were shown to significantly affect the determination of NSAIDs.

Evaluation of durability tests for permeable friction course mixtures

Durability of porous or permeable friction courses (PFC) is an important aspect to address when designing this type of hot mix asphalt. At present, several agencies perform the mix design of PFC primarily by determining volumetric mixture properties. This approach ensures adequate mixture functionality, but it does not guarantee mixture durability. This paper evaluates the Cantabro loss test, the Hamburg Wheel-Tracking test (HWTT) and the Overlay test (OT) to determine the one most appropriate for mix design and laboratory performance evaluations. The Cantabro loss test, performed in both dry and wet conditions, is recommended for PFC mix design to corroborate the suitability of the optimum asphalt content defined based on volumetric determinations. The HWTT and the OT are not recommended, since the variability of the test results indicated that these tests may not be suitable for PFC mixtures.

Lessons from Multicenter Studies on CSF Biomarkers for Alzheimer's Disease

Several single-center studies have confirmed the usability of cerebrospinal fluid (CSF) biomarkers for the diagnosis of Alzheimer's disease (AD), even in early disease stages. Large scale multicenter studies have principally confirmed this, although such studies have also indicated the presence of significant intercenter and interlaboratory variations in biomarker measurements. Such variations may hamper the development of biomarkers and their introduction into clinical routine practice. Recently a quality control program run by the Alzheimer's Association was started in order to harmonize procedures of laboratories world-wide. This program provides both standardized guide lines and external control CSF samples, and will allow longitudinal evaluation of laboratory performance.

Laboratory Performance Evaluation of Subgrade Soils Stabilized with Sulfate-Bearing Cementitious Additives

AbstractA laboratory study was conducted on both raw and stabilized soil specimens to evaluate the performance of two subgrade soils stabilized with three different additives, namely, hydrated lime (or lime), class C fly ash (CFA), and cement kiln dust (CKD). Lime, CFA, and CKD have low sulfate (&amp;lt;40 ppm), moderate sulfate (≈3,280 ppm), and high sulfate content (≈28,133 ppm), respectively. Cylindrical specimens were prepared with different percentages of additives and cured for 28 days. Following the curing period, specimens were tested for resilient modulus (Mr), and unconfined compressive strength tests (UCS), representing short-term performance. Long-term laboratory performance (or durability) was evaluated in terms of moisture susceptibility (tube suction test) and three-dimensional (3D) swell during 60-day capillary soaking of cured specimens. At the end of 60 days, the specimens were also tested for Mr and UCS, as additional indicators to long-term subgrade performance. Short-term performance evaluation projected 15 % CKD with the highest improvement after 28 days of curing, while long-term performance evaluation showed 9 % lime providing maximum enhancements in Mr and UCS values after 60 days of capillary soaking. The study further indicated decrease in durability of specimens stabilized with CKD; however, lime and CFA helped by reducing 3D swell and moisture susceptibility. This difference in behavior was attributed to the presence of elevated levels of sulfate in CKD resulting in sulfate-induced heaving of stabilized soil specimens. Mineralogical studies such as scanning electron microscopy and energy dispersive spectroscopy were also used to verify the research findings observed from the macro test results.

PT-WFD: the network of PT providers to support the implementation of the European water framework directive

This paper introduces a newly formed network of proficiency testing (PT) providers, who are active and experienced in the field of PTs with relevance to the water framework directive (WFD). The objective of this new network is to support the implementation of the WFD in Europe, by providing PT schemes that meet the specific requirements of the WFD and which are run and evaluated in a harmonised way. The main value of creating such a network lies in the potential to foster the harmonised performance evaluation of European laboratories involved in WFD monitoring analyses, thus enhancing the equivalence of WFD monitoring data obtained throughout Europe. In its first year the network organised two joint PT rounds, one for volatile organic substances (VOC) and another for selected pesticides from the list of Priority Substances, with greater than 100 water analysis labs from a number of European countries participating.

Erratum to: Evaluation of laboratory performance through interlaboratory comparison

Erratum to: Evaluation of laboratory performance through interlaboratory comparison

Ultrasensitive detection of nitric oxide at 5.33 μm by using external cavity quantum cascade laser-based Faraday rotation spectroscopy

A transportable prototype Faraday rotation spectroscopic system based on a tunable external cavity quantum cascade laser has been developed for ultrasensitive detection of nitric oxide (NO). A broadly tunable laser source allows targeting the optimum Q(3/2)(3/2) molecular transition at 1875.81 cm(-1) of the NO fundamental band. For an active optical path of 44 cm and 1-s lock-in time constant minimum NO detection limits (1sigma) of 4.3 parts per billion by volume (ppbv) and 0.38 ppbv are obtained by using a thermoelectrically cooled mercury-cadmium-telluride photodetector and liquid nitrogen-cooled indium-antimonide photodetector, respectively. Laboratory performance evaluation and results of continuous, unattended monitoring of atmospheric NO concentration levels are reported.

A proficiency testing study to evaluate laboratory performance for the detection of different genotypes of parvovirus B19

In Europe, it is a regulatory requirement that parvovirus B19 (B19V) DNA nucleic acid amplification technique-based testing is performed on plasma pools for certain classes of plasma-derived medicinal products. This proficiency testing study set out to examine the ability of public quality control laboratories and plasma fractionation organizations to detect different genotypes of B19V using nucleic acid amplification technique-based assays. Laboratories were supplied with cloned DNAs representing the main genotypes of B19V. All samples were adjusted to equivalent copy number and were distributed as part of a routine external quality assessment programme investigating the evaluation of B19V containing plasma samples by these laboratories. The plasmid clones were distributed to 25 laboratories, representing 13 quality control laboratories and 12 manufacturers of plasma derivatives. The criteria for acceptable detection of the different genotypes of B19V DNA was based upon the maximum theoretical efficiency for polymerase chain reaction amplification. Proficient laboratories were deemed to be those reporting results within 1 log(10) dilution for each of the different virus genotypes. Data were returned by 23 of the participating laboratories. Some laboratories returned data for more than one type of assay and in total 27 data sets were analysed. Nine of the participating laboratories were able to successfully detect all the virus genotypes according to the applied criteria, all except one used in-house assays. The results of the study highlight that there are still discrepancies in the detection of a broad spectrum of B19V genotypes, with implications for batch release testing.