Standard Deviation For Proficiency Assessment Research Articles

Interlaboratory exercise for the analysis of carotenoids and related compounds in dried mango fruit (Mangifera indica L.)

An interlaboratory comparison was done for the analysis of carotenoids in freeze-dried mango. The study was performed from July to September 2018. Mango fruit was freeze-dried, homogenized, and packaged under vacuum conditions in portions of 6 g (test sample). Two test samples were sent to the participating laboratories for analysis. Laboratory results were rated using Z-scores in accordance with ISO 13528 and ISO 17043. The standard deviation for proficiency assessment (also called target standard deviation) was determined using a modified Horwitz function and varied between 10% and 25%, depending on the analyte. Out of 14 laboratories from 10 different countries, 9 laboratories (64%) obtained a satisfactory performance (Z ≤ 2) for the analysis of β-carotene. While for 7 laboratories that analyzed α-carotene, (9Z)-β-carotene, β-cryptoxanthin, and zeaxanthin, 4 laboratories (57%) obtained a satisfactory performance. However, only 2 laboratories out of 7 (29%) obtained a satisfactory performance for lutein. Based on the comparability of the analytical results, this study concludes that freeze-dried mango pulp can be used as a reference material for the analysis of α and β-carotene, (9Z)-β-carotene, β-cryptoxanthin, and zeaxanthin by applying different analytical procedures for their extraction and quantification.

The mycotoxin proficiency testing schemes organized by Turkish National Food Reference Laboratory between the years 2013 and 2020

The Turkish National Food Reference Laboratory has been organizing mycotoxin proficiency tests since 2013. Matrices were mainly selected from the products that Turkey produces and exports most. Therefore, control of mycotoxin contamination in these products intended for domestic consumption and for export to the European Union (EU) and other countries have a special importance. Laboratory controls are one of the important steps in this control. As an external quality control tool, proficiency tests provide the quality assurance of the laboratory test results. For performance evaluation, standard deviation for proficiency assessment is used to calculate z-scores. According to ISO 13528:2015, the standard deviation for proficiency assessment can be determined by experience with previous rounds of proficiency test. As a result of this study, a fit-for-purpose relative standard deviation of 17 % obtained from reevaluation of the previous proficiency tests can be used to calculate standard deviation for proficiency assessment for performance evaluation.

Practical guidance for organizing small interlaboratory comparisons

This article brings practical guidance for organization of interlaboratory comparisons (ILCs) with small number of participants (n = 2–7) that is not arranged by a proficiency testing provider. The methodological guideline aims to provide practical advices in following technical requirements of small ILC organization: design of a scheme, preparation, packaging and distribution of ILC test items, instructions for participants, evaluation of results and participants’ performance, and participation report. The determination of the assigned value and standard deviation for proficiency assessment is also covered and is accompanied by the practical example where the assigned value is established using order statistics.

Proficiency test on the determination of polyethylene and polybutylene terephthalate cyclic oligomers in a food simulant.

The outcome of a proficiency test (PT) organised by the European Union Reference Laboratory for Food Contact Materials (EURL-FCM) is presented. The PT was set up to assess the analytical performance of National Reference Laboratories (NRLs) and Official Control Laboratories (OCLs) in the determination of mass fractions of polyethylene terephthalate (PET) and polybutylene terephthalate (PBT) cyclic dimers and trimers in the official food simulant D1 containing ethanol and water (50:50 v/v). The EURL-FCM had developed and validated an analytical method based on HPLC-UV to monitor the homogeneity and stability of the target oligomers in the PT test items and to determine the respective assigned values, as prescribed in ISO 17043, 2010ISO 17043, 2010. The standard operating procedure of the method was provided to the participants and could be used instead of their own routine methods. Laboratory results were rated using z, z' and ζ scores in accordance with ISO 13528, 2015. The standard deviation for proficiency assessment, σpt , was set to 20 % of the respective assigned value, for all the four studied oligomers, based on the perception of experts. A total of 36 participants from 26 countries have registered to the exercise. They received two test items. Solution 1 consisted of food simulant D1 fortified with a known mass fraction of the four oligomers, while Solution 2 was obtained by a migration experiment with PET bottles and food simulant D1 and further fortification of the resulting solution with the four oligomers. The majority of the participating laboratories presented satisfactory results for the four PET and PBT oligomers. For the analysis of Solution 1, 79-88 % of the participants obtained |z (or z')-scores| below 2, while the satisfactory performances ranged from 71 to 85 % for Solution 2. This PT has been organised for the first time at EU level for the analysis of polyester oligomers and confirms that most of the NRLs are able to monitor properly these oligomers in the frame of Regulation (EU) No 10/2011.

A critical view at the ISO 13528 and IUPAC's harmonized protocol approach for proficiency testing for homogeneity assessment for quantitative variables.

To ensure a correct interpretation of Proficiency Testing (PT) or External Quality Assessment (EQA) results, sample batches need to be homogeneous. The importance of homogeneity testing has been underlined by ISO 13528 and the IUPAC harmonized protocol for proficiency testing. They require that the batch heterogeneity should be smaller than 0.3σpt, with σpt the standard deviation for proficiency assessment according to ISO/IEC 17043:2010 and provide procedures for assessing the batch heterogeneity. In this study, a critical appraisal of these procedures is given. Firstly, an assessment is made of the influence of the criterion of 0.3σpt on laboratory evaluation. Secondly, the assessment procedure of the batch heterogeneity is evaluated, with an emphasis on the calculation and the evaluation of the measured batch standard deviation. It was found that the 0.3σpt criterion may be loosened without a detrimental effect on laboratory evaluation. In addition, it was shown that a flaw in the calculation procedure forces the batch standard deviation to be 0 with a probability of 21.4% for batch standard deviations of 0.3σpt and repeatability of 0.5σpt. The hypothesis test for accepting a batch exhibits a high probability of accepting the batch, even for batch standard deviations that are a multiple of 0.3σpt. To conclude, this study shows that the criteria for homogeneity check of PT items by ISO 13528 and the IUPAC protocol do not assure batch homogeneity. They may be a combination of a criterion that is too strict and a permissive assessment. Alternative assessment procedures should be followed.

Investigation of homogeneity and stability of items for proficiency testing of solid recovered fuels (SRF) analysis

Solid recovered fuel (SRF) is made of waste and used as a substitute for fossil fuel. Physical and chemical analyses are needed to evaluate the usefulness of the waste in power and heat generation. This paper reports the assessment of homogeneity and stability of laboratory and analytical samples, with different particle size, obtained from three materials of different composition. The components of first material included soft plastics, plastics foam, composite packaging with the content of aluminum foil, cigarette filters, paper, cardboard, textiles, wood, and cotton. The second material mainly contained soft plastics and the needles from artificial Christmas trees, a lower amount of artificial leather, and aluminum foil. The third material was mainly composed of soft plastics and paper. The parameter used in the homogeneity and stability assessment was ash content on a dry basis. The homogeneity and stability assessment was carried out according to ISO 13528:2015; on the basis of a standard deviation for proficiency assessment calculated using the repeatability and reproducibility standard deviations from a previous collaborative study of precision of a measurement method. The homogeneity assessment was carried out using laboratory and analytical samples of SRF. Based on measurements of ash content, laboratory samples as the test items OB1 with a particle size below 4 mm were not sufficiently homogeneous, regardless of the SRF type, among our samples those with the largest particles. The analytical samples as the test items OB2 with a particle size below 1 mm and OB3 with a particle size below 0.5 mm were sufficiently homogeneous. For these samples, the stability assessment was carried out and studied samples were stable. The homogeneity and stability of SRF material were associated not so much with the grinding but the composition of the material.

State of the art in the determination of trace elements in seawater: a worldwide proficiency test.

This manuscript presents the results of the International Measurement Evaluation Programme 40 (IMEP-40) study, a proficiency test (PT) which was organised to assess the worldwide performance of laboratories for the determination of trace elements in seawater. This PT supports the implementation of the European Union Water Framework Directive 2000/60/EC, which aims at achieving a long-term high level protection of the aquatic environment, covering lakes, ground water and coastal waters. Forty-six participants reported results. The test item was seawater containing the trace elements As, Cd, Co, Cr, Cu, Fe, Mn, Mo, Ni, Pb, Se and Zn. The trace elements in the test item were present in very low concentrations to mimic natural levels. The results of the participants were rated with z and zeta (ζ) scores in accordance with ISO 13528 and ISO 17043. The standard deviation for proficiency assessment, widehat{upsigma} , was set at 25 % of the respective assigned values for the 12 measured elements based on previous experience with similar PTs. The low levels of the trace elements combined with the high salt concentration of the seawater made the measurements challenging. Many laboratories were unable to detect or quantify the elements and reported “lower than X” values. The percentage of satisfactory performances (expressed as z scores) ranged from 41 % (Cr, Fe) to 86 % (Mo). The PT study showed that the use of proper standard methods, like ISO 17294-2, and sensitive techniques, like inductively coupled plasma mass spectrometry (ICP-MS), contributed to performing well in this PT round.

ABS수지 중 polybrominated diphenyl ether(PBDE)류 분석 숙련도 평가 및 정확도 향상

In order to evaluate and improve the ability of Korean testing laboratories to measure Polybrominated diphenyl ethers in acrylonitrile-butadiene-styrene (ABS), a proficiency test was organised by Korea Research Institute of Standards and Science (KRISS) based on ISO/IEC 17043. The proficiency test material used was 10 g of a granular ABS fortified with a mixture of congeners of PBDE (BDE-154, 183, 206, 209). Homogeneity and stability were investigated to assess the adequacy of the test material. The certified value established by KRISS based on the national reference was used for assigned value of each PBDE. The test materials were distributed to the 16 participating laboratories. The participating laboratories were requested to analyse the samples employing the methods used in their routine analysis. Each laboratory was given it’s own code to secure the anonymity. Participants results were evaluated with z-scores according to ISO/IEC 17043. The standard deviation for proficiency assessment was set by standard deviation of the participants results except for outlier. The results, the laboratory`s performance and improvement of accuracy were discussed.

Comparison of different approaches used for the interpretation and presentation of PT results

The laboratory for precious metals within the Institute of Metrology of Bosnia and Herzegovina acts as a state reference laboratory in the field of quality control of articles manufactured of precious metals, e.g. jewellery. It disseminates reference values through the organisation of proficiency testing (PT) schemes. Two separate PT schemes for the determination of mass fraction of gold in typical jewellery alloys were conducted in 2013 and 2014. Laboratories taking part in these PT schemes are affiliated to the National Metrology Institutes of several countries and two private goldsmith laboratories. Different approaches in the design of the PT schemes and the evaluation of the PT results were used with the purpose of exploring their features regarding the interpretation of the results. For PT.13, z-scores were used as laboratory performance indicators. The assigned value and the standard deviation for proficiency assessment were calculated from the results of the participating laboratories by applying robust statistic. z-score and z′-score were used as performance indicators for the scheme PT.14, and the assigned value of the test sample and its measurement uncertainty were determined by the PT provider. The standard deviation of the testing method of 0.5 mg/g as prescribed in ISO 11426 as fit for purpose in the field of interest is used as standard deviation for proficiency assessment σ p. By comparing the z-scores and z′-scores assigned to the PT.14 results, it is evident that the measurement uncertainty of the assigned value provided by the PT provider does not affect the overall successfulness of the participants. The Q-score based on relative deviation was used as an alternative approach to identify future partners in PT schemes with the purpose of reference value assignment (Q ≤ 0.05).

Preparation of an infant formula proficiency testing material and assessment of its homogeneity and stability

The European Union Reference Laboratory for Chemical Elements in Food of Animal Origin (EURL-CEFAO) is a proficiency testing (PT) provider accredited according to ISO Guide 43-1 and later to ISO/IEC 17043:2010. It organizes exercises devoted to expert laboratories of the EU Member States. PTs are proposed taking into account matrices, analytes, maximum levels (MLs) ruled in Commission Regulation (EC) 1881/2006 (and following amendments) and discussions about new or modified MLs. Following this approach, an exercise was organized in 2012 on the measurement of the cadmium (Cd) and lead (Pb) mass fraction in powdered infant formula based on animal proteins (17th PT). One of the goals was to prepare a material containing Cd around the probable new ML so as to allow participants to test their performance at such a level. The procedure to produce the PT material was set on the basis of the EURL long experience in preparing freeze-dried samples and was tested on a preliminary batch. Homogeneity and stability of the PT material were evaluated through procedures based on international standards. For all tests, a standard deviation for proficiency assessment (σpt), more restrictive than that from the Horwitz equation (σH), was used. The mass fraction of Cd and Pb in the material was proven to be homogeneous and stable for 15 months. These results are extremely satisfactory taking into account the low mass fraction proposed and the restrictive standard deviation for proficiency assessment used to perform the tests. Furthermore, the PT was particularly useful considering that Commission Regulation (EC) 1881/2006 has been recently amended [CR (EU) 488/2014] with the introduction of the new ML for Cd in infant formulas consistent with the mass fraction value proposed in the 17th PT.

Proficiency Testing for Global Solar Collector Certification

Abstract The IEA-SHC launched Task43 about 3 years ago. Now all of the preparations for the program are almost ready and practical application for certification will begin. In order to increase the confidence of all of the interested parties in the testing report and certified documents, peer review will be undertaken by the global solar collector networks for laboratories, certification bodies and inspection agencies. This article is regarding proficiency testing as an important method to assess the laboratory's competence. The following are the key points during the proficiency testing procedure, namely homogeneity and stability for samples, assigned values and uncertainty, and standard deviation for proficiency assessment. Based on ISO/IEC 17043:2010, ISO 13528:2005 and ISO/ FDIS 9806:2013 the proficiency testing scheme for solar collectors is discussed in this article in detail.

Coordination of organochlorine pesticides in water proficiency testing schemes in South Africa

The National Metrology Institute of South Africa (NMISA) developed a proficiency testing (PT) scheme for the analysis of organochlorine pesticides (OCPs) in drinking water, for a limited number of participants. Two identical 1 L water samples fortified with OCPs at the ng/L concentration level were distributed to each participant. The concentration range of pesticides was selected to mimic regulated trace levels in drinking water. The assigned value was based on the traceable, gravimetrically prepared value of each of the analytes spiked into the water samples. The samples were adequately homogeneous and stable over the 3-week reporting period. The standard deviation for proficiency assessment was determined using the alternate Horwitz model, and participant performance was reported using z-scores. En numbers were also included in the PT reports to motivate laboratories to evaluate the suitability of their measurement uncertainty estimations. One of the biggest challenges associated with coordinating this PT scheme remains courier costs and delays in sample distribution within Africa, whilst the accurate determination of pesticides at trace concentration levels remains the biggest challenge for PT scheme participants.

Determination of total cadmium, lead, arsenic, mercury and inorganic arsenic in mushrooms: outcome of IMEP-116 and IMEP-39

The Institute for Reference Materials and Measurements (IRMM) of the Joint Research Centre (JRC), a Directorate General of the European Commission, operates the International Measurement Evaluation Program (IMEP). IMEP organises inter-laboratory comparisons in support of European Union policies. This paper presents the results of two proficiency tests (PTs): IMEP-116 and IMEP-39, organised for the determination of total Cd, Pb, As, Hg and inorganic As (iAs) in mushrooms. Participation in IMEP-116 was restricted to National Reference Laboratories (NRLs) officially appointed by national authorities in European Union member states. IMEP-39 was open to all other laboratories wishing to participate. Thirty-seven participants from 25 countries reported results in IMEP-116, and 62 laboratories from 36 countries reported for the IMEP-39 study. Both PTs were organised in support to Regulation (EC) No. 1881/2006, which sets the maximum levels for certain contaminants in food. The test item used in both PTs was a blend of mushrooms of the variety shiitake (Lentinula edodes). Five laboratories, with demonstrated measurement capability in the field, provided results to establish the assigned values (Xref). The standard uncertainties associated to the assigned values (uref) were calculated by combining the uncertainty of the characterisation (uchar) with a contribution for homogeneity (ubb) and for stability (ust), whilst uchar was calculated following ISO 13528. Laboratory results were rated with z- and zeta (ζ)-scores in accordance with ISO 13528. The standard deviation for proficiency assessment, σp, ranged from 10% to 20% depending on the analyte. The percentage of satisfactory z-scores ranged from 81% (iAs) to 97% (total Cd) in IMEP-116 and from 64% (iAs) to 84% (total Hg) in IMEP-39.

การเตรียมตัวอย่างสำหรับกิจกรรมทดสอบความชำนาญห้องปฏิบัติการของรายการโลหะหนักในนํ้า

The sample preparation for laboratory proficiency testing of heavy metals in water must be accurate, precise, and meet the requirements of ISO/IEC 17043:2010. The aim of this study is to find the suitability of sample preparation for cadmium (Cd), nickel (Ni) and lead (Pb) in water by adding known standard solutions in the deionized water, preserving with nitric acid at pH-value about 2.0, mixing, filling in bottles and labelling. The samples are randomly selected for homogeneity and stability testing. The statistical process was designed for homogeneity and stability evaluation according to the ISO 13528:2005 by comparing with the standard deviation for proficiency assessment which is the target standard deviation by perception from previous rounds, standard method and expert judgment. It was set at 7% of the assigned values which were the reference values from National Metrology Institute (Thailand). The results of homogeneity and stability assessment of three sets of samples showed that they were sufficiently homogeneous and stable. These ensured that all laboratories receive distribution units that do not differ significantly in the parameters to be analyzed.

Harmonization and transferability of performance assessment: experience from four serum aluminum proficiency testing schemes

Proficiency testing schemes monitor laboratory performance and provide a stimulus for improvement in accuracy. Where several schemes operate in the same analytical sector, there are risks that assessments of performance may be in conflict. Performance assessment for the determination of trace elements such as aluminum in serum is particularly important due to the high risk of contamination and therefore erroneous results. The objectives of this work were (1) to compare several mathematical models to establish a predefined standard deviation for proficiency assessment and (2) to evaluate the influence of instrumental methods and proficiency testing scheme on the assessment of performance for serum aluminum measurements. For this purpose, three samples were sent to the participants of four proficiency testing schemes. Assigned values were calculated according to algorithm A according to ISO 13528 and standard deviation for proficiency assessment according to three methods based on individual variability, state of the art or previous proficiency testing results. The method based on individual variability produced a more stringent standard deviation compared to analytical imprecision based on the state of the art. The instrumental methods gave similar results, whereas significant differences were observed between the four proficiency testing schemes indicating that harmonization of the standard deviation for proficiency assessment fails to allow transferability from one proficiency testing scheme to another and that additional factor(s) contribute to variability in performance assessment.

Setting maximum limits for trace elements in baby food in European legislation: the outcome of International Measurement Evaluation Programme®–33

The Institute for Reference Materials and Measurements (IRMM) of the Joint Research Centre (JRC), a Directorate-General of the European Commission, operates the International Measurement Evaluation Programme® (IMEP). It organises various types of inter-laboratory comparisons in support of European Union policies. This paper presents the results of a proficiency testing exercise (PT) focusing on the determination of total cadmium (Cd) and total lead (Pb) mass fractions in baby food in support to Commission Regulation (EC) 1881/2006 of 19 December 2006 setting maximum levels for certain contaminants in foodstuffs. The test material used in this exercise was soya-based baby food formula purchased in a local pharmacy and prepared by the Reference Materials Unit of the IRMM for this exercise. Sixty-six laboratories from 23 countries registered to the exercise and 61 of them reported results. Each participant received one bottle containing approximately 15 g of test material. Participants were asked to quantify the measurands in the powder and in the reconstituted formula. Reference values independent from the participants’ results were established using isotope dilution inductively coupled plasma mass spectrometry. The total Cd mass fraction was determined by IRMM and LGC Ltd (UK), while the total Pb was determined by IRMM. The standard deviation for proficiency assessment was set at 22% of the assigned value for all measurands. Laboratories were rated with z- and ζ- (zeta) scores in accordance with ISO 13528. The outcome of this exercise is clearly influenced by the very low level of Cd and Pb content in the test material which triggered: a high number of ‘less than’ values; overestimated values especially for Pb very likely due to contamination; and a visible method influence in the case of Pb (methods based on atomic absorption were not sensitive enough to attain such low limits of detection). The results were also evaluated with regard to the reported limit of detection and some incoherencies were observed.

Establishing the standard deviation for proficiency assessment ( $$ \hat{\sigma } $$ ). in microbiology PT/EQA schemes

Microbiological testing can be challenging due to the unique nature of microorganisms. Studies have shown that values for reproducibility in microbiology are difficult to establish as they can vary depending upon a number of factors such as the organism under test, the matrix and the test method. The relative lack of consistent data on reproducibility means it can be difficult for providers of microbiology PT schemes to establish a suitable standard deviation for proficiency assessment ( $$ \hat{\sigma } $$ ). Methods for establishing $$ \hat{\sigma } $$ are described in BS ISO 13528: 2005, but these are not all appropriate in microbiology, given the lack of available data and the large range of possible variables. Using the robust standard deviation of participant results is not recommended as this can vary greatly from round to round and means that performance is not comparable over time. Use of a fixed standard deviation depends very much on the perception of the PT scheme organiser as to what constitutes fitness-for-purpose. In microbiology, is it really necessary to obtain precise results? Should the emphasis be on the correct isolation and identification of organisms, and on performance over time to detect laboratory bias rather than on enumeration results alone?

Determination of the standard deviation for proficiency assessment from past participant’s performances

The “uncertainty function” introduced by Thompson et al. estimates the reproducibility standard deviation as a function of concentration or mass fraction. This model was successfully applied to data derived from three proficiency testing schemes aiming at the quantification of cadmium, lead and mercury in blood and urine. This model allows the estimation of standard deviation for the performance assessment for proficiency testing rounds.

Determination of brominated flame retardants: a proficiency test

This manuscript presents the results of a proficiency test for the determination of total bromine and several polybrominated biphenyls and diphenyl ethers (PBB and PBDE) in plastic. The test material used is a poly(ethylene terephthalate) (PET) granulate fortified with a mixture of PBB and PBDE. Up to twenty laboratories from 15 countries registered for the exercise and reported results. Homogeneity and stability were investigated to asses the adequacy of the selected test material. Laboratory results were rated with z and ζ scores according to ISO 13528. The standard deviation for proficiency assessment was set to 25 % of the assigned values. This exercise highlights the difficulties laboratories have to provide consistent values for the investigated measurands. Many participants reported underestimated measurand values; satisfactory z scores ranged from 61 to 88 %. The critical experimental parameters are identified and discussed.

Criteria to define the standard deviation for proficiency assessment for the determination of essential trace elements in serum: comparison of Z-scores based on the Horwitz function or on biological variability

A critical issue in the organisation of Proficiency Testing/External Quality Assessment Schemes is the definition of the criteria against which the performance of individual laboratories should be evaluated. Organisers of EQAS in Occupational and Environmental Laboratory Medicine (http://www.occupational-environmental-laboratory.com) collaborate to define common acceptable levels of performance. The aim of this study was to compare the Horwitz function to the Fraser’s approach. Sets of results obtained from the distribution of test materials in the Network schemes (for the measurands: copper, selenium or zinc in serum) were used to calculate Z-scores according to both approaches. Quality specifications derived from both approaches were also compared to the standard deviations obtained. Except for selenium, Horwitz criteria suggests a more stringent evaluation than Fraser criteria, the latter being very stringent as regard the participant analytical variability.