Patient-based Real-time Quality Control Research Articles

Enhanced patient-based real-time quality control using the graph-based anomaly detection.

Patient-based real-time quality control (PBRTQC) is an alternative tool for laboratories that has gained increasing attention. Despite the progress made by using various algorithms, the problems of data volume imbalance between in-control and out-of-control results, as well as the issue of variation remain challenges. We propose a novel integrated framework using anomaly detection and graph neural network, combining clinical variables and statistical algorithms, to improve the error detection performance of patient-based quality control. The testing results of three representative analytes (sodium, potassium, and calcium) and eight independent variables of patients (test date, time, gender, age, department, patient type, and reference interval limits) were collected. Graph-based anomaly detection network was modeled and used to generate control limits. Proportional and random errors were simulated for performance evaluation. Five mainstream PBRTQC statistical algorithms were chosen for comparison. The framework of a patient-based graph anomaly detection network for real-time quality control (PGADQC) was established and proven feasible for error detection. Compared with classic PBRTQC, the PGADQC showed a more balanced performance for both positive and negative biases. For different analytes, the average number of patient samples until error detection (ANPed) of PGADQC decreased variably, and reductions could reach up to approximately 95 % at a small bias of 0.02 taking calcium as an example. The PGADQC is an effective framework for patient-based quality control, integrating statistical and artificial intelligence algorithms. It improves error detection in a data-driven fashion and provides a new approach for PBRTQC from the data science perspective.

Integrating Patient-Based Real-Time Quality Control (PBRTQC) in a New Field: Inter-Comparison between Biochemical Instrumentations with LDL-C.

Patient-based real-time quality control (PBRTQC) can be a valuable tool in clinical laboratories due to its cost-effectiveness and constant monitoring. More focus is placed on discovering and improving algorithms that compliment conventional internal control techniques. The practical implementation of PBRTQC with a biochemical instrument comparison is lacking. We aim to evaluate PBRTQC's efficacy and practicality by comparing low-density lipoprotein cholesterol (LDL-C) test results to ensure consistent real-time monitoring across biochemical instrumentations in clinical laboratories. From 1 September 2021 to 30 August 2022, the First Affiliated Hospital of Xi'an Jiaotong University collected data from 158,259 both healthy and diseased patients, including 84,187 male and 74,072 female patients, and examined their LDL-C results. This dataset encompassed a group comprising 50,556 individuals undergoing health examinations, a group comprising 42,472 inpatients (IP), and a group comprising 75,490 outpatients (OP) for the PBRTQC intelligent monitoring platform to conduct daily tests, parameter configuration, program development, real-time execution, and performance validation of the patients' data. Moreover 40 patients' LDL-C levels were assessed using two biochemical analyzers, designated as the reference and comparator instruments. A total of 160 LDL-C results were obtained from 40 both healthy and diseased patients, including 14 OP, 16 IP, and 10 health examination attendees, who were selected to represent LDL-C levels broadly. Two biochemical instruments measured LDL-C measurements from the same individuals to investigate consistency and reproducibility across patient statuses and settings. We employed exponentially weighted moving average (EWMA) and moving median (MM) methods to calculate inter-instrument bias and ensure analytical accuracy. Inter-instrument bias for LDL-C measurements was determined by analyzing fresh serum samples, different concentrations of quality control (QC), and commercialized calibrators, employing both EWMA and MM within two assay systems. The assessment of inter-instrumental bias with five different methods adhered to the external quality assessment standards of the Clinical Laboratory Center of the Health Planning Commission, which mandates a bias within ±15.0%. We calculated inter-instrument comparison bias with each of the five methods based on patient big data. The comparison of fresh serum samples, different concentrations of QC, commercialized calibrators, and EWMA were all in the permissive range, except for MM. MM showed that the bias between two biochemical instruments in the concentration ranges of 1.5 mmoL/L-6.2 mmoL/L exceeded the permissible range. This was mainly due to the small number of specimens, affected by variations among individual patients, leading to increased false alarms and reduced effectiveness in monitoring the consistency of the inter-instrumental results. Moreover, the inter-comparison bias derived from EWMA was less than 3.01%, meeting the 15% range assessment criteria. The bias result for MM was lower than 24.66%, which was much higher than EWMA. Thus, EWMA is better than MM for monitoring inter-instrument comparability. PBRTQC can complement the use of inter-comparison bias between biochemical analyzers. EWMA has comparable inter-instrument comparability monitoring efficacy. The utilization of AI-based PBRTQC enables the automated real-time comparison of test results across different biochemical instruments, leading to a reduction in laboratory operating costs, enhanced work efficiency, and improved QC. This advanced technology facilitates seamless data integration and analysis, ultimately contributing to a more streamlined and efficient laboratory workflow in the biomedical field.

Application of Patient-Based Real-Time Quality Control Based on Artificial Intelligence Monitoring Platform in Continuously Quality Risk Monitoring of Down Syndrome Serum Screening.

Patient-based real-time quality control (PBRTQC) has gained attention because of its potential to continuously monitor the analytical quality insituations wherein internal quality control (IQC) is less effective. Therefore, we tried to investigate the application of PBRTQC method based on an artificial intelligence monitoring (AI-MA) platform in quality risk monitoring of Down syndrome (DS) serum screening. The DS serum screening item determination data and relative IQC data from January 4 to September 7 in 2021 were collected. Then, PBRTQC exponentially weighted moving average (EWMA) and moving average (MA) procedures were built and optimized in the AI-MA platform. The efficiency of the EWMA and MA procedures with intelligent and traditional control rules were compared. Next, the optimal EWMA procedures that contributed to the quality assurance of serum screening were run and generated early warning cases were investigated. Optimal EWMA and MA procedures on the AI-MA platform were built. Comparison results showed the EWMA procedure with intelligent QC rules but not traditional quality rules contained the best efficiency. Based on the AI-MA platform, two early warning cases were generated by using the optimal EWMA procedure, which finally found were caused by instrument failure. Moreover, the EWMA procedure could truly reflect the detection accuracy and quality insituations wherein traditional IQC products were unstable or concentrations were inappropriate. The EWMA procedure built by the AI-MA platform could be a good complementary control tool for the DS serum screening by truly and timely reflecting the detection quality risks.

Exploring optimization algorithms for establishing patient-based real-time quality control models

BackgroundPatient-based real-time quality control (PBRTQC) models must be optimized for use in different clinical laboratories, but the grid search (GS) algorithm explored in recent studies for this purpose is inefficient. Thus, finding an efficient optimization algorithm is critical for future research and implementation of the PBRTQC. MethodsWe compared the efficiency and performance of five commonly used optimization algorithms, including GS, simulated annealing (SA), genetic algorithms (GA), differential evolution (DE), and particle swarm optimization (PSO), to optimize conventional PBRTQC and regression-adjusted real-time quality control (RARTQC) models for serum alanine aminotransferase and sodium. ResultsThe GS, GA, DE, and PSO provided models with similar performances. However, GA and DE required significantly less computation time than GS. The results also demonstrate a general tradeoff between the optimization method's chance of discovering the optimum and the computation time required. ConclusionMore efficient optimization methods should be adopted when establishing PBRTQC or RARTQC models to save time and computing power that will enable the development of more complex models and increase the scalability of extensive PBRTQC applications.

Development and Evaluation of "The Delta Plus-Minus Even Distribution Check": A Novel Patient-Based Real-Time Quality Control Method for Laboratory Tests.

Laboratory testing of large sample numbers necessitates high-volume rapid processing, and these test results require immediate validation and a high level of quality assurance. Therefore, real-time quality control including delta checking is an important issue. Delta checking is a process of identifying errors in individual patient results by reviewing differences from previous results of the same patient (Δ value). Under stable analytical conditions, Δ values are equally positively and negatively distributed. The previous 20 Δ values from 3 tests (cholesterol, albumin, and urea nitrogen) were analyzed by calculating the R-value: "the positive Δ value ratio minus 0.5." This method of monitoring optimized R-values is referred to as the even-check method (ECM) and was compared with quality control (QC) testing in terms of error detection. Bias was observed on 4 of the 120 days for the 3 analytes measured. When QC detected errors, the ECM captured the same systematic errors and more rapidly. In contrast, the ECM did not generate an alarm for the one random error that occurred in QC. While QC did not detect any errors, the percentage of R-values exceeding the acceptable range was under 2%, the number of days generating alarms was between 16 and 21 days, with short alarm periods, and a median number of samples per alarm period between 7 and 9 samples. The ECM is a practical real-time QC method, controlled by setting R-value conditions, that quickly detects bias values.

The effect of ratios upon improving patient-based real-time quality control (PBRTQC) performance.

Large biological variation hinders application of patient-based real-time quality control (PBRTQC). The effect of analyte ratios on the ability of PBRTQC to improve error detection was investigated. Four single analyte-ratio pairs (alanine aminotransferase [ALT] vs. ALT to aspartate aminotransferase ratio [RALT]; creatinine [Cr] vs. Cr to cystatin C ratio [RCr]; lactate dehydrogenase [LDH] vs. LDH to hydroxybutyrate dehydrogenase ratio [RLDH]; total bilirubin [TB] vs. TB to direct bilirubin ratio [RTB]) were chosen for comparison. Various procedures, including four conventional algorithms (moving average [MA], moving median [MM], exponentially weighted moving average [EWMA] and moving standard deviation [MSD]) were assessed. A new algorithm that monitors the number of defect reports per analytical run (NDR) was also evaluated. When a single analyte and calculated ratio used the same PBRTQC parameters, fewer samples were needed to detect systematic errors (SE) by taking ratios (p<0.05). Application of ratios in MA, MM and EWMA significantly enhanced their ability to detect SE. The influence of ratio on random error (RE) detection depended upon the analytes and PBRTQC parameters, as consistent advantage was not demonstrated. The NDR method performed well when appropriate parameters were used, but was only effective for unilateral SE. Rearrangement of sample order led to a significant deterioration of conventional algorithms' performance, while NDR remained almost unaffected. For analytes with large variation and poor PBRTQC performance, using ratios as PBRTQC indexes may significantly improve performance and achieve better anti-interference ability, providing a new class of monitoring indicators for PBRTQC.

Knowledge, attitude, and practice of patient-based real-time quality control in Australasia

Knowledge, attitude, and practice of patient-based real-time quality control in Australasia

Advances in clinical chemistry patient-based real-time quality control (PBRTQC).

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Moving average quality control of routine chemistry and hematology parameters- a toolbox for implementation.

Moving average quality control (MA QC) is apatient-based real-time quality control system. Advantages compared to conventional periodic internal quality control (IQC) include absence of commutability problems and continuous monitoring of performance. We implemented MA QC for multiple routine hematology and chemistry parameters. We describe the evaluation process and provide practical tools to aid MA QC implementation. Nine parameters (serum sodium, calcium, bicarbonate and free thyroxine, hemoglobin [Hb], mean corpuscular volume, mean corpuscular hemoglobin concentration [MCHC], reticulocyte count and erythrocyte sedimentation rate [ESR]) were chosen for initial consideration. Using data extractions from the laboratory information system (LIS;General Laboratory Information Management System), evaluation of usefulness and optimization of MA QC settings was performed using bias detection curves. After this, MA QC settings were incorporated in our LIS for further evaluation and implementation in routine care. Three out of nine parameters (Hb, ESR, and sodium) were excluded from MA QC implementation due tohigh variation and technical issues in the LIS. For the sixremaining parameters, MA QC showed added value to IQC and was therefore implemented in the LIS. For three parameters a direct MA alarm work-up method was set up, including newly developed built-in features in the LIS. Forthe other parameters, we identified MA utilization beyond real-time monitoring. Implementation of MA QC has added value for our laboratory setting. Additional utilization beyond real-time QC monitoring was identified. We find MA QC especially useful for trend monitoring, detection of small shifts after maintenance and inter-analyzer comparisons.

A study on quality control using delta data with machine learning technique

BackgroundIn the big data era, patient-based real-time quality control (PBRTQC), as an emerging quality control (QC) method, is expanding within the clinical laboratory industry. However, the main issue of current PBRTQC methodology is data stability. Our study is aimed to explore a novel protocol for data stability by combining delta data with machine learning (ML) technique to improve the capacity of QC event detection. MethodsA data set of 423,290 laboratory results from Beijing Chao-yang Hospital 2019 patient results were used as a training set (n = 380960, 90%) and internal validation set (n = 42330, 10%). A further 22,460 results from Beijing Long-fu Hospital 2019 patient results were used as a test set. Three-type data (1) Single-type data processed by truncation limits; (2) delta-type data processed by truncation limits and (3)delta-type data processed by Isolated Forest (IF) algorithm were evaluated with accuracy, sensitivity, NPed, etc., and compared with previously published statistical methods. ResultsThe optimal model was based on Random Forest (RF) algorithm by using delta-type data processed by IF algorithm. The model had a better accuracy (0.99), sensitivity (0.99) specificity (0.99) and AUC (0.99) with the dependent test set, surpassing the critical bias of PBRTQC by over 50%. For the LYMPH#, HGB, and PLT, the cumulative MNPed of MLQC were reduced by 95.43%, 97.39%, and 97.97% respectively when compared to the best of the PBRTQC. ConclusionFinal results indicate that by integrating an innovative ML algorithm with the overall data processing protocol the detection of QC events is improved.

Patient-based real-time quality control for quantitative hepatitis B virus DNA test using moving rate of positive and negative patient results.

Patient-based real-time quality control (PBRTQC) has gained increasing attention in the field of laboratory quality management in recent years. However, PBRTQC has not been reported for use in molecular diagnostics. This study introduces PBRTQC to quantitative hepatitis B virus (HBV) DNA test using moving rate (MR) of positive and negative patient results. In contrast to the MR protocols described in other literature, MR protocol for HBV-DNA test has an additional logarithmic transformation and binary conversion steps before using a common statistical process control algorithm, such as the MR. We used all patient test results of HBV-DNA assay from August 2018 to August 2021 at the Second Affiliated Hospital of Guangzhou University of Chinese Medicine, for parameters setting, optimization, and performance validation. The false rejection rate, error detection curves and validation charts were used to assess the MR protocols. The false rejection rates of two MR protocols were both <0.7%. The optimal block sizes for positive and negative errors in each cut-off value were not the same, so we first proposed a combined protocol that used different block size to detect negative and positive errors. It turned out that the combined protocols outperformed the simple protocols for each cut-off value, especially detecting positive errors. The performances of MR protocols using positive or negative patient results to detect constant errors of HBV-DNA test could meet laboratory requirements. Therefore, we have provided an effective alternative toolfor internal quality control in the field of molecular diagnostics.

Optimization of patient-based real-time quality control based on the Youden index

AimThis study sets out to investigate the utility of exponentially weighted moving average (EWMA) as patient-based real-time quality control (PBRTQC) by conducting a simulation study and subsequent real-patient data implementation to determine optimal EWMA features (weighting factors, control limits, and truncation methods) based on the Youden index. MethodsA simulation experiment was conducted in the first stage to investigate optimal EWMA features for the tests, including aspartate aminotransferase, blood urea nitrogen, and glucose, calcium, creatinine, potassium, sodium, triglycerides, thyroid - stimulating hormone (TSH), and vitamin B12 tests. In the second stage of the study, EWMA was applied to real patient data to elucidate practical utility and achieve final optimal EWMA features. Different degrees of systematic errors (SE) including total allowable error (TEa) as a maximum error level were added to both simulation and patient results, and then the EWMA performance was assessed for different EWMA features. We calculated Youden’s index for each combination of EWMA features to find their optimal features to achieve minimum false positive rate (FPR) and maximum error detection rate at the SE level corresponding to TEa. ResultsEWMA implementation on real patient data revealed optimal EWMA features for each test. FPR values of creatinine and glucose were 18.48% and 10.17%, respectively, which exceeded the acceptable criteria for FPR (10%). The remaining six analytes showed acceptable FPR. ConclusionsWe showed the implementation of EWMA as PBRTQC, and optimization of its features based on the Youden index by conducting extensive performance evaluations and simulations in this study.

Intelligent Quality Management 2 with IntraSpect™ technology for quality control of GEM® Premier™ 5000 blood gas analyzers— A novel application of the patient sample as its own control

BackgroundQuality control (QC) in point-of-care (POC) testing has been greatly improved by automatic control processes, such as the Intelligent Quality Management (iQM®) technology found in GEM Premier blood gas analyzers (Werfen, Bedford, MA). The 2nd generation technology, iQM2, provides additional capabilities, notably the incorporation of IntraSpect software that monitors the response curves of individual tests to detect transient errors caused by micro-clots, micro-bubbles or any event that disturbs the sensor response during sample data acquisition. IntraSpect is a novel form of patient-based, real-time quality control (PBRTQC). MethodsIntraSpect pattern recognition software monitors the last 15 measurements of each patient-response curve. Control limits for slope coefficients have been established from theoretical models and empirical data. Abnormal measurement behavior is flagged to identify transient errors that invalidate test results. Results from 1,013,391 patient samples were collected on 4,985 GEM Premier 5000 cartridges and 2,765 instruments in clinical use worldwide. Results and conclusionsTotal pre-analytic and transient errors detected by IntraSpect were 1.91% worldwide. iQM2 with IntraSpect technology provides a unique control function detecting transient errors that would otherwise go undetected with traditional QC. Together with the statistical QC technology in iQM2, pre-analytic, analytic, and transient analytic errors are detected much faster—seconds versus hours—than by traditional statistical QC.

A study of the moving rate of positive results for use in a patient-based real-time quality control program on a procalcitonin point-of-care testing analyzer.

ObjectiveTo establish an applicable and highly sensitive patient‐based real‐time quality control (PBRTQC) program based on a data model constructed with patients’ results of a procalcitonin point‐of‐care testing (POCT) analyzer.MethodsPatients’ results were retrospectively collected within one year. The Excel software was used to establish quality control (QC) programs of the moving average (MA) and the moving rate of positive results (MR). A Monte Carlo simulation was used to introduce positive and negative biases between 0.01 and 1 ng/ml at random points of the testing data set. Different parameters were used to detect the biases, and the detection efficiency was expressed using the median number of patient samples affected until error detection (MNPed). After comparing the MNPeds of different programs, MA and MR programs with appropriate parameters were selected, and validation plots were generated using MNPeds and maximum number of the patient samples affected (MAX). β curves were generated using the power function of the programs, the performances were compared with that of the conventional QC program.ResultsNeither the conventional QC nor MA program was sensitive to small bias, While MR program can detect the minimum positive bias of 0.06 ng/ml and negative of 0.4 ng/ml at an average daily run size of 10 specimens, with FRs < 1.0%, βs < 1%.ConclusionThe MR program, which is more sensitive to small biases than conventional QC and MA programs, with low FR and β. As such, it can be used as a PBRTQC program with high performance.

Patient-based real-time quality control: A primer

Patient-based real-time quality control (PBRTQC) techniques utilise routinely generated patient results to monitor assay performance. These techniques have been described in the clinical laboratory for more than 50 years. PBRTQC has a number of purported advantages over traditional quality control (QC) including commutability, cost and the opportunity for real-time monitoring. Despite these benefits, the widespread uptake of PBRTQC methods in many laboratories has been slow. In this session, the basics of a PBRTQC program in clinical chemistry will be discussed along with recent developments in the field.

Application of patient-based real-time quality control using exponentially weighted moving average method on internal quality control procedures for serum ion items

Application of patient-based real-time quality control using exponentially weighted moving average method on internal quality control procedures for serum ion items

Comparison and optimization of various moving patient-based real-time quality control procedures for serum sodium.

BackgroundPatient‐based real‐time quality control (PBRTQC) is a valuable tool for monitoring the performance of testing processes. We aimed to compare and optimize various PBRTQC procedures for serum sodium.MethodsIn a computer simulation, artificial errors were added to 680,000 real patients’ results. The characteristics of error detection of various algorithms—moving average, moving median, moving SD and moving proportion of normal results including different control limits (CLs)—were assessed on their ability to detect critical errors early.ResultsThe moving average and moving median were sensitive to system error, and the moving SD tended to detect random error. P3SD (moving proportion of normal results, CLs based on mean and SD of proportion of normal results) demonstrated excellent performance for both system error and random error. The increase of block sizes (N) leads to the delay of error detection and the decrease of false rejection, except for QC procedures with minimum and maximum as CLs. CLs calculation with “0.1% false alarm rate” had more effective performance than that set false alarm to zero (minimum and maximum as CLs). The impact of truncation on QC performance depended on truncation limits, algorithms and the types of error. The significant improvement in QC performance due to truncation was only found in moving SD.Conclusion“P3SD,N = 50, without truncation” and “moving SD, N = 25, set 0.1% false alarm as CLs and set 1% outliers exclusion as truncation limits” were recommended as the optimized procedures for serum sodium to monitor system error and random error, respectively.

Regression-Adjusted Real-Time Quality Control.

Patient-based real-time quality control (PBRTQC) has gained increasing attention in the field of clinical laboratory management in recent years. Despite the many upsides that PBRTQC brings to the laboratory management system, it has been questioned for its performance and practical applicability for some analytes. This study introduces an extended method, regression-adjusted real-time quality control (RARTQC), to improve the performance of real-time quality control protocols. In contrast to the PBRTQC, RARTQC has an additional regression adjustment step before using a common statistical process control algorithm, such as the moving average, to decide whether an analytical error exists. We used all patient test results of 4 analytes in 2019 from Zhongshan Hospital, Fudan University, to compare the performance of the 2 frameworks. Three types of analytical error were added in the study to compare the performance of PBRTQC and RARTQC protocols: constant, random, and proportional errors. The false alarm rate and error detection charts were used to assess the protocols. The study showed that RARTQC outperformed PBRTQC. RARTQC, compared with the PBRTQC, improved the trimmed average number of patients affected before detection (tANPed) at total allowable error by about 50% for both constant and proportional errors. The regression step in the RARTQC framework removes autocorrelation in the test results, allows researchers to add additional variables, and improves data transformation. RARTQC is a powerful framework for real-time quality control research.

Impact of combining data from multiple instruments on performance of patient-based real-time quality control.

IntroductionIt is unclear what is the best strategy for applying patient-based real-time quality control (PBRTQC) algorithm in the presence of multiple instruments. This simulation study compared the error detection capability of applying PBRTQC algorithms for instruments individually and in combination using serum sodium as an example.Materials and methodsFour sets of random serum sodium measurements were generated with differing means and standard deviations to represent four simulated instruments. Moving median with winsorization was selected as the PBRTQC algorithm. The PBRTQC parameters (block size and control limits) were optimized and applied to the four simulated laboratory data sets individually and in combination.ResultsWhen the PBRTQC algorithm were individually optimized and applied to the data of the individual simulated instruments, it was able to detect bias several folds faster than when they were combined. Similarly, the individually applied algorithms had perfect error detection rates across different magnitudes of bias, whereas the error detection rates of the algorithm applied on the combined data missed smaller biases. The performance of the individually applied PBRTQC algorithm performed more consistently among the simulated instruments compared to when the data were combined.DiscussionWhile combining data from different instruments can increase the data stream and hence, increase the speed of error detection, it may widen the control limits and compromising the probability of error detection. The presence of multiple instruments in the data stream may dilute the effect of the error when it only affects a selected instrument.

Benefits, limitations, and controversies on patient-based real-time quality control (PBRTQC) and the evidence behind the practice.

In recent years, there has been renewed interest in the "old" average of normals concept, now generally referred to as moving average quality control (MA QC) or patient-based real-time quality control (PBRTQC). However, there are some controversies regarding PBRTQC which this review aims to address while also indicating the current status of PBRTQC. This review gives the background of certain newly described optimization and validation methods. It also indicates how QC plans incorporating PBRTQC can be designed for greater effectiveness and/or (cost) efficiency. Furthermore, it discusses controversies regarding the complexity of obtaining PBRTQC settings, the replacement of iQC, and software functionality requirements. Finally, it presents evidence of the added value and practicability of PBRTQC. Recent developments in, and availability of, simulation methods to optimize and validate laboratory-specific PBRTQC procedures have enabled medical laboratories to implement PBRTQC in their daily practice. Furthermore, these methods have made it possible to demonstrate the practicability and added value of PBRTQC by means of two prospective "clinical" studies and other investigations. Although internal QC will remain an essential part of any QC plan, applying PBRTQC can now significantly improve its performance and (cost) efficiency.