Risk-adjusted Cumulative Sum Chart Research Articles

A Method for Continuous Surgeon Improvement in Rectal Cancer: Risk-Adjusted Cumulative Sum.

To develop and analyze a risk-adjusted cumulative sum (RA-CUSUM) chart as a potential method to monitor individual surgeon performance in robotic total mesorectal excision (TME) for rectal cancer. Currently, surgeons lack real-time tools to monitor and enhance their performance beyond residency completion. While national quality programs exist, granular, individual-level data is crucial for continuous improvement. Previous studies suggest CUSUM charts hold promise in identifying performance trends and outliers. This retrospective study analyzed data from 640 robotic TME cases performed by 12 surgeons at two institutions. RA-CUSUM charts were generated for three outcomes: complications, operative time, and length of stay. The overall RA-CUSUM curves for operative time and complications showed an initial learning phase followed by a plateau or downward slope, indicating proficiency or improvement. However, individual surgeon curves revealed significant heterogeneity. Three surgeons consistently excelled in operative time, while five minimized complications most effectively. Potential quality improvement could be implemented to drive performance toward positive outliers. No differences were found in unadjusted outcomes, including conversion, number of lymph nodes harvested, and positive circumferential margins. The RA-CUSUM chart is a promising method for identifying individual surgeon performance in robotic TME. It could help surgeons, teams, and leaders identify improvement areas and benchmark themselves against positive outliers. Further studies are needed to explore the potential of RA-CUSUM for implementing interventions to improve surgical quality.

A new phase Ⅱ risk-adjusted CUSUM chart for monitoring surgical performance

In recent years, the use of risk-adjusted control charts for monitoring surgical performance has gained significant attention among researchers. However, many existing monitoring methods only consider patient risk factors that impact surgical performance, and fail to account for other risk factors, such as the surgeon's level of experience. This study presents a new phase II risk-adjusted cumulative sum (CUSUM) chart based on the score test statistic, which includes a risk-adjusted mortality model with dummy variables. This new control chart not only monitors shifts in location and scale parameters related to surgical performance, but also accounts for changes in the risk distribution of the patient population by using dynamic probability control limits (DPCLs). The proposed method is further elaborated and validated through simulations in a practical context, and simulation results indicate that this method is more sensitive to small abnormal changes.

Learning curves for minimally invasive total mesorectal excision beyond the competency phase - a risk-adjusted cumulative sum analysis of 1000 rectal resections.

The learning curve of total mesorectal excision (TME) by minimally invasive surgery (MIS) beyond the competency phase has not been adequately reported with large numbers or using a statistical control limit. The aim of this work was to study the learning curve of MIS TME in the proficiency phase. Risk-adjusted (RA) cumulative sum (CUSUM) and RA Bernoulli CUSUM charts were plotted for sequential MIS TME performed by a surgical team over 1000 cases. Surgical failure, a composite endpoint of conversions, complications of grade IIIA or above, R1 resections and inadequate nodal yield were used to monitor the performance. The RA CUSUM detected an inflection point around the 600th operation. Two peaks were identified that could be traced back to probable causes of surgical failure. Similar inflection points were detected at the 450th case for laparoscopic TME and the 367th case for sphincter preservation. No single definite threshold point was noticed for robotic or abdominoperineal operations. At no point did the curves cross the safety threshold. The probability of surgical failure reduced with increasing experience in the multivariate regression (OR 0.899, p = 0.000). This association persisted irrespective of the surgical approach (laparoscopic versus robotic) or the type of operation (sphincter preservation versus abdominoperineal resection). The learning curves for MIS TME did not cross the safety threshold beyond the competency phase. However, a 10% reduction of relative risk in surgical failure was observed for every 100 cases operated on.

Modeling the patient mix for risk-adjusted CUSUM charts.

The improvement of surgical quality and the corresponding early detection of its changes is of increasing importance. To this end, sequential monitoring procedures such as the risk-adjusted CUmulative SUM chart are frequently applied. The patient risk score population (patient mix), which considers the patients’ perioperative risk, is a core component for this type of quality control chart. Consequently, it is important to be able to adapt different shapes of patient mixes and determine their impact on the monitoring scheme. This article proposes a framework for modeling the patient mix by a discrete beta-binomial and a continuous beta distribution for risk-adjusted CUSUM charts. Since the model-based approach is not limited by data availability, any patient mix can be analyzed. We examine the effects on the control chart’s false alarm behavior for more than 100,000 different scenarios for a cardiac surgery data set. Our study finds a negative relationship between the average risk score and the number of false alarms. The results indicate that a changing patient mix has a considerable impact and, in some cases, almost doubles the number of expected false alarms.

Risk-adjusted zero-inflated Poisson CUSUM charts for monitoring influenza surveillance data

BackgroundThe influenza surveillance has been received much attention in public health area. For the cases with excessive zeroes, the zero-inflated Poisson process is widely used. However, the traditional control charts based on zero-inflated Poisson model, ignore the association between influenza cases and risk factors, and thus may lead to unexpected mistakes when implementing monitoring charts.MethodIn this paper, we proposed risk-adjusted zero-inflated Poisson cumulative sum control charts, in which the risk factors were put to adjust the risk of influenza and the adjustment was made by zero-inflated Poisson regression. We respectively proposed the control chart monitoring the parameters individually and simultaneously.ResultsThe performance of our proposed risk-adjusted zero-inflated Poisson cumulative sum control chart was evaluated and compared with the unadjusted standard cumulative sum control charts in simulation studies. The results show that for different distribution of impact factors and different coefficients, the risk-adjusted cumulative sum charts can generate much less false alarm than the standard ones. Finally, the influenza surveillance data from Hong Kong is used to illustrate the application of the proposed chart.ConclusionsOur results suggest that the adjusted cumulative sum control chart we proposed is more accurate and credible than the unadjusted standard control charts because of the lower false alarm rate of the adjusted ones. Even the unadjusted control charts may signal a little faster than the adjusted ones, the alarm they raise may have low credibility since they also raise alarm frequently even the processes are in control. Thus we suggest using the risk-adjusted cumulative sum control charts to monitor the influenza surveillance data to alert accurately, credibly and relatively quickly.

Multi-objective design of risk-adjusted control chart in healthcare systems with economic and statistical considerations

Using control charts to monitor healthcare systems has gained particular attention. In this paper, a risk-adjusted cumulative sum control chart is designed to monitor surgery outputs. Before undergoing surgery, the patients have some unique risk factors which influence the surgery outputs. Thus, risk-adjustment is carried out with the purpose of taking these risks into account using an accelerated failure time model. But the technical implementation of the chart requires determining the design parameters which should be selected in an optimal way putting the desired statistical and economic considerations into service. To this end, a multi-objective model, considering multiple assignable causes with different occurrence rates and effects, has been proposed and the model is solved with the help of a 5-stage algorithm based on the data envelopment analysis method. Then, to show the performance of the proposed procedure, a real case study has been conducted in the cardiac surgery center. Finally, sensitivity analysis on some cost parameters has been conducted. Furthermore, comparisons of the proposed multi-objective model with the pure economic and pure statistical design models have been made. The results reveal that the multi-objective design of the risk-adjusted cumulative sum chart shows acceptable performance in the presence of multiple assignable.

Learning Curve for Endovascular Treatment of Anterior Circulation Large Vessel Occlusion at a Single Center.

Background and purpose: Data concerning the learning curve for endovascular treatment (EVT) of anterior circulation large vessel occlusion are scarce. This study aimed to investigate the relationship between operator experience and the outcome of EVT and to further identify the number of cases needed to acquire the ability to perform successful reperfusion.Materials and methods: Four hundred and thirty-four patients who underwent EVT by seven operators at a single center from January 2016 to September 2019 were enrolled. Procedural experience was defined by the number of cases performed by each operator. Multivariable backward regression analyses were used to investigate the association between procedural experience and functional independence (defined as a modified Rankin Scale score of 0–2), 90-days mortality, successful reperfusion (defined as a modified Thrombolysis in Cerebral Infarction score of 2b-3), and puncture-to-reperfusion time after adjusting for covariates. A risk-adjusted cumulative sum (RA-CUSUM) chart was utilized to identify the number of caseloads needed to overcome the learning curve effect.Results: Procedural experience was independently associated with functional independence, 90-days mortality, successful reperfusion, and puncture-to-reperfusion time reduction (per 10-case increment: OR 1.219, 95% CI: 1.079–1.383, P < 0.001; OR 0.847, 95% CI: 0.738–0.968, P = 0.016; OR 1.553, 95% CI: 1.332–1.830, P < 0.001 and β 8.087 min, 95% CI: 6.184–9.991, P < 0.001, respectively). The RA-CUSUM chart indicated that at least 29 cases were required to overcome the learning curve effect.Conclusions: There was a dose-response relationship between operator case volume and clinical outcome, procedure time, and successful reperfusion. The experience needed for successful EVT was at least 29 cases.

Risk-adjusted CUSUM control charts for shared frailty survival models with application to hip replacement outcomes: a study using the NJR dataset

BackgroundContinuous monitoring of surgical outcomes after joint replacement is needed to detect which brands’ components have a higher than expected failure rate and are therefore no longer recommended to be used in surgical practice. We developed a monitoring method based on cumulative sum (CUSUM) chart specifically for this application.MethodsOur method entails the use of the competing risks model with the Weibull and the Gompertz hazard functions adjusted for observed covariates to approximate the baseline time-to-revision and time-to-death distributions, respectively. The correlated shared frailty terms for competing risks, corresponding to the operating unit, are also included in the model. A bootstrap-based boundary adjustment is then required for risk-adjusted CUSUM charts to guarantee a given probability of the false alarm rates. We propose a method to evaluate the CUSUM scores and the adjusted boundary for a survival model with the shared frailty terms. We also introduce a unit performance quality score based on the posterior frailty distribution. This method is illustrated using the 2003-2012 hip replacement data from the UK National Joint Registry (NJR).ResultsWe found that the best model included the shared frailty for revision but not for death. This means that the competing risks of revision and death are independent in NJR data. Our method was superior to the standard NJR methodology. For one of the two monitored components, it produced alarms four years before the increased failure rate came to the attention of the UK regulatory authorities. The hazard ratios of revision across the units varied from 0.38 to 2.28.ConclusionsAn earlier detection of failure signal by our method in comparison to the standard method used by the NJR may be explained by proper risk-adjustment and the ability to accommodate time-dependent hazards. The continuous monitoring of hip replacement outcomes should include risk adjustment at both the individual and unit level.

Reducing door-to-needle times in stroke thrombolysis to 13 min through protocol revision and simulation training: a quality improvement project in a Norwegian stroke centre

BackgroundIn eligible patients with acute ischaemic stroke, rapid revascularisation is crucial for good outcome. At our treatment centre, we had achieved and sustained a median door-to-needle time of under 30...

Risk-adjusted monitoring of surgical performance.

We propose a nonparametric risk-adjusted cumulative sum chart to monitor surgical outcomes for patients with different risks of post-operative mortality due to risk factors that exist before the surgery. Using varying-coefficient logistic regression models, we accomplish the risk adjustment. Unknown coefficient functions are estimated by global polynomial spline approximation based on the maximum likelihood principle. We suggest a bisection minimization approach and a bootstrap method to determine the chart testing limit value. Compared with the previous (parametric) risk-adjusted cumulative sum chart, a major advantage of our method is that the morality rate can be modeled more flexibly by related covariates, which significantly enhances the monitoring efficiency. Simulations demonstrate nice performance of our proposed procedure. An application to a UK cardiac surgery dataset illustrates the use of our methodology.

The learning curve associated with intracranial angioplasty and stenting: analysis from a single center.

Intracranial angioplasty and stenting is an optional treatment for patients with symptomatic intracranial artery stenosis (IAS) who have failed to respond to standard medical therapy. We aimed to analyze the effect of the operator's learning curve in our center in terms of the safety of intracranial angioplasty and stenting. In this retrospective study, we analyzed patients treated with intracranial angioplasty and stenting in our center between January 2007 and December 2015. The endpoint was peri-procedure complications within 30 days. A risk-adjusted cumulative sum chart was used for the learning curve analysis. Multivariable logistic analysis was also used to evaluate the effect of the learning curve on intracranial angioplasty and stenting. A total of 329 patients were included. The overall peri-procedure complication rate was 8.21% (27/329). Multivariate logistic regression analysis showed that operators with experience of >20 cases [odds ratio (OR) =0.229; 95% confidence interval (CI): 0.059-0.893] and >40 cases (OR =0.024; 95% CI: 0.006-0.093) were associated with a low risk of complications compared with operators with experience of ≤20 cases. Furthermore, patients with hypertension (OR =3.595; 95% CI: 1.104-11.702) and stenosis of Mori classification C (OR =28.562; 95% CI: 4.788-170.395) were associated with a high risk of complications. The observed outcome was better than expected (more negative deflections) after the 30th consecutive case. The level of operator experience can influence the risk of peri-procedure complications associated with intracranial angioplasty and stenting. A total of 30 consecutive cases is required for an operator to overcome the learning curve.

Use of risk-adjusted CUSUM charts to monitor 30-day mortality in Danish hospitals.

BackgroundMonitoring hospital outcomes and clinical processes as a measure of clinical performance is an integral part of modern health care. The risk-adjusted cumulative sum (CUSUM) chart is a frequently used sequential analysis technique that can be implemented to monitor a wide range of different types of outcomes.ObjectiveThe aim of this study was to describe how risk-adjusted CUSUM charts based on population-based nationwide medical registers were used to monitor 30-day mortality in Danish hospitals and to give an example on how alarms of increased hospital mortality from the charts can guide further in-depth analyses.Materials and methodsWe used routinely collected administrative data from the Danish National Patient Registry and the Danish Civil Registration System to create risk-adjusted CUSUM charts. We monitored 30-day mortality after hospital admission with one of 77 selected diagnoses in 24 hospital units in Denmark in 2015. The charts were set to detect a 50% increase in 30-day mortality, and control limits were determined by simulations.ResultsAmong 1,085,576 hospital admissions, 441,352 admissions had one of the 77 selected diagnoses as their primary diagnosis and were included in the risk-adjusted CUSUM charts. The charts yielded a total of eight alarms of increased mortality. The median of the hospitals’ estimated average time to detect a 50% increase in 30-day mortality was 50 days (interquartile interval, 43;54). In the selected example of an alarm, descriptive analyses indicated performance problems with 30-day mortality following hip fracture surgery and diagnosis of chronic obstructive pulmonary disease.ConclusionThe presented implementation of risk-adjusted CUSUM charts can detect significant increases in 30-day mortality within 2 months, on average, in most Danish hospitals. Together with descriptive analyses, it was possible to use an alarm from a risk-adjusted CUSUM chart to identify potential performance problems.

Online profile monitoring for surgical outcomes using a weighted score test

ABSTRACTIn the past decade, risk-adjusted control charts have been widely used to monitor surgical outcomes. However, most existing approaches focus on monitoring shifts in location parameters and may not be able to detect a scale change that is also likely to occur in surgical data. In this article, we derive a weighted score test statistic to construct an exponentially weighted moving average chart and propose a new charting method to simultaneously monitor location and scale parameters. This new chart may be applied to monitoring surgical performance. Simulation results indicate that the proposed method is more efficient than existing methods, such as the risk-adjusted cumulative sum chart, in detecting the heterogeneity of surgical outcomes. A data set from the Surgical Outcome Monitoring and Improvement Program in Hong Kong is used to illustrate the applicability of the proposed chart.

Improved implementation of the risk-adjusted Bernoulli CUSUM chart to monitor surgical outcome quality.

The traditional implementation of the risk-adjusted Bernoulli cumulative sum (CUSUM) chart for monitoring surgical outcome quality requires waiting a pre-specified period of time after surgery before incorporating patient outcome information. We propose a simple but powerful implementation of the risk-adjusted Bernoulli CUSUM chart that incorporates outcome information as soon as it is available, rather than waiting a pre-specified period of time after surgery. A simulation study is presented that compares the performance of the traditional implementation of the risk-adjusted Bernoulli CUSUM chart to our improved implementation. We show that incorporating patient outcome information as soon as it is available leads to quicker detection of process deterioration. Deterioration of surgical performance could be detected much sooner using our proposed implementation, which could lead to the earlier identification of problems.

Dynamic probability control limits for risk-adjusted CUSUM charts based on multiresponses.

For a patient who has survived a surgery, there could be several levels of recovery. Thus, it is reasonable to consider more than two outcomes when monitoring surgical outcome quality. The risk-adjusted cumulative sum (CUSUM) chart based on multiresponses has been developed for monitoring a surgical process with three or more outcomes. However, there is a significant effect of varying risk distributions on the in-control performance of the chart when constant control limits are applied. To overcome this disadvantage, we apply the dynamic probability control limits to the risk-adjusted CUSUM charts for multiresponses. The simulation results demonstrate that the in-control performance of the charts with dynamic probability control limits can be controlled for different patient populations because these limits are determined for each specific sequence of patients. Thus, the use of dynamic probability control limits for risk-adjusted CUSUM charts based on multiresponses allows each chart to be designed for the corresponding patient sequence of a surgeon or a hospital and therefore does not require estimating or monitoring the patients' risk distribution. Copyright © 2017 John Wiley & Sons, Ltd.

Performance of risk-adjusted cumulative sum charts when some assumptions are not met

ABSTRACTMonitoring health care performance outcomes such as post-operative mortality rates has recently become more common, spurring new statistical methodologies designed for this purpose. One such methodology is the Risk-adjusted Cumulative Sum chart (RA-CUSUM) for monitoring binary outcomes such as mortality after cardiac surgery. When building RA-CUSUMs, independence and model correctness are assumed. We carry out a simulation study to examine the effect of violating these two assumptions on the chart's performance.

A new risk-adjusted Bernoulli cumulative sum chart for monitoring binary health data.

To monitor a health event in patients with a specific risk of developing the event, a risk-adjusted cumulative sum chart is needed. The risk-adjusted cumulative sum chart proposed in the literature has some limitations. Setting appropriate control limits is not straightforward, there is no simple formula for constructing them, and they remain sensitive to changes in the underlying risk distribution and the baseline incidence rate. To overcome these limits, we propose a new risk-adjusted Bernoulli cumulative sum chart as a simple and efficient solution. Analyses of simulated and real data sets illustrate the performance and usefulness of the proposed procedure.

Reduction of the Effect of Estimation Error on In-control Performance for Risk-adjusted Bernoulli CUSUM Chart with Dynamic Probability Control Limits

The in-control performance of any control chart is highly associated with the accuracy of estimation for the in-control parameter(s). For the risk-adjusted Bernoulli cumulative sum (CUSUM) chart with a constant control limit, it had been shown that the estimation error could have a substantial effect on the in-control performance. In our study, we examine the effect of estimation error on the in-control performance of the risk-adjusted Bernoulli CUSUM chart with dynamic probability control limits (DPCLs). Our simulation results show that the in-control performance of risk-adjusted Bernoulli CUSUM chart with DPCLs is also affected by the estimation error. The most important factors affecting estimation error are the specified desired in-control average run length, the Phase I sample size, and the adverse event rate. However, the effect of estimation error is uniformly smaller for the risk-adjusted Bernoulli CUSUM chart with DPCLs than for the corresponding chart with a constant control limit under various realistic scenarios. In addition, we found a substantial reduction in the mean and variation of the standard deviation of the in-control run length when DPCLs are used. Therefore, use of DPCLs has yet another advantage when designing a risk-adjusted Bernoulli CUSUM chart. Copyright © 2016 John Wiley & Sons, Ltd.

Monitoring foreclosure rates with a spatially risk-adjusted Bernoulli CUSUM chart for concurrent observations

ABSTRACTFrequently in process monitoring, situations arise in which the order that events occur cannot be distinguished, motivating the need to accommodate multiple observations occurring at the same time, or concurrent observations. The risk-adjusted Bernoulli cumulative sum (CUSUM) control chart can be used to monitor the rate of an adverse event by fitting a risk-adjustment model, followed by a likelihood ratio-based scoring method that produces a statistic that can be monitored. In our paper, we develop a risk-adjusted Bernoulli CUSUM control chart for concurrent observations. Furthermore, we adopt a novel approach that uses a combined mixture model and kernel density estimation approach in order to perform risk-adjustment with regard to spatial location. Our proposed method allows for monitoring binary outcomes through time with multiple observations at each time point, where the chart is spatially adjusted for each Bernoulli observation's estimated probability of the adverse event. A simulation study is presented to assess the performance of the proposed monitoring scheme. We apply our method using data from Wayne County, Michigan between 2005 and 2014 to monitor the rate of foreclosure as a percentage of all housing transactions.

Dynamic probability control limits for risk-adjusted Bernoulli CUSUM charts

The risk-adjusted Bernoulli cumulative sum (CUSUM) chart developed by Steiner et al. (2000) is an increasingly popular tool for monitoring clinical and surgical performance. In practice, however, the use of a fixed control limit for the chart leads to a quite variable in-control average run length performance for patient populations with different risk score distributions. To overcome this problem, we determine simulation-based dynamic probability control limits (DPCLs) patient-by-patient for the risk-adjusted Bernoulli CUSUM charts. By maintaining the probability of a false alarm at a constant level conditional on no false alarm for previous observations, our risk-adjusted CUSUM charts with DPCLs have consistent in-control performance at the desired level with approximately geometrically distributed run lengths. Our simulation results demonstrate that our method does not rely on any information or assumptions about the patients' risk distributions. The use of DPCLs for risk-adjusted Bernoulli CUSUM charts allows each chart to be designed for the corresponding particular sequence of patients for a surgeon or hospital.