Pre-intervention Trends Research Articles

Effects of Paying Physicians Based on their Relative Performance for Quality

Studies examining the effectiveness of pay-for-performance programs to improve quality of care primarily have been confined to bonus-type arrangements that reward providers for performance above a predetermined threshold. No studies to date have evaluated programs placing providers at financial risk for performance relative to other participants in the program. The objective of the study is to evaluate the impact of an incentive program conferring limited financial risk to primary care physicians. There were 334 participating primary care physicians in Rochester, New York. The design of the study is a retrospective cohort study using pre/post analysis. The measurements adhere to 4 diabetes performance measures between 1999 and 2004. While absolute performance levels increased across all measures immediately following implementation, there was no difference between the post- and pre-intervention trends indicating that the overall increase in performance was largely a result of secular trends. However, there was evidence of a modest 1-time improvement in physician adherence for eye examination that appeared attributable to the incentive program. For this measure, physicians improved their adherence rate on average by 7 percentage points in the year after implementation of the program. This study demonstrates a modest effect in improving provider adherence to quality standards for a single measure of diabetes care during the early phase of a pay-for-performance program that placed physicians under limited financial risk. Further research is needed to determine the most effective incentive structures for achieving substantial gains in quality of care.

Investigation into the use of the CUSUM technique in identifying changes in mean air pollution levels following introduction of a traffic management scheme

There is an increasing need for statistical techniques to identify and quantify the effects of traffic management schemes on ambient pollution levels. Cumulative sum (CUSUM) charts have been used extensively in industrial process control to detect deviations in production parameters from pre-determined values. This study investigates the use of the CUSUM procedure to identify change in ambient air pollution levels following the introduction of a traffic management scheme at a specific location in Central London. The CUSUM methods of Lucas first compute the standardised deviations of time series observations from the desired process mean. These are accumulated over time to compute the CUSUM at each time point. Data for the analysis were taken from a kerbside monitoring site on Marylebone Road, a six lane trunk route in Central London. In August 2001 the lane adjacent to the monitoring site was designated as a permanent bus lane. The CUSUM analysis clearly identifies a sustained decrease in carbon monoxide concentrations beginning in 2002. However, seasonality and other factors precluded precise characterisation of the timing of the change. When the analysis was repeated using a reference mean that extrapolated the pre-intervention trend in carbon monoxide concentrations, the CUSUM chart no longer identified a sustained decrease. CUSUM appears to offer a simple and rapid method for identifying sustained changes in pollution levels, but the range of confounding influences on carbon monoxide concentrations, most notably underlying trends, seasonality and independent interventions, complicate its interpretation. Its application in assessing the presence or timing of a stepped change in pollution or similar environmental time series data is recommended in its basic form only where the predicted change is large by comparison with other independent influences. The authors believe that further development of the technique beyond this initial study is worthwhile in order to improve the technique's sensitivity.

Effect of Increased Cost-Sharing on Oral Hypoglycemic Use in Five Managed Care Organizations

For patients with a chronic disease, increased cost-sharing for medications may lead to unintended consequences, including reduced use of medications essential for control of their disease. The objective of this study was to estimate the effects of small ($1-6 per 30-day supply), moderate ($7-10), and large (>$10) increases in medication cost-sharing on 12-month trends in oral hypoglycemic (OH) use among adults with type 2 diabetes. We conducted a quasiexperimental study using a time series with comparison group design. Data were obtained from computerized membership, benefit, and pharmacy dispensing data of 5 managed care organizations (MCOs). A total of 13,110 12-month episodes of OH use and a medication cost-sharing increase ("intervention") were matched with 13,110 that had no increase. The dependent variable was OH average daily dose (ADD) standardized to each episode's mean OH ADD in the 6-month preintervention period. The principal independent variable was change in cost per 30-day OH supply between the 6-month pre- and postintervention periods. Effects of changes in cost-sharing on OH ADD were estimated using segmented time series regression. Episodes with >$10 increase in cost-sharing had significantly (alpha=0.05) decreased OH ADD in the postintervention period. At 6 months after this increase, OH ADD had decreased by 18.5% from that predicted from the preintervention trend. Episodes with a $1 to $10 increase in cost-sharing and those with no increase in cost-sharing had significant linear increases in OH use over the 12-month period. Large increases in medication cost-sharing were associated with immediate and persistent reductions in OH use. Small and moderate increases had little effect on OH use in the 6-month period after the increase.

Effects of health maintenance organization coverage of self-monitoring devices on diabetes self-care and glycemic control.

Increasingly, government mandates require insurance coverage of blood glucose monitors and test strips for patients with type 1 and type 2 diabetes. No data exist on the effects of such coverage on self-monitoring of blood glucose (SMBG), medication compliance, or blood glucose control. We evaluated whether a policy providing free blood glucose monitors increased SMBG and whether initiating SMBG was associated with increased regularity of medication use and improved glucose control (hemoglobin A(1c) [HbA(1c)] level). Using interrupted time-series analysis and controlling for preintervention trends, we determined changes in rates of SMBG 2 years before and after the policy among 3219 continuously enrolled patients with diabetes receiving drug therapy within a multispecialty medical group (part of a health maintenance organization) serving approximately 300 000 patients. We also compared changes over time in regularity of medication use (mean days between dispensings) and mean HbA(1c) level among initiators and noninitiators of SMBG. The policy resulted in a small, significant increase in SMBG among insulin-treated patients (n = 1428). Among sulfonylurea-treated patients (n = 1791), the monthly initiation rate of SMBG increased by 14 new patients per 1000 (95% confidence interval [CI], 10 to 17), a doubling of the expected initiation rate. Test strip consumption increased during the first 6 months after the policy by 17.9 strips per cohort member (75% relative increase by 6 months; 95% CI, 50% to 101%). Compared with noninitiators of SMBG, initiators (n = 593) showed sudden, significant improvements in regularity of medication use by 6 months after initiation (-19.5 days between dispensings among those with low refill regularity [95% CI, -27.7 to -11.3]; -9.7 days among those with moderate regularity [95% CI, -12.3 to -7.1]), and in glucose control (-0.63% mean HbA(1c) level [as percentage of total hemoglobin] among those with poor baseline glycemic control [HbA(1c) >10%; 95% CI, -1.14% to -0.12%]). Providing free glucose monitors improved rates of self-monitoring in this health maintenance organization population, possibly by offering an initial incentive for patients to engage in more desirable patterns of care. Initiating SMBG was associated with increased regularity of medication use and a reduction in high blood glucose levels.

Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies.

In an interrupted time series (ITS) design, data are collected at multiple instances over time before and after an intervention to detect whether the intervention has an effect significantly greater than the underlying secular trend. We critically reviewed the methodological quality of ITS designs using studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies). Quality criteria were developed, and data were abstracted from each study. If the primary study analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression. Twenty mass media studies and thirty-eight guideline studies were included. A total of 66% of ITS studies did not rule out the threat that another event could have occurred at the point of intervention. Thirty-three studies were reanalyzed, of which eight had significant preintervention trends. All of the studies were considered "effective" in the original report, but approximately half of the reanalyzed studies showed no statistically significant differences. We demonstrated that ITS designs are often analyzed inappropriately, underpowered, and poorly reported in implementation research. We have illustrated a framework for appraising ITS designs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.

Retrospective drug utilization review, prescribing errors, and clinical outcomes.

Retrospective drug utilization review is required of all state Medicaid programs and is performed by most private-sector prescription programs. However, it has not been shown to improve clinical outcomes or reduce the rate of potential prescribing errors, known as "exceptions." To look for an effect of retrospective drug utilization review on the rate of exceptions and of clinical outcomes in patients with an exception. Longitudinal ecologic study of the rate of exceptions, controlling for preintervention trends and calendar time; and a cohort study of all-cause and cause-specific hospitalizations in patients with an exception, controlling for potential individual-level confounders in 6 Medicaid programs using the same software in the mid-1990s. The rate of exceptions was examined as a function of retrospective drug utilization review implementation. In addition, before-after comparisons were made of the incidence of all-cause and cause-specific hospitalization in patients with exceptions. We found no reduction in the rate of exceptions coincident with retrospective drug utilization review implementation (rate increase, 0.064 exceptions per 1000 prescriptions per month; 95% confidence interval [CI], -0.006 to 0.133). We also found no effect of retrospective drug utilization review on the incidence of all-cause hospitalization (odds ratio, 0.99; 95% CI, 0.98-1.00) or cause-specific hospitalization. These results persisted in multiple subgroup analyses. Study states intervened using physician alerts in between 1% and 25% of exceptions. We were unable to identify an effect of retrospective drug utilization review on the rate of exceptions or on clinical outcomes. Given the lack of evidence for effectiveness, and suggestions from previous research of possible harm, policymakers should consider withdrawing the legislative mandate for retrospective drug utilization review.

EFFECT OF PERMETHRIN-TREATED BED NETS ON THE SPATIAL DISTRIBUTION OF MALARIA VECTORS IN WESTERN KENYA

The effect of insecticide (permethrin)-treated bed nets (ITNs) on the spatial distribution of malaria vectors in neighboring villages lacking ITNs was studied during a randomized controlled trial of ITNs in western Kenya. There was a trend of decreased abundance of Anopheles gambiae with decreasing distance from intervention villages both before (P = 0.027) and after (P = 0.002) introduction of ITNs, but this trend was significantly stronger after ITNs were introduced (P = 0.05). For An. funestus, no pre-intervention trend was observed (P = 0.373), but after the intervention, a trend of decreased abundance with closer proximity to intervention compounds developed (P = 0.027). Reduction in mosquito populations in villages lacking ITNs was most apparent in compounds located within 600 meters of intervention villages. Sporozoite infection rates decreased in control areas following the introduction of ITNs (P < 0.001 for both species), but no spatial association was detected between sporozoite rates and distance to nearest intervention village. We conclude that high coverage of ITNs is associated with a community-wide suppression of mosquito populations that is detectable in neighboring villages lacking ITNs, thereby affording individuals residing in these villages some protection against malaria.