Single-group Interrupted Time Series Analysis Research Articles

Exploring the Impact of Routine Activity and Financial Strain on Fraud Victimization During the COVID-19 Pandemic in Hong Kong

This study examines the impact of the COVID-19 pandemic on fraud victimization in Hong Kong, providing valuable insights beyond the Western context. Drawing on general strain theory and routine activity theory, this research explores the influence of economic recession at the local and adjacent societal levels, as well as residential duration (refers to relative time spent at residences), on fraud victimization in Hong Kong. Utilizing 10 years (120 months) of monthly police-recorded victimization data, this study employs various methodologies, including ARIMA forecasting, single-group interrupted time series analysis (ITSA), and Poisson regression, to explore the impact of the COVID-19 pandemic on fraud victimization in Hong Kong. The ARIMA framework reveals an unexpected and significant increase in fraud victimization during the COVID-19 period, surpassing the predicted levels. The ITSA results demonstrate that the pandemic had a short-term and long-term effect on fraud victimization in Hong Kong. To further understand the factors contributing to this change, a Poisson regression analysis is conducted. The findings highlight the positive and significant impact of residential duration and the unemployment rate in mainland China on fraud victimization, aligning with the propositions of routine activity theory and general strain theory. Limitations and policy implications at both the local and international levels are discussed.

Implementation of Opioid Safety Efforts: Influence of Academic Detailing on Adverse Outcomes Among Patients in the Veterans Health Administration.

The Veterans Health Administration (VA) implemented academic detailing (AD) to support safer opioid prescribing and overdose prevention initiatives. Patient-level data were extracted monthly from VA's electronic health record to evaluate whether AD implementation was associated with changes in all-cause mortality, opioid poisoning inpatient admissions, and opioid poisoning emergency department (ED) visits in an observational cohort of patients with long-term opioid prescriptions (≥45-day supply of opioids 6 months prior to a given month with ≤15 days between prescriptions). A single-group interrupted time series analysis using segmented logistic regression for mortality and Poisson regression for counts of inpatient admissions and ED visits was used to identify whether the level and slope of these outcomes changed in response to AD implementation. Among 955 376 unique patients (19 431 241 person-months), there were 53 369 deaths (29 025 pre-AD; 24 344 post-AD), 1927 opioid poisoning inpatient admissions (610 pre-AD; 1317 post-AD), and 408 opioid poisoning ED visits (207 pre-AD; 201 post-AD). Immediately after AD implementation, there was a 5.8% reduction in the odds of all-cause mortality (95% confidence interval [CI]: 0.897, 0.990). However, patients had a significantly increased incidence rate of inpatient admissions for opioid poisoning immediately after AD implementation (incidence rate ratio = 1.523; 95% CI: 1.118, 2.077). No significant differences in ED visits for opioid poisoning were observed. AD was associated with decreased all-cause mortality but increased inpatient hospitalization for opioid poisoning among patients prescribed long-term opioids. Mechanisms via which AD's efforts influenced opioid-related outcomes should be explored.

Changes in anticoagulant and antiplatelet drug supply at US hospitals before and during the COVID-19 pandemic (2018-2021).

Antithrombotic agents have a role in coronavirus disease 2019 (COVID-19) treatment, but the pandemic disrupted medication supply. This study examined changes in the volume of oral and parenteral anticoagulant and antiplatelet medications at US hospitals during the pandemic. IQVIA National Sales Perspective (NSP) data was used to determine the monthly volume of anticoagulants and antiplatelets purchased at US hospitals between January 2018 and February 2021. Mean monthly medication volumes, reported as extended units (EUs), and year-over-year changes in medication volume were determined. A single-group interrupted time series analysis was used to evaluate changes in the rate of growth of monthly medication volumes before (January 2019-February 2020) and during (March 2020-February 2021) the COVID-19 pandemic. Overall, there was a 43.4% decline in the total volume of anticoagulants and antiplatelets at US hospitals in March 2020, driven by a decrease in heparin volume. Mean monthly volumes decreased significantly (P < 0.05) for parenteral anticoagulants (-106,691,340 EU [95% CI, -200,033,910 to -13,348,780]), oral anticoagulants (-354,800 EU [95% CI, -612,180 to -97,420]), and parenteral antiplatelets (-391,880 EU [95% CI, -535,420 to -248,330]). During the pandemic, the monthly volume of oral anticoagulants, parenteral anticoagulants, and parenteral antiplatelets grew significantly more than in the prepandemic period. This growth was primarily seen in volumes of apixaban, argatroban, enoxaparin, heparin, eptifibatide, and tirofiban. Apixaban and heparin volumes continued a prepandemic uptrend, while argatroban and eptifibatide volumes reversed trend. Rapid changes in anticoagulant and antiplatelet volume at US hospitals during the COVID-19 pandemic highlight the need for institutional protocols to manage fluctuating medication volume demands.

Impact of school closures and reopening on COVID-19 caseload in 6 cities of Pakistan: An Interrupted Time Series Analysis.

Schools were closed all over Pakistan on November 26, 2020 to reduce community transmission of COVID-19 and reopened between January 18 and February 1, 2021. However, these closures were associated with significant economic and social costs, prompting a review of effectiveness of school closures to reduce the spread of COVID-19 infections in a developing country like Pakistan. A single-group interrupted time series analysis (ITSA) was used to measure the impact of school closures, as well as reopening schools, on daily new COVID-19 cases in 6 major cities across Pakistan: Lahore, Karachi, Islamabad, Quetta, Peshawar, and Muzaffarabad. However, any benefits were contingent on continued closure of schools, as cases bounced back once schools reopened. School closures are associated with a clear and statistically significant reduction in COVID-19 cases by 0.07 to 0.63 cases per 100,000 population, while reopening schools is associated with a statistically significant increase. Lahore is an exception to the effect of school closures, but it too saw an increase in COVID-19 cases after schools reopened in early 2021. We show that closing schools was a viable policy option, especially before vaccines became available. However, its social and economic costs must also be considered.

The effect of non-pharmaceutical policy interventions on COVID-19 transmission across three cities in Colombia.

Governments across the globe have implemented different strategies to handle the COVID-19 pandemic. A national mandatory quarantine was the most applied policy tool. While there are studies that tested the effectiveness of a national mandatory quarantine, the question about the effectiveness of additional quarantine policies is not yet answered. In this study we focus on three large cities in Colombia (Bogota, Medellin and Cali) with similar socio-economic conditions but made use of different COVID-19 prevention measures. We examine whether different non-pharmaceutical policy interventions (NPIs) conducted in these three cities are effective against the spread of the COVID-19 pandemic. We inspect the effect of the quarantine policies restricting exit from home by sex, ID number, whereby only Bogota implemented the restriction to leave the home according to sex followed by a restriction according to ID number, and Medellin and Cali implemented a restriction by ID number only. Data for the analysis are obtained from the National Administrative Department of Statistics of Colombia [Departamento Administrativo Nacional de Estadística (DANE)]. The data on pandemic severity is measured by the number of confirmed COVID-19 cases per city. We conduct single-group interrupted time series analysis (ITSA) to examine differences in the extent of the pandemic severity in Bogota, Medellin and Cali. We found that NPIs in all three Colombian cities had a positive effect on slowing the spread of the pandemic.

The impact of National Centralised Drug Procurement policy on the use of policy-related original and generic drugs in China.

To evaluate the effects of the first round of National Centralised Drug Procurement pilot (so-called '4+7' policy) on the use of policy-related original and generic drugs. This study used drug purchasing order data from the Centralised Drug Procurement Survey in Shenzhen 2019, covering 24months from January 2018 to December 2019. '4+7' policy-related drugs were selected as study samples, including 25 drugs in the '4+7' procurement list and 69 alternative drugs that have an alternative relationship with '4+7' List drugs in clinical use. '4+7' List drugs were then divided into bid-winning and bid-non-winning products according to the bidding results. Included drugs were sorted into original and generic drugs. Purchase volume, expenditures, and daily costs were selected as outcome variables, and were measured using Defined Daily Doses (DDDs), Chinese Yuan (CNY), and Defined Daily Drug cost (DDDc). A single-group Interrupted Time Series analysis was adopted to quantify policy effect. After policy intervention, the overall policy-related original drugs significantly decreased by 0.39 CNY (95% CI: -0.62 to -0.17, p<0.01) in DDDc, 5949.36 thousand DDDs (95% CI: -8276.67 to -3622.05, p<0.001) in volume, and 31,575.08 thousand CNY (95% CI: -41,812.68 to -21,337.49, p<0.001) in expenditures. The volume proportion of generic drugs increased from 78.6% to 91.0%, and the expenditure proportion of increased from 30.9% to 49.8%. '4+7' policy promoted the substitution use of domestic generics against original branded drugs and played positive effects on drug price cut and medication burden reduction. The proportion of original branded drugs and generics that passed generic consistency evaluation significantly increased after policy intervention, indicating the improvement of the overall quality level of drug use in China.

Monthly biological larviciding associated with a tenfold decrease in larval density in fish farming ponds and reduced community-wide malaria incidence in northwestern Brazil

BackgroundLarvicides are typically applied to fixed and findable mosquito breeding sites, such as fish farming ponds used in commercial aquaculture, to kill immature forms and thereby reduce the size of adult malaria vector populations. However, there is little evidence suggesting that larviciding may suppress community-wide malaria transmission outside Africa. Here, we tested whether the biological larvicide VectoMax FG applied at monthly intervals to fish farming ponds can reduce malaria incidence in Amazonian Brazil.MethodsThis study was carried out in Vila Assis Brasil (VAB; population 1700), a peri-urban malaria hotspot in northwestern Brazil with a baseline annual parasite incidence of 553 malaria cases per 1000 inhabitants. The intervention consisted of monthly treatments with 20 kg/ha of VectoMax FG of all water-filled fish ponds in VAB (n ranging between 167 and 170) with a surface area between 20 and 8000 m2, using knapsack power mistblowers. We used single-group interrupted time-series analysis to compare monthly larval density measurements in fish ponds during a 14-month pre-intervention period (September 2017–October 2018), with measurements made during November 2018–October 2019 and shortly after the 12-month intervention (November 2019). We used interrupted time-series analysis with a comparison group to contrast the malaria incidence trends in VAB and nearby nonintervention localities before and during the intervention.ResultsAverage larval densities decreased tenfold in treated fish farming ponds, from 0.467 (95% confidence interval [CI], 0.444–0.490) anopheline larvae per dip pre-intervention (September 2017–October 2018) to 0.046 (95% CI, 0.041–0.051) larvae per dip during (November 2018–October 2019) and shortly after the intervention (November 2019). Average malaria incidence rates decreased by 0.08 (95% CI, 0.04–0.11) cases per 100 person-months (P < 0.0001) during the intervention in VAB and remained nearly unchanged in comparison localities. We estimate that the intervention averted 24.5 (95% CI, 6.2–42.8) malaria cases in VAB between January and December 2019.ConclusionsRegular larviciding is associated with a dramatic decrease in larval density and a modest but significant decrease in community-wide malaria incidence. Larviciding may provide a valuable complementary vector control strategy in commercial aquaculture settings across the Amazon.Graphical abstract

Opioid-related Emergency Department Visits During COVID-19 in a Large Health System.

Objective:Multiple states have reported increases in opioid overdose deaths during the coronavirus disease 2019 (COVID-19) pandemic, however little is known about opioid-related presentations to the emergency department (ED).Methods:This was a time series analysis of visits to 7 EDs in greater Chicago, Illinois from October 20, 2019 to July 25, 2020. We compared the number of ED visits for opioid-related diagnoses in the time period preceding the World Health Organization pandemic declaration (prepandemic period, October 20, 2019–July 3, 2020) to the time period following the World Health Organization declaration (pandemic period, March 8, 2020 to July 25, 2020) using a single-group interrupted time series analysis with Newey-West standard errors. We also present data on alcohol-related ED visits for comparison.Results:We evaluated a total of 177,405 visits across the 7 EDs during the study period. The mean number of weekly ED visits in the prepandemic and pandemic periods was 4841 and 4029 weekly visits, respectively. In the interrupted time series analysis, there was no significant immediate effect of the pandemic start on opioid-related ED visits (−0.44 visits per 1000 ED visits, 95% CI −2.47 to 1.58, P = 0.66), however, there was a significant immediate effect of the pandemic start on alcohol-related ED visits (−4.1, 95% CI: −8.25 to −0.01, P < 0.05).Conclusions:Despite reductions in overall ED visit volumes and alcohol-related visits during COVID-19, the number of opioid-related visits was not significantly reduced during the early pandemic. These data reinforce the need to provide comprehensive treatment services for opioid use disorder during the co-occurring COVID-19 and opioid crises.

Hospital performance and payment: impact of integrating pay-for-performance on healthcare effectiveness in Lebanon.

Background: In 2014 the Lebanese Ministry of Public Health integrated pay-for-performance into setting hospital reimbursement tiers, to provide hospitalization service coverage for the majority of the Lebanese population. This policy was intended to improve effectiveness by decreasing unnecessary hospitalizations, and improve fairness by including risk-adjustment in setting hospital performance scores. Methods: We applied a systematic approach to assess the impact of the new policy on hospital performance. The main impact measure was a national casemix index, calculated across 2011-2016 using medical discharge and surgical procedure codes. A single-group interrupted time series analysis model with Newey ordinary least squares regression was estimated, including adjustment for seasonality, and stratified by case type. Code-level analysis was used to attribute and explain changes in casemix index due to specific diagnoses and procedures. Results: Our final model included 1,353,025 cases across 146 hospitals with a post-intervention lag-time of two months and seasonality adjustment. Among medical cases the intervention resulted in a positive casemix index trend of 0.11% per month (coefficient 0.002, CI 0.001-0.003), and a level increase of 2.25% (coefficient 0.022, CI 0.005-0.039). Trend changes were attributed to decreased cases of diarrhea and gastroenteritis, abdominal and pelvic pain, essential hypertension and fever of unknown origin. A shift from medium to short-stay cases for specific diagnoses was also detected. Level changes were attributed to improved coding practices, particularly for breast cancer, leukemia and chemotherapy. No impact on surgical casemix index was found. Conclusions: The 2014 policy resulted in increased healthcare effectiveness, by increasing the casemix index of hospitals contracted by the Ministry. This increase was mainly attributed to decreased unnecessary hospitalizations and was accompanied by improved medical discharge coding practices. Integration of pay-for-performance within a healthcare system may contribute to improving effectiveness. Effective hospital regulation can be achieved through systematic collection and analysis of routine data.

Hospital performance and payment: impact of integrating pay-for-performance on healthcare effectiveness in Lebanon

Background: In 2014 the Lebanese Ministry of Public Health integrated pay-for-performance into setting hospital reimbursement tiers, to provide hospitalization service coverage for the majority of the Lebanese population. This policy was intended to improve effectiveness by decreasing unnecessary hospitalizations, and improve fairness by including risk-adjustment in setting hospital performance scores. Methods: We applied a systematic approach to assess the impact of the new policy on hospital performance. The main impact measure was a national casemix index, calculated across 2011-2016 using medical discharge and surgical procedure codes. A single-group interrupted time series analysis model with Newey ordinary least squares regression was estimated, including adjustment for seasonality, and stratified by case type. Code-level analysis was used to attribute and explain changes in casemix index due to specific diagnoses and procedures. Results: Our final model included 1,353,025 cases across 146 hospitals with a post-intervention lag-time of two months and seasonality adjustment. Among medical cases the intervention resulted in a positive casemix index trend of 0.11% per month (coefficient 0.002, CI 0.001-0.003), and a level increase of 2.25% (coefficient 0.022, CI 0.005-0.039). Trend changes were attributed to decreased cases of diarrhea and gastroenteritis, abdominal and pelvic pain, essential hypertension and fever of unknown origin. A shift from medium to short-stay cases for specific diagnoses was also detected. Level changes were attributed to improved coding practices, particularly for breast cancer, leukemia and chemotherapy. No impact on surgical casemix index was found. Conclusions: The 2014 policy resulted in increased healthcare effectiveness, by increasing the casemix index of hospitals contracted by the Ministry. This increase was mainly attributed to decreased unnecessary hospitalizations and was accompanied by improved medical discharge coding practices. Integration of pay-for-performance within a healthcare system may contribute to improving effectiveness. Effective hospital regulation can be achieved through systematic collection and analysis of routine data.

Comparison of inpatient distribution amongst different medical alliances in a county: a longitudinal study on a healthcare reform in rural China

BackgroundChina has launched the medical alliances (MAs) reform to drive the development of primary medical institutions and decrease health inequality in rural areas. Three different types of MAs were built to promote township hospitals in Y County. This study aims to evaluate the actual effect of China’s MAs reform in rural areas on inpatient distribution especially amongst different types of MAs.MethodsWe obtain 2008–2015 claims data from the New Cooperative Medical Scheme (NCMS) in Y County, Hubei Province of China. We consider January 2008–December 2010 as the pre-reform period and January 2011–December 2015 as the post-reform period.We use independent sample t-test and single-group interrupted time series analysis (ITSA) to compare the number of inpatients per month in the three MAs, including three county and 10 township hospitals before and after the reform. We use paired t-test and multiple-group ITSA between seven township hospitals within MAs and seven township hospitals outside MAs.ResultsThe MAs reform in Y County increased the number of inpatients in county and township hospitals within MAs. After the reform, the number of inpatients per month in county hospitals had an upward trend, with a slope of 31.01 person/month (P < 0.000). Approximately 19.99 new inpatients were admitted to township hospitals monthly after the reform (P < 0.000). Furthermore, township hospitals within MAs had a substantial increase in the number of inpatients (10.45 new inpatients monthly) compared with those outside MAs.ConclusionThe MAs reform in Y County significantly improved the capability of medical institutions within MAs. After the reform, township hospitals within MAs had greater development advantages than those outside MAs. However, it also caused further imbalance in the county region, which contained the new health inequality risk.

Hospital performance and payment: impact of integrating pay-for-performance on healthcare effectiveness in Lebanon.

Background: In 2014 the Lebanese Ministry of Public Health integrated pay-for-performance into setting hospital reimbursement tiers, to provide hospitalization service coverage for the majority of the Lebanese population. This policy was intended to improve effectiveness by decreasing unnecessary hospitalizations, and improve fairness by including risk-adjustment in setting hospital performance scores. Methods: We applied a systematic approach to assess the impact of the new policy on hospital performance. The main impact measure was a national casemix index, calculated across 2011-2016 using medical discharge and surgical procedure codes. A single-group interrupted time series analysis model with Newey ordinary least squares regression was estimated, including adjustment for seasonality, and stratified by case type. Code-level analysis was used to attribute and explain changes in casemix index due to specific diagnoses and procedures. Results: Our final model included 1,353,025 cases across 146 hospitals with a post-intervention lag-time of two months and seasonality adjustment. Among medical cases the intervention resulted in a positive casemix index trend of 0.11% per month (coefficient 0.002, CI 0.001-0.003), and a level increase of 2.25% (coefficient 0.022, CI 0.005-0.039). Trend changes were attributed to decreased cases of diarrhea and gastroenteritis, abdominal and pelvic pain, essential hypertension and fever of unknown origin. A shift from medium to short-stay cases for specific diagnoses was also detected. Level changes were attributed to improved coding practices, particularly for breast cancer, leukemia and chemotherapy. No impact on surgical casemix index was found. Conclusions: The 2014 policy resulted in increased healthcare effectiveness, by increasing the casemix index of hospitals contracted by the Ministry. This increase was mainly attributed to decreased unnecessary hospitalizations and was accompanied by improved medical discharge coding practices. Integration of pay-for-performance within a healthcare system may contribute to improving effectiveness. Effective hospital regulation can be achieved through systematic collection and analysis of routine data.

Accuracy and Effects of Clinical Decision Support Systems Integrated With BMJ Best Practice-Aided Diagnosis: Interrupted Time SeriesStudy.

BackgroundClinical decision support systems (CDSS) are an integral component of health information technologies and can assist disease interpretation, diagnosis, treatment, and prognosis. However, the utility of CDSS in the clinic remains controversial.ObjectiveThe aim is to assess the effects of CDSS integrated with British Medical Journal (BMJ) Best Practice–aided diagnosis in real-world research.MethodsThis was a retrospective, longitudinal observational study using routinely collected clinical diagnosis data from electronic medical records. A total of 34,113 hospitalized patient records were successively selected from December 2016 to February 2019 in six clinical departments. The diagnostic accuracy of the CDSS was verified before its implementation. A self-controlled comparison was then applied to detect the effects of CDSS implementation. Multivariable logistic regression and single-group interrupted time series analysis were used to explore the effects of CDSS. The sensitivity analysis was conducted using the subgroup data from January 2018 to February 2019.ResultsThe total accuracy rates of the recommended diagnosis from CDSS were 75.46% in the first-rank diagnosis, 83.94% in the top-2 diagnosis, and 87.53% in the top-3 diagnosis in the data before CDSS implementation. Higher consistency was observed between admission and discharge diagnoses, shorter confirmed diagnosis times, and shorter hospitalization days after the CDSS implementation (all P<.001). Multivariable logistic regression analysis showed that the consistency rates after CDSS implementation (OR 1.078, 95% CI 1.015-1.144) and the proportion of hospitalization time 7 days or less (OR 1.688, 95% CI 1.592-1.789) both increased. The interrupted time series analysis showed that the consistency rates significantly increased by 6.722% (95% CI 2.433%-11.012%, P=.002) after CDSS implementation. The proportion of hospitalization time 7 days or less significantly increased by 7.837% (95% CI 1.798%-13.876%, P=.01). Similar results were obtained in the subgroup analysis.ConclusionsThe CDSS integrated with BMJ Best Practice improved the accuracy of clinicians’ diagnoses. Shorter confirmed diagnosis times and hospitalization days were also found to be associated with CDSS implementation in retrospective real-world studies. These findings highlight the utility of artificial intelligence-based CDSS to improve diagnosis efficiency, but these results require confirmation in future randomized controlled trials.

The limits of single-group interrupted time series analysis in assessing the impact of smoke-free laws on short-term mortality

BackgroundDecreases in circulatory/respiratory morbimortality after the January-2006 Spanish partial smoke-free law have been found using designs without control groups, such as single-group interrupted time series (ITS), which are prone to biases. The aim was to reassess the law’s impact on mortality using ITS designs with robustness checks. MethodsA comprehensive cohort of people aged ≥25 in each calendar-year of 2002–2007, living in 13 of 18 Spanish regions, was followed up between 01/2002 and 12/2007. The law included a smoking ban in indoor public and workplaces, allowing exceptions in catering, hospitality and leisure venues, and other interventions. Post-law changes in monthly coronary/respiratory mortality were estimated using segmented regression, adjusting for relevant covariates, including seasonality, extreme temperatures, influenza incidence and air pollution. The validity of results was assessed using control outcomes, hypothetical law dates, and non-equivalent control groups, analysing their results as difference-in-differences (DID) designs. ResultsSignificant immediate post-law decreases in coronary, respiratory and non-tobacco-related mortality were observed among people aged ≥70. A significant immediate post-law decrease in respiratory mortality (−12.7%) was also observed among people age 25–69, although this was neutralized by a subsequent upward trend before 1.5 years. More favourable post-law changes in coronary/respiratory mortality among the target (people aged 25–69) than control groups (people aged ≥70 or women aged ≥80) were not identified in DID designs. Establishing hypothetical law dates, immediate decreases began in February/March 2005 with maxima between April and July 2005. ConclusionsAfter robustness checks, the results do not support a clear positive impact of the 2006 Spanish smoke-free law on short-term coronary/respiratory mortality. The favourable immediate changes observed pre- and post-law could derive mainly from the harvesting effect of the January-2005 cold wave. This highlights the risks of assessing the impact of health interventions using both morbimortality outcomes and designs without a control group and adequate robustness checks.

Now trending: Coping with non-parallel trends in difference-in-differences analysis.

Difference-in-differences (DID) analysis is used widely to estimate the causal effects of health policies and interventions. A critical assumption in DID is "parallel trends": that pre-intervention trends in outcomes are the same between treated and comparison groups. To date, little guidance has been available to researchers who wish to use DID when the parallel trends assumption is violated. Using a Monte Carlo simulation experiment, we tested the performance of several estimators (standard DID; DID with propensity score matching; single-group interrupted time-series analysis; and multi-group interrupted time-series analysis) when the parallel trends assumption is violated. Using nationwide data from US hospitals (n = 3737) for seven data periods (four pre-interventions and three post-interventions), we used alternative estimators to evaluate the effect of a placebo intervention on common outcomes in health policy (clinical process quality and 30-day risk-standardized mortality for acute myocardial infarction, heart failure, and pneumonia). Estimator performance was assessed using mean-squared error and estimator coverage. We found that mean-squared error values were considerably lower for the DID estimator with matching than for the standard DID or interrupted time-series analysis models. The DID estimator with matching also had superior performance for estimator coverage. Our findings were robust across all outcomes evaluated.

Using forecast modelling to evaluate treatment effects in single-group interrupted time series analysis.

Interrupted time series analysis (ITSA) is an evaluation methodology in which a single treatment unit's outcome is studied serially over time and the intervention is expected to "interrupt" the level and/or trend of that outcome. ITSA is commonly evaluated using methods which may produce biased results if model assumptions are violated. In this paper, treatment effects are alternatively assessed by using forecasting methods to closely fit the preintervention observations and then forecast the post-intervention trend. A treatment effect may be inferred if the actual post-intervention observations diverge from the forecasts by some specified amount. The forecasting approach is demonstrated using the effect of California's Proposition 99 for reducing cigarette sales. Three forecast models are fit to the preintervention series-linear regression (REG), Holt-Winters (HW) non-seasonal smoothing, and autoregressive moving average (ARIMA)-and forecasts are generated into the post-intervention period. The actual observations are then compared with the forecasts to assess intervention effects. The preintervention data were fit best by HW, followed closely by ARIMA. REG fit the data poorly. The actual post-intervention observations were above the forecasts in HW and ARIMA, suggesting no intervention effect, but below the forecasts in the REG (suggesting a treatment effect), thereby raising doubts about any definitive conclusion of a treatment effect. In a single-group ITSA, treatment effects are likely to be biased if the model is misspecified. Therefore, evaluators should consider using forecast models to accurately fit the preintervention data and generate plausible counterfactual forecasts, thereby improving causal inference of treatment effects in single-group ITSA studies.

Persistent threats to validity in single-group interrupted time series analysis with a cross over design.

The basic single-group interrupted time series analysis (ITSA) design has been shown to be susceptible to the most common threat to validity-history-the possibility that some other event caused the observed effect in the time series. A single-group ITSA with a crossover design (in which the intervention is introduced and withdrawn 1 or more times) should be more robust. In this paper, we describe and empirically assess the susceptibility of this design to bias from history. Time series data from 2 natural experiments (the effect of multiple repeals and reinstatements of Louisiana's motorcycle helmet law on motorcycle fatalities and the association between the implementation and withdrawal of Gorbachev's antialcohol campaign with Russia's mortality crisis) are used to illustrate that history remains a threat to ITSA validity, even in a crossover design. Both empirical examples reveal that the single-group ITSA with a crossover design may be biased because of history. In the case of motorcycle fatalities, helmet laws appeared effective in reducing mortality (while repealing the law increased mortality), but when a control group was added, it was shown that this trend was similar in both groups. In the case of Gorbachev's antialcohol campaign, only when contrasting the results against those of a control group was the withdrawal of the campaign found to be the more likely culprit in explaining the Russian mortality crisis than the collapse of the Soviet Union. Even with a robust crossover design, single-group ITSA models remain susceptible to bias from history. Therefore, a comparable control group design should be included, whenever possible.

Challenges to validity in single-group interrupted time series analysis.

Single-group interrupted time series analysis (ITSA) is a popular evaluation methodology in which a single unit of observation is studied; the outcome variable is serially ordered as a time series, and the intervention is expected to "interrupt" the level and/or trend of the time series, subsequent to its introduction. The most common threat to validity is history-the possibility that some other event caused the observed effect in the time series. Although history limits the ability to draw causal inferences from single ITSA models, it can be controlled for by using a comparable control group to serve as the counterfactual. Time series data from 2 natural experiments (effect of Florida's 2000 repeal of its motorcycle helmet law on motorcycle fatalities and California's 1988 Proposition 99 to reduce cigarette sales) are used to illustrate how history biases results of single-group ITSA results-as opposed to when that group's results are contrasted to those of a comparable control group. In the first example, an external event occurring at the same time as the helmet repeal appeared to be the cause of a rise in motorcycle deaths, but was only revealed when Florida was contrasted with comparable control states. Conversely, in the second example, a decreasing trend in cigarette sales prior to the intervention raised question about a treatment effect attributed to Proposition 99, but was reinforced when California was contrasted with comparable control states. Results of single-group ITSA should be considered preliminary, and interpreted with caution, until a more robust study design can be implemented.

Using machine learning to identify structural breaks in single-group interrupted time series designs.

Single-group interrupted time series analysis (ITSA) is a popular evaluation methodology in which a single unit of observation is being studied, the outcome variable is serially ordered as a time series and the intervention is expected to 'interrupt' the level and/or trend of the time series, subsequent to its introduction. Given that the internal validity of the design rests on the premise that the interruption in the time series is associated with the introduction of the treatment, treatment effects may seem less plausible if a parallel trend already exists in the time series prior to the actual intervention. Thus, sensitivity analyses should focus on detecting structural breaks in the time series before the intervention. In this paper, we introduce a machine-learning algorithm called optimal discriminant analysis (ODA) as an approach to determine if structural breaks can be identified in years prior to the initiation of the intervention, using data from California's 1988 voter-initiated Proposition 99 to reduce smoking rates. The ODA analysis indicates that numerous structural breaks occurred prior to the actual initiation of Proposition 99 in 1989, including perfect structural breaks in 1983 and 1985, thereby casting doubt on the validity of treatment effects estimated for the actual intervention when using a single-group ITSA design. Given the widespread use of ITSA for evaluating observational data and the increasing use of machine-learning techniques in traditional research, we recommend that structural break sensitivity analysis is routinely incorporated in all research using the single-group ITSA design.