Estimating effects of health policy interventions using interrupted time-series analyses: a simulation study
Standard
Estimating effects of health policy interventions using interrupted time-series analyses: a simulation study. / Jiang, Huan; Feng, Xinyang; Lange, Shannon; Tran, Alexander; Manthey, Jakob; Rehm, Jürgen.
In: BMC MED RES METHODOL, Vol. 22, No. 1, 235, 31.08.2022.Research output: SCORING: Contribution to journal › SCORING: Journal article › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Estimating effects of health policy interventions using interrupted time-series analyses: a simulation study
AU - Jiang, Huan
AU - Feng, Xinyang
AU - Lange, Shannon
AU - Tran, Alexander
AU - Manthey, Jakob
AU - Rehm, Jürgen
N1 - © 2022. The Author(s).
PY - 2022/8/31
Y1 - 2022/8/31
N2 - BACKGROUND: A classic methodology used in evaluating the impact of health policy interventions is interrupted time-series (ITS) analysis, applying a quasi-experimental design that uses both pre- and post-policy data without randomization. In this paper, we took a simulation-based approach to estimating intervention effects under different assumptions.METHODS: Each of the simulated mortality rates contained a linear time trend, seasonality, autoregressive, and moving-average terms. The simulations of the policy effects involved three scenarios: 1) immediate-level change only, 2) immediate-level and slope change, and 3) lagged-level and slope change. The estimated effects and biases of these effects were examined via three matched generalized additive mixed models, each of which used two different approaches: 1) effects based on estimated coefficients (estimated approach), and 2) effects based on predictions from models (predicted approach). The robustness of these two approaches was further investigated assuming misspecification of the models.RESULTS: When one simulated dataset was analyzed with the matched model, the two analytical approaches produced similar estimates. However, when the models were misspecified, the number of deaths prevented, estimated using the predicted vs. estimated approaches, were very different, with the predicted approach yielding estimates closer to the real effect. The discrepancy was larger when the policy was applied early in the time-series.CONCLUSION: Even when the sample size appears to be large enough, one should still be cautious when conducting ITS analyses, since the power also depends on when in the series the intervention occurs. In addition, the intervention lagged effect needs to be fully considered at the study design stage (i.e., when developing the models).
AB - BACKGROUND: A classic methodology used in evaluating the impact of health policy interventions is interrupted time-series (ITS) analysis, applying a quasi-experimental design that uses both pre- and post-policy data without randomization. In this paper, we took a simulation-based approach to estimating intervention effects under different assumptions.METHODS: Each of the simulated mortality rates contained a linear time trend, seasonality, autoregressive, and moving-average terms. The simulations of the policy effects involved three scenarios: 1) immediate-level change only, 2) immediate-level and slope change, and 3) lagged-level and slope change. The estimated effects and biases of these effects were examined via three matched generalized additive mixed models, each of which used two different approaches: 1) effects based on estimated coefficients (estimated approach), and 2) effects based on predictions from models (predicted approach). The robustness of these two approaches was further investigated assuming misspecification of the models.RESULTS: When one simulated dataset was analyzed with the matched model, the two analytical approaches produced similar estimates. However, when the models were misspecified, the number of deaths prevented, estimated using the predicted vs. estimated approaches, were very different, with the predicted approach yielding estimates closer to the real effect. The discrepancy was larger when the policy was applied early in the time-series.CONCLUSION: Even when the sample size appears to be large enough, one should still be cautious when conducting ITS analyses, since the power also depends on when in the series the intervention occurs. In addition, the intervention lagged effect needs to be fully considered at the study design stage (i.e., when developing the models).
KW - Computer Simulation
KW - Health Policy
KW - Humans
KW - Interrupted Time Series Analysis
KW - Research Design
KW - Sample Size
U2 - 10.1186/s12874-022-01716-4
DO - 10.1186/s12874-022-01716-4
M3 - SCORING: Journal article
C2 - 36045338
VL - 22
JO - BMC MED RES METHODOL
JF - BMC MED RES METHODOL
SN - 1471-2288
IS - 1
M1 - 235
ER -