Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression
Standard
Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression. / Schuster, Raphael; Schreyer, Manuela Larissa; Kaiser, Tim; Berger, Thomas; Klein, Jan Philipp; Moritz, Steffen; Laireiter, Anton-Rupert; Trutschnig, Wolfgang.
in: INTERNET INTERV, Jahrgang 20, 04.2020, S. 100313.Publikationen: SCORING: Beitrag in Fachzeitschrift/Zeitung › SCORING: Zeitschriftenaufsatz › Forschung › Begutachtung
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression
AU - Schuster, Raphael
AU - Schreyer, Manuela Larissa
AU - Kaiser, Tim
AU - Berger, Thomas
AU - Klein, Jan Philipp
AU - Moritz, Steffen
AU - Laireiter, Anton-Rupert
AU - Trutschnig, Wolfgang
N1 - © 2020 The Authors.
PY - 2020/4
Y1 - 2020/4
N2 - Smartphone-based devices are increasingly recognized to assess disease symptoms in daily life (e.g. ecological momentary assessment, EMA). Despite this development in digital psychiatry, clinical trials are mainly based on point assessments of psychopathology. This study investigated expectable increases in statistical power by intense assessment in randomized controlled trials (RCTs). A simulation study, based on three scenarios and several empirical data sets, estimated power gains of two- or fivefold pre-post-assessment. For each condition, data sets of various effect sizes were generated, and AN(C)OVAs were applied to the sample of interest (N = 50-N = 200). Power increases ranged from 6% to 92%, with higher gains in more underpowered scenarios and with higher number of repeated assessments. ANCOVA profited from a more precise estimation of the baseline covariate, resulting in additional gains in statistical power. Fivefold pre-post EMA resulted in highest absolute statistical power and clearly outperformed traditional questionnaire assessments. For example, ANCOVA of automatized PHQ-9 questionnaire data resulted in absolute power of 55 (for N = 200 and d = 0.3). Fivefold EMA, however, resulted in power of 88.9. Non-parametric and multi-level analyses resulted in comparable outcomes. Besides providing psychological treatment, digital mental health can help optimizing sensitivity in RCT-based research. Intense assessment appears advisable whenever psychopathology needs to be assessed with high precision at pre- and post-assessment (e.g. small sample sizes, small treatment effects, or when applying optimization problems like machine learning). First empiric studies are promising, but more evidence is needed. Simulations for various effects and a short guide for popular power software are provided for study planning.
AB - Smartphone-based devices are increasingly recognized to assess disease symptoms in daily life (e.g. ecological momentary assessment, EMA). Despite this development in digital psychiatry, clinical trials are mainly based on point assessments of psychopathology. This study investigated expectable increases in statistical power by intense assessment in randomized controlled trials (RCTs). A simulation study, based on three scenarios and several empirical data sets, estimated power gains of two- or fivefold pre-post-assessment. For each condition, data sets of various effect sizes were generated, and AN(C)OVAs were applied to the sample of interest (N = 50-N = 200). Power increases ranged from 6% to 92%, with higher gains in more underpowered scenarios and with higher number of repeated assessments. ANCOVA profited from a more precise estimation of the baseline covariate, resulting in additional gains in statistical power. Fivefold pre-post EMA resulted in highest absolute statistical power and clearly outperformed traditional questionnaire assessments. For example, ANCOVA of automatized PHQ-9 questionnaire data resulted in absolute power of 55 (for N = 200 and d = 0.3). Fivefold EMA, however, resulted in power of 88.9. Non-parametric and multi-level analyses resulted in comparable outcomes. Besides providing psychological treatment, digital mental health can help optimizing sensitivity in RCT-based research. Intense assessment appears advisable whenever psychopathology needs to be assessed with high precision at pre- and post-assessment (e.g. small sample sizes, small treatment effects, or when applying optimization problems like machine learning). First empiric studies are promising, but more evidence is needed. Simulations for various effects and a short guide for popular power software are provided for study planning.
U2 - 10.1016/j.invent.2020.100313
DO - 10.1016/j.invent.2020.100313
M3 - SCORING: Journal article
C2 - 32215257
VL - 20
SP - 100313
JO - INTERNET INTERV
JF - INTERNET INTERV
SN - 2214-7829
ER -