Clinical and methodological implications for research elements in systematic reviews on COVID-19 treatment were often unstructured and under-reported: a metaresearch study
Standard
Clinical and methodological implications for research elements in systematic reviews on COVID-19 treatment were often unstructured and under-reported: a metaresearch study. / Siemens, Waldemar; Bantle, Gina; Mahler, Sonja; Nothacker, Julia; Stadelmaier, Julia; Bitzer, Eva Maria; Schmucker, Christine; Meerpohl, Joerg J.
in: J CLIN EPIDEMIOL, Jahrgang 166, 111236, 02.2024, S. 111236.Publikationen: SCORING: Beitrag in Fachzeitschrift/Zeitung › SCORING: Zeitschriftenaufsatz › Forschung › Begutachtung
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Clinical and methodological implications for research elements in systematic reviews on COVID-19 treatment were often unstructured and under-reported: a metaresearch study
AU - Siemens, Waldemar
AU - Bantle, Gina
AU - Mahler, Sonja
AU - Nothacker, Julia
AU - Stadelmaier, Julia
AU - Bitzer, Eva Maria
AU - Schmucker, Christine
AU - Meerpohl, Joerg J
PY - 2024/2
Y1 - 2024/2
N2 - OBJECTIVES: Numerous systematic reviews (SRs) have been published in the first months of the COVID-19 pandemic and clinical trials were designed rapidly highlighting the importance of informative implications for research (IfRs) sections in SRs. IfR is one item of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 checklist and the Cochrane Handbook suggests considering population, intervention, control, outcome (PICO) and Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) domains when developing IfR. We aimed (1) to assess whether SRs on COVID-19 treatments included any IfR statements and, for SRs with an IfR statement, (2) to examine which elements informed the IfR statement.STUDY DESIGN AND SETTING: We conducted a metaresearch study based on SRs on COVID-19 treatment identified in the Living OVerview of the Evidence COVID-19 database in May 2021 as part of another research project (CRD42021240423). We defined an IfR statement as at least one sentence that contained at least one bit of information that could be informative for planning future research. We extracted any IfR statements anywhere in the SRs on predefined IfR variables, in particular PICO elements, study design, and concepts underlying GRADE domains. Three authors extracted data independently after piloting the data extraction form. We resolved discrepancies in weekly discussions to ensure a high-quality data extraction.RESULTS: We included 326 SRs, of which 284 SRs (87.1%) stated IfR. Of these 284 SRs, 201 (70.8%) reported using Preferred Reporting Items for Systematic Reviews and Meta-Analyses and 66 (23.2%) using GRADE. IfR statements (n = 284) addressing PICO were unstructured and commonly reported 'population' (n = 195, 68.7%), 'intervention' (n = 242, 85.2%), and 'outcome' (n = 127, 44.7%) but not 'control' (n = 29, 10.2%). Concepts underlying GRADE domains were infrequently reported in IfR statements of SRs (n = 284): 'risk of bias' (n = 14, 4.9%), 'imprecision' (n = 8, 2.8%), 'inconsistency' (n = 7, 2.5%), 'publication bias' (n = 3, 1.1%), and 'indirectness' (n = 1, 0.4%). Additional IfR elements mentioned in IfR were 'better reporting' of future studies (n = 17, 6.0%) and 'standardization of procedures in clinical trials' (n = 12, 4.2%).CONCLUSION: Almost 90% of SRs on COVID-19 treatments reported IfR. IfR statements addressing PICO were unstructured across SRs and concepts underlying GRADE were rarely reported to inform IfR. Further work is needed to assess generalizability beyond COVID-19 and to define more precisely which IfR elements should be considered, and how they should be reported in SRs of interventions. Until then, considering PICO elements and concepts underlying GRADE to derive IfR seems to be a sensible starting point.
AB - OBJECTIVES: Numerous systematic reviews (SRs) have been published in the first months of the COVID-19 pandemic and clinical trials were designed rapidly highlighting the importance of informative implications for research (IfRs) sections in SRs. IfR is one item of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 checklist and the Cochrane Handbook suggests considering population, intervention, control, outcome (PICO) and Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) domains when developing IfR. We aimed (1) to assess whether SRs on COVID-19 treatments included any IfR statements and, for SRs with an IfR statement, (2) to examine which elements informed the IfR statement.STUDY DESIGN AND SETTING: We conducted a metaresearch study based on SRs on COVID-19 treatment identified in the Living OVerview of the Evidence COVID-19 database in May 2021 as part of another research project (CRD42021240423). We defined an IfR statement as at least one sentence that contained at least one bit of information that could be informative for planning future research. We extracted any IfR statements anywhere in the SRs on predefined IfR variables, in particular PICO elements, study design, and concepts underlying GRADE domains. Three authors extracted data independently after piloting the data extraction form. We resolved discrepancies in weekly discussions to ensure a high-quality data extraction.RESULTS: We included 326 SRs, of which 284 SRs (87.1%) stated IfR. Of these 284 SRs, 201 (70.8%) reported using Preferred Reporting Items for Systematic Reviews and Meta-Analyses and 66 (23.2%) using GRADE. IfR statements (n = 284) addressing PICO were unstructured and commonly reported 'population' (n = 195, 68.7%), 'intervention' (n = 242, 85.2%), and 'outcome' (n = 127, 44.7%) but not 'control' (n = 29, 10.2%). Concepts underlying GRADE domains were infrequently reported in IfR statements of SRs (n = 284): 'risk of bias' (n = 14, 4.9%), 'imprecision' (n = 8, 2.8%), 'inconsistency' (n = 7, 2.5%), 'publication bias' (n = 3, 1.1%), and 'indirectness' (n = 1, 0.4%). Additional IfR elements mentioned in IfR were 'better reporting' of future studies (n = 17, 6.0%) and 'standardization of procedures in clinical trials' (n = 12, 4.2%).CONCLUSION: Almost 90% of SRs on COVID-19 treatments reported IfR. IfR statements addressing PICO were unstructured across SRs and concepts underlying GRADE were rarely reported to inform IfR. Further work is needed to assess generalizability beyond COVID-19 and to define more precisely which IfR elements should be considered, and how they should be reported in SRs of interventions. Until then, considering PICO elements and concepts underlying GRADE to derive IfR seems to be a sensible starting point.
U2 - 10.1016/j.jclinepi.2023.111236
DO - 10.1016/j.jclinepi.2023.111236
M3 - SCORING: Journal article
C2 - 38072174
VL - 166
SP - 111236
JO - J CLIN EPIDEMIOL
JF - J CLIN EPIDEMIOL
SN - 0895-4356
M1 - 111236
ER -