Research ExplorerPublicationsBenchmark of thirteen bioinformatic pipelines for metagenomi...

Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples

Standard

Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples. / de Vries, Jutte J C; Brown, Julianne R; Fischer, Nicole; Sidorov, Igor A; Morfopoulou, Sofia; Huang, Jiabin; Munnink, Bas B Oude; Sayiner, Arzu; Bulgurcu, Alihan; Rodriguez, Christophe; Gricourt, Guillaume; Keyaerts, Els; Beller, Leen; Bachofen, Claudia; Kubacki, Jakub; Samuel, Cordey; Florian, Laubscher; Dennis, Schmitz; Beer, Martin; Hoeper, Dirk; Huber, Michael; Kufner, Verena; Zaheri, Maryam; Lebrand, Aitana; Papa, Anna; van Boheemen, Sander; Kroes, Aloys C M; Breuer, Judith; Lopez-Labrador, F Xavier; Claas, Eric C J; ESCV Network on Next-Generation Sequencing.

In: J CLIN VIROL, Vol. 141, 104908, 08.2021.

Research output: SCORING: Contribution to journal › SCORING: Journal article › Research › peer-review

Harvard

de Vries, JJC, Brown, JR, Fischer, N, Sidorov, IA, Morfopoulou, S, Huang, J, Munnink, BBO, Sayiner, A, Bulgurcu, A, Rodriguez, C, Gricourt, G, Keyaerts, E, Beller, L, Bachofen, C, Kubacki, J, Samuel, C, Florian, L, Dennis, S, Beer, M, Hoeper, D, Huber, M, Kufner, V, Zaheri, M, Lebrand, A, Papa, A, van Boheemen, S, Kroes, ACM, Breuer, J, Lopez-Labrador, FX, Claas, ECJ & ESCV Network on Next-Generation Sequencing 2021, 'Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples', J CLIN VIROL, vol. 141, 104908. https://doi.org/10.1016/j.jcv.2021.104908

APA

de Vries, J. J. C., Brown, J. R., Fischer, N., Sidorov, I. A., Morfopoulou, S., Huang, J., Munnink, B. B. O., Sayiner, A., Bulgurcu, A., Rodriguez, C., Gricourt, G., Keyaerts, E., Beller, L., Bachofen, C., Kubacki, J., Samuel, C., Florian, L., Dennis, S., Beer, M., ... ESCV Network on Next-Generation Sequencing (2021). Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples. J CLIN VIROL, 141, [104908]. https://doi.org/10.1016/j.jcv.2021.104908

Vancouver

de Vries JJC, Brown JR, Fischer N, Sidorov IA, Morfopoulou S, Huang J et al. Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples. J CLIN VIROL. 2021 Aug;141. 104908. https://doi.org/10.1016/j.jcv.2021.104908

Bibtex

@article{162031f95f9b4d5497c91cc64cf27352,
title = "Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples",
abstract = "INTRODUCTION: Metagenomic sequencing is increasingly being used in clinical settings for difficult to diagnose cases. The performance of viral metagenomic protocols relies to a large extent on the bioinformatic analysis. In this study, the European Society for Clinical Virology (ESCV) Network on NGS (ENNGS) initiated a benchmark of metagenomic pipelines currently used in clinical virological laboratories.METHODS: Metagenomic datasets from 13 clinical samples from patients with encephalitis or viral respiratory infections characterized by PCR were selected. The datasets were analyzed with 13 different pipelines currently used in virological diagnostic laboratories of participating ENNGS members. The pipelines and classification tools were: Centrifuge, DAMIAN, DIAMOND, DNASTAR, FEVIR, Genome Detective, Jovian, MetaMIC, MetaMix, One Codex, RIEMS, VirMet, and Taxonomer. Performance, characteristics, clinical use, and user-friendliness of these pipelines were analyzed.RESULTS: Overall, viral pathogens with high loads were detected by all the evaluated metagenomic pipelines. In contrast, lower abundance pathogens and mixed infections were only detected by 3/13 pipelines, namely DNASTAR, FEVIR, and MetaMix. Overall sensitivity ranged from 80% (10/13) to 100% (13/13 datasets). Overall positive predictive value ranged from 71-100%. The majority of the pipelines classified sequences based on nucleotide similarity (8/13), only a minority used amino acid similarity, and 6 of the 13 pipelines assembled sequences de novo. No clear differences in performance were detected that correlated with these classification approaches. Read counts of target viruses varied between the pipelines over a range of 2-3 log, indicating differences in limit of detection.CONCLUSION: A wide variety of viral metagenomic pipelines is currently used in the participating clinical diagnostic laboratories. Detection of low abundant viral pathogens and mixed infections remains a challenge, implicating the need for standardization and validation of metagenomic analysis for clinical diagnostic use. Future studies should address the selective effects due to the choice of different reference viral databases.",
keywords = "Benchmarking, Computational Biology, High-Throughput Nucleotide Sequencing, Humans, Metagenomics, Viruses/genetics",
author = "{de Vries}, {Jutte J C} and Brown, {Julianne R} and Nicole Fischer and Sidorov, {Igor A} and Sofia Morfopoulou and Jiabin Huang and Munnink, {Bas B Oude} and Arzu Sayiner and Alihan Bulgurcu and Christophe Rodriguez and Guillaume Gricourt and Els Keyaerts and Leen Beller and Claudia Bachofen and Jakub Kubacki and Cordey Samuel and Laubscher Florian and Schmitz Dennis and Martin Beer and Dirk Hoeper and Michael Huber and Verena Kufner and Maryam Zaheri and Aitana Lebrand and Anna Papa and {van Boheemen}, Sander and Kroes, {Aloys C M} and Judith Breuer and Lopez-Labrador, {F Xavier} and Claas, {Eric C J} and {ESCV Network on Next-Generation Sequencing}",
note = "Copyright {\textcopyright} 2021. Published by Elsevier B.V.",
year = "2021",
month = aug,
doi = "10.1016/j.jcv.2021.104908",
language = "English",
volume = "141",
journal = "J CLIN VIROL",
issn = "1386-6532",
publisher = "Elsevier",

}

RIS

TY - JOUR

T1 - Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples

AU - de Vries, Jutte J C

AU - Brown, Julianne R

AU - Fischer, Nicole

AU - Sidorov, Igor A

AU - Morfopoulou, Sofia

AU - Huang, Jiabin

AU - Munnink, Bas B Oude

AU - Sayiner, Arzu

AU - Bulgurcu, Alihan

AU - Rodriguez, Christophe

AU - Gricourt, Guillaume

AU - Keyaerts, Els

AU - Beller, Leen

AU - Bachofen, Claudia

AU - Kubacki, Jakub

AU - Samuel, Cordey

AU - Florian, Laubscher

AU - Dennis, Schmitz

AU - Beer, Martin

AU - Hoeper, Dirk

AU - Huber, Michael

AU - Kufner, Verena

AU - Zaheri, Maryam

AU - Lebrand, Aitana

AU - Papa, Anna

AU - van Boheemen, Sander

AU - Kroes, Aloys C M

AU - Breuer, Judith

AU - Lopez-Labrador, F Xavier

AU - Claas, Eric C J

AU - ESCV Network on Next-Generation Sequencing

N1 - Copyright © 2021. Published by Elsevier B.V.

PY - 2021/8

Y1 - 2021/8

N2 - INTRODUCTION: Metagenomic sequencing is increasingly being used in clinical settings for difficult to diagnose cases. The performance of viral metagenomic protocols relies to a large extent on the bioinformatic analysis. In this study, the European Society for Clinical Virology (ESCV) Network on NGS (ENNGS) initiated a benchmark of metagenomic pipelines currently used in clinical virological laboratories.METHODS: Metagenomic datasets from 13 clinical samples from patients with encephalitis or viral respiratory infections characterized by PCR were selected. The datasets were analyzed with 13 different pipelines currently used in virological diagnostic laboratories of participating ENNGS members. The pipelines and classification tools were: Centrifuge, DAMIAN, DIAMOND, DNASTAR, FEVIR, Genome Detective, Jovian, MetaMIC, MetaMix, One Codex, RIEMS, VirMet, and Taxonomer. Performance, characteristics, clinical use, and user-friendliness of these pipelines were analyzed.RESULTS: Overall, viral pathogens with high loads were detected by all the evaluated metagenomic pipelines. In contrast, lower abundance pathogens and mixed infections were only detected by 3/13 pipelines, namely DNASTAR, FEVIR, and MetaMix. Overall sensitivity ranged from 80% (10/13) to 100% (13/13 datasets). Overall positive predictive value ranged from 71-100%. The majority of the pipelines classified sequences based on nucleotide similarity (8/13), only a minority used amino acid similarity, and 6 of the 13 pipelines assembled sequences de novo. No clear differences in performance were detected that correlated with these classification approaches. Read counts of target viruses varied between the pipelines over a range of 2-3 log, indicating differences in limit of detection.CONCLUSION: A wide variety of viral metagenomic pipelines is currently used in the participating clinical diagnostic laboratories. Detection of low abundant viral pathogens and mixed infections remains a challenge, implicating the need for standardization and validation of metagenomic analysis for clinical diagnostic use. Future studies should address the selective effects due to the choice of different reference viral databases.

AB - INTRODUCTION: Metagenomic sequencing is increasingly being used in clinical settings for difficult to diagnose cases. The performance of viral metagenomic protocols relies to a large extent on the bioinformatic analysis. In this study, the European Society for Clinical Virology (ESCV) Network on NGS (ENNGS) initiated a benchmark of metagenomic pipelines currently used in clinical virological laboratories.METHODS: Metagenomic datasets from 13 clinical samples from patients with encephalitis or viral respiratory infections characterized by PCR were selected. The datasets were analyzed with 13 different pipelines currently used in virological diagnostic laboratories of participating ENNGS members. The pipelines and classification tools were: Centrifuge, DAMIAN, DIAMOND, DNASTAR, FEVIR, Genome Detective, Jovian, MetaMIC, MetaMix, One Codex, RIEMS, VirMet, and Taxonomer. Performance, characteristics, clinical use, and user-friendliness of these pipelines were analyzed.RESULTS: Overall, viral pathogens with high loads were detected by all the evaluated metagenomic pipelines. In contrast, lower abundance pathogens and mixed infections were only detected by 3/13 pipelines, namely DNASTAR, FEVIR, and MetaMix. Overall sensitivity ranged from 80% (10/13) to 100% (13/13 datasets). Overall positive predictive value ranged from 71-100%. The majority of the pipelines classified sequences based on nucleotide similarity (8/13), only a minority used amino acid similarity, and 6 of the 13 pipelines assembled sequences de novo. No clear differences in performance were detected that correlated with these classification approaches. Read counts of target viruses varied between the pipelines over a range of 2-3 log, indicating differences in limit of detection.CONCLUSION: A wide variety of viral metagenomic pipelines is currently used in the participating clinical diagnostic laboratories. Detection of low abundant viral pathogens and mixed infections remains a challenge, implicating the need for standardization and validation of metagenomic analysis for clinical diagnostic use. Future studies should address the selective effects due to the choice of different reference viral databases.

KW - Benchmarking

KW - Computational Biology

KW - High-Throughput Nucleotide Sequencing

KW - Humans

KW - Metagenomics

KW - Viruses/genetics

U2 - 10.1016/j.jcv.2021.104908

DO - 10.1016/j.jcv.2021.104908

M3 - SCORING: Journal article

C2 - 34273858

VL - 141

JO - J CLIN VIROL

JF - J CLIN VIROL

SN - 1386-6532

M1 - 104908

ER -