Powered by OpenAIRE graph
Found an issue? Give us feedback
addClaim

Parallel and Memory-Efficient Preprocessing for Metagenome Assembly

Authors: Vasudevan Rengasamy; Paul Medvedev; Kamesh Madduri;

Parallel and Memory-Efficient Preprocessing for Metagenome Assembly

Abstract

The analysis of high-throughput metagenomic sequencingdata poses significant computational challenges. Mostcurrent de novo assembly tools use the de Bruijn graph-basedmethodology. In prior work, a connected components decompositionof the de Bruijn graph and subsequent partitioningof sequence read data was shown to be an effective memory reducingpreprocessing step for de novo assembly of largemetagenomic datasets. In this paper, we present METAPREP, a new end-to-end parallel implementation of a similar preprocessingstep. METAPREP has efficient implementations ofseveral computational subroutines (e.g., k-mer enumerationand counting, parallel sorting, graph connectivity) that occurin other genomic data analysis problems, and we show thatour implementations are comparable to the state-of-the-art. METAPREP is primarily designed to execute on large shared memorymulticore servers, but scales gracefully to use multiplecompute nodes and clusters with parallel I/O capabilities. WithMETAPREP, we can process the Iowa Continuous Corn soilmetagenomics dataset, comprising 1.13 billion reads totaling223 billion base pairs, in around 14 minutes, using just 16 nodesof the NERSC Edison supercomputer. We also evaluate theperformance impact of METAPREP on MEGAHIT, a parallelmetagenome assembler.

Related Organizations
  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    2
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
2
Average
Average
Average
Upload OA version
Are you the author of this publication? Upload your Open Access version to Zenodo!
It’s fast and easy, just two clicks!