Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2018
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2018
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2018
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
https://doi.org/10.5281/zenodo...
Dataset . 2018
License: CC BY
Data sources: Sygma
versions View all 3 versions
addClaim

Nephele Big Data Experiment

Authors: Chroz, Peter Aziz;

Nephele Big Data Experiment

Abstract

The big data experiments use HiBench for benchmarking, which generates data using a number of data generators. These generators generate text data, web-data that follows a Zipfian distribution, numeric data for k-means clustering following a uniform or Gaussian distribution. Each of those benchmark instances consists of ten micro benchmarks. Some of those micro benchmarks provide multiple implementations. These differ in language of implementation and framework. The frameworks includes Hadoop and Spark. The Spark-based implementations feature code in Java, Python, and Scala. Below is a list of evaluated workloads and data: 1. Sort – This micro benchmark sorts text data, which is generated by RandomTextWriter. 2. WordCount – This micro-benchmark counts the occurrences of words in the input data, which is generated by RandomTextWriter. 3. TeraSort – This micro benchmark sorts a large data set generated by TeraGen. This micro benchmark has been used to push the boundaries of sorting very large data sets. 4. Sleep – This micro benchmark sleeps for a set amount of time to test the scheduler. 5. SQL (Scan, Join, Aggregate) – This benchmark tests typical SQL commands using Hive queries. The benchmark operates on generated hyperlinks. 6. PageRank – This benchmark computes the PageRank on generated hyperlinks. 7. Nutch indexing – Nutch is an Apache search engine and implements an indexing algorithm. The benchmark operates on generated web sites with words and hyperlinks. 8. Bayesian classification – This benchmarks uses Spark MLLib and Mahout to perform Naive Bayesian classification. The data consists of generated text documents. 9. K-means clustering – In this benchmark the k-means clustering algorithm in Spark MLLib/Mahout is used to cluster generated input data. 10. Enhanced DFSIO – This benchmark tests the HDFS throughput by generating a large number of tasks that simultaneously write and read data from HDFS.

Keywords

Hibench, Spark, Hadoop

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 5
    download downloads 1
  • 5
    views
    1
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
0
Average
Average
Average
5
1