Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2019
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2019
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2019
License: CC BY
Data sources: ZENODO
image/svg+xml Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao Closed Access logo, derived from PLoS Open Access logo. This version with transparent background. http://commons.wikimedia.org/wiki/File:Closed_Access_logo_transparent.svg Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao
versions View all 3 versions
addClaim

Computational Artifacts for Performance Feedback Autoscaling Experiments with Workloads of Workflows in Apache Airflow

Authors: Ilyushkin, Alexey; Bauer, André; Papadopoulos, Alessandro Vittorio; Deelman, Ewa; Iosup, Alexandru;

Computational Artifacts for Performance Feedback Autoscaling Experiments with Workloads of Workflows in Apache Airflow

Abstract

These computational artifacts are related to the software artifacts DOI:10.5281/zenodo.2635571 The content of the computational artifacts: experiments.pdf contains the list of all the conducted experiments with the Airflow system. Experiment IDs are not sequential since some experiments required rerunning, etc., we report only successful results. The file lists different experiment configurations, e.g., the number of processed workflows, the name of the used workload, the user budgets, and PFA settings. db.tar.gz contains directories with Airflow database snapshots and autoscaler logs. The names of the directories correspond to those listed in `experiments.pdf`. Each experiment directory contains an autoscaler log and a full copy of a PostgreSQL database directory just after each experiment finished. The database name is `airflow`, the user name is `ailyushk`. Within each database, most of the paper-related data are stored in the `stat_log` table. The scripts for extracting data from these databases are available as software artifacts in `tools/analysis`. gurobi.tar.gz contains the results obtained from the Gurobi solver when solving the MIP model. pdf.tar.gz contains all the figures in pdf format, also those that were not included neither in the paper nor in the technical report. The scripts for creating this plots are delivered as software artifacts. csv.tar.gz contains the analysis results extracted from Airflow database snapshots. These files are used to create the plots in the `pdf` directory. The scripts for doing this are delivered as software artifacts. wl1.tar.gz is the first synthetic realistic workload (WL I) with three subsets of 200 workflows each (`1_0`, `1_1`, `1_2`). Each directory contains the file with interarrivals `interarrivals.txt`, and the file with workflow IDs `workload.txt` in the subset. The `dags` directory contains Python-based Airflow descriptors and CSV files that summarise the same descriptors in CSV format for simpler analysis. The scripts for extracting workload statistics from these CSV files are available in the software artifacts in `tools/analysis`. The `inputs` directory contains initial input files for each worfklow. The `dax` contains original DAX files obtained from the generator: https://github.com/pegasus-isi/WorkflowGenerator/tree/master/bharathi/src/simulation/generator wl2.tar.gz is the second synthetic realistic workload (WL II) with three subsets of 200 workflows each (`4_0`, `4_1`, `4_2`). Has similar structure as `wl1.tar.gz`, except that `dax` directory is omitted, as WL II uses the same DAX structures as WL I. wl3.tar.gz is the small synthetic workload based on WL I for the experiment with the MIP solver, contains three subsets with 5 workflows in each, all in the `3_0` directory (thus, the structure differs from the WL I and WL II). The input data files are empty. The identifiers of workflows forming each subset are stored in the `workload_1.txt`, `workload_2.txt`, and `workload_3.txt` files.

Country
Netherlands
  • BIP!
    Impact byBIP!
    citations
    The total number of citations received by all articles of the researcher of interest.
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 8
    download downloads 4
  • 8
    views
    4
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
citations
The total number of citations received by all articles of the researcher of interest.
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
0
Average
Average
Average
8
4