Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Project deliverable . 2019
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Other literature type . 2019
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Project deliverable . 2019
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
https://doi.org/10.5281/zenodo...
Article . 2019
License: CC BY
Data sources: Sygma
versions View all 4 versions
addClaim

D4.1 Definition of Architecture for Extreme-Scale Analytics

Authors: Project Consortium Members;

D4.1 Definition of Architecture for Extreme-Scale Analytics

Abstract

This deliverable defines the building blocks of the INFORE Architecture, their functionality and their interconnections in the scope of a holistic, pluggable, extensible INFORE framework that will evolve to an omnibus solution for extreme-scale streaming analytics. Aligned with the objectives stated in the project proposal, INFORE aims at: (i) supporting the non-programmer data analyst in rapid setup of streaming workflows tailored for her application scenario needs by providing graphical workflow design facilities, (ii) automating the tuning of the underlying Big Data platform infrastructure that materializes the visually designed workflow as well as the provisioned physical resources in a way that optimizes specific performance measures, (iii) providing real-time, interactive machine learning and data mining tools that can be leveraged by the designed workflows, (iv) enhanced interactivity via data summarization and approximate query processing techniques, (v) distributed complex event processing and forecasting techniques to not only detect business events of interest as soon as they occur, but also forecast their occurrence well in advance. To achieve these goals, the definition of the INFORE Architecture includes the following loosely coupled, modular components: (i) Graphical Editor Component, (ii) Connection Component, (iii) Manager Component, (iv) Optimizer Component, (v) Synopsis Data Engine Component, (vi) Interactive Online Machine Learning Component, (vii) Complex Event Forecasting Component. The Graphical Editor Component is an extension of the RapidMiner Studio developed in the scope of the project. An elaborate Streaming Nest operator is being developed within the Studio. The Streaming Nest operator is essentially an umbrella encompassing a family of streaming, logical (i.e., abstract, not tied to a particular implementation on a Big Data platform) operators developed in the scope of the project. This family of operators includes both data management operators such as filtering, join, projection, map, reduce, aggregations etc operators as well as logical operators for online machine learning, complex event forecasting and data approximation. Having designed the desired workflow using drag and drop functionality of the Graphical Editor Component, the user proceeds with visually creating connection objects for input, output streams and streaming backends (available Big Data platforms and respective clusters hosting them) in the Graphical Editor Component. These visually defined connection details are internally handled by the Connection Component. Upon submitting the workflow, the Manager Component takes over to convey the submitted workflow to the Optimizer Component. The Optimizer Component returns to the Manager Component a modified, optimized workflow where it has attached execution plan information related to (a) the Big Data platform on which each operator of the workflow will be executed, (b) the cluster resources that will be provisioned, (c) the cluster in which each operator will be deployed in case of multiple geo-dispersed clusters, (d) replacements of exact workflow operators with approximate ones provided by the Synopses Data Engine Component should the user have defined that some predefined inaccuracy guarantees can be tolerated by the application for reducing workflow execution time. The Manager Component may visualize the modified workflow and ask for user approval or execute the actual plan provided by the optimizer. To do so, all logical operators in the execution plan provided by the Optimizer are instantiated by their physical implementations. A dispatcher module as part of the Manager Component submits separate jobs for each Big Data platform and respective cluster, while output streams are provided to the desired applications. In that scope, the physical implementation of approximate query processing operators is included in the Synopses Data Engine Component. The physical implementation of machine learning operators resides in the Interactive Online Machine Learning Component and similarly for the Complex Event Forecasting Component. This deliverable is in direct relation to deliverables of WP1, WP2, WP3 (up to date, use case requirements have been expressed in Deliverables D1.2, D2.1, D3.1) which aid in realizing the INFORE framework to specific application scenarios. WP5 specifies the internal details of the Optimizer Component starting in Deliverable D5.1 to be submitted in Month 16 of the project. WP6 develops the Synopses Data Engine Component (Deliverable D6.1 on Month 12 together with the current one, which is later enhanced in D6.3), the Interactive Online Machine Learning Component and the Complex Event Forecasting Component described in Deliverables described in Deliverables D6.2 (Month 16), D6.4, D6.5. The first, complete prototype of the INFORE Architecture is presented in the follow up Deliverable D4.2 on Month 16.

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 12
    download downloads 13
  • 12
    views
    13
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
0
Average
Average
Average
12
13
Green
Funded by