Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Article . 2021
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
image/svg+xml Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao Closed Access logo, derived from PLoS Open Access logo. This version with transparent background. http://commons.wikimedia.org/wiki/File:Closed_Access_logo_transparent.svg Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao
versions View all 6 versions
addClaim

This Research product is the result of merged Research products in OpenAIRE.

You have already added 0 works in your ORCID record related to the merged Research product.

WindFlow: High-Speed Continuous Stream Processing With Parallel Building Blocks

Authors: Mencagli G.; Torquati M.; Cardaci A.; Fais A.; Rinaldi L.; Danelutto M.;

WindFlow: High-Speed Continuous Stream Processing With Parallel Building Blocks

Abstract

Nowadays, we are witnessing the diffusion of Stream Processing Systems (SPSs) able to analyze data streams in near realtime. Traditional SPSs like Storm and Flink target distributed clusters and adopt the continuous streaming model , where inputs are processed as soon as they are available while outputs are continuously emitted. Recently, there has been a great focus on SPSs for scale-up machines. Some of them (e.g., BriskStream ) still use the continuous model to achieve low latency. Others optimize throughput with batching approaches that are, however, often inadequate to minimize latency for live-streaming applications. Our contribution is to show a novel software engineering approach to design the runtime system of SPSs targeting multicores, with the aim of providing a uniform solution able to optimize throughput and latency. The approach has a formal nature based on the assembly of components called building blocks , whose composition allows optimizations to be easily expressed in a compositional manner. We use this methodology to build a new SPS called WindFlow . Our evaluation showcases the benefits of WindFlow : it provides lower latency than SPSs for continuous streaming, and can be configured to optimize throughput, to perform similarly and even better than batch-based scale-up SPSs.

Country
Italy
Related Organizations
Subjects by Vocabulary

Microsoft Academic Graph classification: Multi-core processor Computer science Continuous modelling Data stream mining Distributed computing Latency (audio) Stream processing Runtime system Latency (engineering) Throughput (business)

Keywords

parallel computing, multicore programming, Data stream processing, Data stream processing; multicore programming; parallel computing, Computational Theory and Mathematics, Hardware and Architecture, Signal Processing

38 references, page 1 of 4

[1] A. Arasu, S. Babu, and J. Widom, “The cql continuous query language: Semantic foundations and query execution,” The VLDB Journal, vol. 15, no. 2, p. 121142, Jun. 2006. [Online]. Available: https://doi.org/10.1007/s00778-004-0147-z

[2] M. V. Bordin, D. Griebler, G. Mencagli, C. F. R. Geyer, and L. G. L. Fernandes, “Dspbench: A suite of benchmark applications for distributed data stream processing systems,” IEEE Access, vol. 8, pp. 222 900-222 917, 2020.

[3] “Apache storm,” http://storm.apache.org/, 2020, [Online; accessed 26-Feb-2020].

[4] “Apache flink,” https://flink.apache.org/, 2020, [Online; accessed 26-Feb-2020].

[5] S. Zhang, B. He, D. Dahlmeier, A. C. Zhou, and T. Heinze, “Revisiting the design of data stream processing systems on multicore processors,” in 2017 IEEE 33rd International Conference on Data Engineering (ICDE), April 2017, pp. 659-670.

[6] Z. Li, H. Shen, and L. Ward, “Accelerating big data analytics using scale-up/out heterogeneous clusters,” in 2019 28th International Conference on Computer Communication and Networks (ICCCN), 2019, pp. 1-9.

[7] A. Addisie and V. Bertacco, “Collaborative accelerators for in-memory mapreduce on scale-up machines,” in Proceedings of the 24th Asia and South Pacific Design Automation Conference, ser. ASPDAC '19. New York, NY, USA: Association for Computing Machinery, 2019, p. 747753. [Online]. Available: https://doi.org/10.1145/3287624.3287636

[8] S. Zhang, J. He, A. C. Zhou, and B. He, “Briskstream: Scaling data stream processing on shared-memory multicore architectures,” in Proceedings of the 2019 International Conference on Management of Data, ser. SIGMOD '19. New York, NY, USA: ACM, 2019, pp. 705-722. [Online]. Available: http://doi.acm.org/10.1145/3299869.3300067

[9] V. Leis, P. Boncz, A. Kemper, and T. Neumann, “Morseldriven parallelism: A numa-aware query evaluation framework for the many-core age,” in Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD 14. New York, NY, USA: Association for Computing Machinery, 2014, p. 743754. [Online]. Available: https://doi.org/10.1145/2588555.2610507

[10] H. Miao, H. Park, M. Jeon, G. Pekhimenko, K. S. McKinley, and F. X. Lin, “Streambox: Modern stream processing on a multicore machine,” in Proceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference, ser. USENIX ATC 17. USA: USENIX Association, 2017, p. 617629.

  • BIP!
    Impact byBIP!
    citations
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    4
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Top 10%
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 52
    download downloads 112
  • citations
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    4
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Top 10%
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    Powered byBIP!BIP!
  • 52
    views
    112
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
citations
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
4
Top 10%
Average
Average
52
112
hybrid