Beyond Analytics: the Evolution of Stream Processing Systems

Tutorial for the 2020 ACM SIGMOD International Conference on Management of Data

Tutorial Information

Wednesday, June 17 2020

Join us on Zoom and Slack

Session 1: 10:30 AM - 12:00 PM PDT

Session 2: 1:30 PM - 3:00 PM PDT


Stream processing has been an active research field for more than 20 years, but it is now witnessing its prime time due to recent successful efforts by the research community and numerous worldwide open-source communities. The goal of this tutorial is threefold. First, we aim to review and highlight noteworthy past research findings, which were largely ignored until very recently. Second, we intend to underline the differences between early (’00-’10) and modern (’11-’18) streaming systems, and how those systems have evolved through the years. Most importantly, we wish to turn the attention of the database community to recent trends: streaming systems are no longer used only for classic stream processing workloads, namely window aggregates and joins. Instead, modern streaming systems are being increasingly used to deploy general event-driven applications in a scalable fashion, challenging the design decisions, architecture and intended use of existing stream processing systems.


Slides and Videos

  1. Introduction and fundamentals [Slides] [Video]
  2. Time, order, and progress [Slides] [Video]
  3. State management and guarantees [Slides] [Video]
  4. Advanced fault recovery and high availability [Slides] [Video - Part I] [Video - Part II]
  5. Load management and elasticity [Slides] [Video]
  6. Prospects and discussion [Slides] [Video]

Cite (PDF)

author = {Carbone, Paris and Fragkoulis, Marios and Kalavri, Vasiliki and Katsifodimos, Asterios},
title = {Beyond Analytics: The Evolution of Stream Processing Systems},
year = {2020},
isbn = {9781450367356},
doi = {10.1145/3318464.3383131},
booktitle = {Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data},
pages = {2651–2658}