Effective Load Management in Stream Processing

In this project, we are investigating techniques that help in coping with large volumes of data that cannot be handled with the available resources at hand, in the context of stream processing systems.

System-controlled semantic load shedding

In this project, we are studying joint system-application optimization techniques to perform more effective load shedding than system-only and application-only approaches. The goal is to leave the arbitration of load shedding decisions to the middleware, whereas the load shedding technique to use is left to the operator developers. A registration interface helps bridge the two components by helping the middleware to reason about quality of service of applications.

Smart compression for streaming transports

In this project, we are studying smart compression techniques (both lossy and lossless) for streaming transports. The idea is to take advantage of the different characteristics of different stream attributes with respect to compression ratio, cost, and size, and to adapt to runtime dynamics such as changes in the bandwidth availability, cpu availability, and workload availability.

Publications

  • Buğra Gedik. “Discriminative Fine-Grained Mixing for Adaptive Compression of Data Streams”, Transactions on Computers, IEEE (TC), ISSN: 0018-9340, DOI: 10.1109/TC.2013.103, 2013.
  • Buğra Gedik. “Partitioning Functions for Stateful Data Parallelism in Stream Processing”, Very Large Data Bases Journal (VLDBJ), (accepted), 2013.
projects/loadmanage.txt · Last modified: 2017/02/09 13:45 (external edit)