With Flink and Kubernetes, it’s possible to deploy stream processing jobs with just SQL and YAML. This low-code approach can certainly save a lot of development time. However, there is more to data pipelines than just streaming SQL. We must wire up many different systems, thread through schemas, and, worst-of-all, write a lot of configuration.
In this talk, we’ll explore just how “declarative” we can make streaming data pipelines on Kubernetes. I’ll show how we can go deeper by adding more and more operators to the stack. How deep can we go?
