Michal Gibowski has been working as a software engineer since 2006. He spent multiple years in the realm of JVM and Scala until he discovered great joy in programming with Elixir and its ecosystem.
He is currently leading an Elixir engineering team at a french company Andjaro, while working from home on a sunny island Tenerife.
Let’s talk about data in a distributed system. How can we avoid the problem of dual writes, reliably implement a message outbox pattern or expose information to analytics storage in real-time?
Databases distribute data to other nodes via a mechanism called replication. It turns out we can hook into it and make it digestible for our applications!
In this talk, we will learn about Change Data Capture and show how Elixir and Broadway are a great fit for an application that consumes streams of data originating from database replication.
At Andjaro we do that to implement the Strangler Fig pattern and to feed our analytics data warehouse.