Vanessa Loviton

Backend Engineer @ Andjaro

Vanessa Loviton lives in Paris, where she works as a backend engineer at Andjaro. Having started with Ruby (on it’s rails), she quickly jumped to Elixir and never looked back. When she’s not busy implementing new features, she spends way too much time on video games or around role-play tables with her friends.

Talk:
Change Data Capture with Elixir and Debezium

Let’s talk about data in a distributed system. How can we avoid the problem of dual writes, reliably implement a message outbox pattern or expose information to analytics storage in real-time?

Databases distribute data to other nodes via a mechanism called replication. It turns out we can hook into it and make it digestible for our applications!

In this talk, we will learn about Change Data Capture and show how Elixir and Broadway are a great fit for an application that consumes streams of data originating from database replication.

At Andjaro we do that to implement the Strangler Fig pattern and to feed our analytics data warehouse.

Talk objectives:

  • to show challenges when passing state in a distributed system,
  • to introduce Change Data Capture and the Debezium project,
  • to showcase the power of unique capabilities of OTP and Elixir when syncing data between different data systems,
  • showcase Broadway as a way to digest big chunks of data.

Target audience:

  • Elixir programmers interested in distributed systems, event streaming, and data ingestion with Broadway.