Hey everyone!

I’ve released version 0.16.47 today (like I do almost every Tuesday, following TigerBeetle release schedule). This time though the version doesn’t just bump the TigerBeetle client version, but also adds support for TigerBeetle Change Data Capture functionality.

To provide a little more context, since version 0.16.43 TigerBeetle has the ability to stream its transactions to RabbitMQ, so in this version of TigerBeetlex I’ve added structs to decode the JSON payload of the events and a guide showing how to use Broadway to build a pipeline processing TigerBeetle CDC data.

How’s your experience with TigerBeetle been so far? I’m evaluating it for an upcoming project, and I still have a bunch of questions.

I’m curious what’s the best approach for something that you expect to be atomic across your general purpose DB (Postgres) and your TigerBeetle DB.

For example, let’s say you have an Invoice in your general purpose DB and you’re using its id as user_data_128 on the TB Transfer. So when you bill an account, you would create a transfer that debits this amount, crediting the account doing the invoicing. And then the opposite when the billed account pays up, crediting them and debiting the account doing the invoicing. At that point, I would probably want to mark the Invoice as paid in some way. But then you run into the questions like, what happens if the server goes down when the transfer has been written in TB but not in the general purpose DB? Or if you go to update the invoice and this write fails because the invoice didn’t ever exist, etc?

I’m wondering if it even makes sense to have an explicit enum like that on the Invoice in the first place or if TigerBeetle should just be the only source of truth, and when you need to look that up, you just hit TB to see if there is a transfer with that Invoices id as user_data_128 and a code that represents an invoice being paid. E.g. code=10 is an invoice billed, code=20 is an invoice being paid, code=30 is an invoice being cancelled or otherwise comped.

To be transparent, I didn’t actually get to use the library in a project yet I’m mainly developing it as an interesting project where I can work with both Elixir and Zig. You can have a look at TigerFans for an example of a project built with TigerBeetle (not in Elixir, but data modeling is language agnostic anyways).

Regarding transactions across TigerBeetle and Postgres, there’s a recent blogpost on the TigerBeetle blog detailing why you would want to use a specific order for write and reads (i.e. write to TigerBeetle last, read from TigerBeetle first). For temporary failures, TigerBeetle pushes towards end-to-end idempotency to be able to safely retry operations, more details here.