Looking for a Change Data Capture (CDC) lib for MySQL

Questions / Help
, ,
1

My Elixir application needs to connect with a MySQL database and listen for insert and updates in certain tables, does anyone know about a library that makes this easier? I would use WalEx or Cainophile but they are intended for PostgreSQL DBs.

2

The MySQL driver doesn’t seem to support this functionality, so I don’t think you will have much luck finding other libraries around, unless you want to fork it and add your functionality there somehow.

I think your best bet is to use ports to integrate with another language that has support for listening for events, like IDK Java or maybe something more lightweight.

3

Hi

You could use Port — Elixir v1.16.3 with the stdout option of https://maxwells-daemon.io/

I have this proof of concept code that might get you started

defmodule MaxwellListener do
  use GenServer

  # Adjust the command to include the full executable path if necessary
  @command "maxwell"
  @args [
    "--user='maxwell'",
    "--password='XXXXXX'",
    "--host='127.0.0.1'",
    "--filter='exclude: *.*, include: holdsport.users, include: holdsport.teams_users'",
    "--producer=stdout"
  ]

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, :ok, opts)
  end

  def init(:ok) do
    command_with_args = Enum.join([@command | @args], " ")
    options = [:binary, :exit_status]
    port = Port.open({:spawn, command_with_args}, options)
    {:ok, %{port: port}}
  end

  def handle_info({port, {:data, data}}, state) do
    process_output(data)
    {:noreply, state}
  end

  def handle_info({:EXIT, _port, _reason}, state) do
    # Handle port exit
    {:stop, :normal, state}
  end

  defp process_output(data) do
    data
    |> String.trim_trailing()
    |> Jason.decode!(keys: :atoms)
    |> handle()
  end

  defp handle(%{table: "teams_users", data: %{users_id: user_id}}) do
    IO.inspect(user_id, label: "User ID")
  end

  defp handle(d) do
    IO.inspect(d, limit: :infinity)
  end
end
1 Like
4

Thank you! I will give it a try