totorigolo

totorigolo

Ecto Sandbox ownership issue when persisting state in terminate callback

Hello,

Context
I am developing a toy website to get myself familiar with Elixir and Phoenix. The website is like multiple shared canvases where everyone with a token can change pixels (a bit like multiple tiny https://pixelcanvas.io/). On the home page, you can see auto-refreshing thumbnails of each canvas, and when you click on one, you see the full-size version with live updates using a Phoenix Channel. The canvases can be created or deleted on the fly in the admin section, which will create the new canvas (UUID, name) in the DB and spin-up a (unique) GenServer (per canvas). Without persistence, I successfully deployed it in a cluster, using global processes (via Horde), so far so good.

Now, I want to persist the images. To do that, whenever a canvas GenServer terminates, it saves the PNG blob in the DB. When it starts, it tries to load it from the DB. This works well, I can add/remove nodes from the cluster and the image survives.

Here is how canvas GenServers are handled:

  • when I create a canvas using Canvases.create_canvas/1, the function creates the row in DB, starts the GenServer under a (Horde) DynamicSupervisor using CanvasManager.start_canvas_server/1, and publishes a :canvas_created PubSub event to update the LiveViews.
  • when I delete a canvas, using Canvas.delete_canvas/1, it deletes the canvas and persisted images from the DB and publishes a :canvas_deleted PubSub event for both the LiveView and the CanvasManager, which will in turn terminate the GenServer.
  • when the application starts, CanvasManager is started under the supervision tree and will start all canvases.

The GenServer is needed to interact with the canvas: get the version, get the image as PNG, get a pixel color, set a pixel color. The DB is only used to list the existing canvases and persist the image.

Problem
I have a hard time figuring out how to test the GenServer now that there is persistence. I understand the issue, but I don’t know what’s the best way to address it: when I create a new canvas, I call Canvases.create_canvas/1 via a fixture; this will also start the GenServer under my application’s supervisor (not ExUnit’s). To stop the GenServer a the end of the test, the fixture calls ExUnit.Callbacks.on_exit/1. The GenServer cannot work without the row in DB (will fail on termination), and I want to keep the supervision tree clean after each test. The issue is that on_exit is executed in a different process than the test one, which is not the owner of the sandbox DB connection.

I read in some issue on GitHub that Jose does not recommend updating the DB from inside on_exit, so I think that there is something wrong with my design. What would you suggest then?

  • Manually waiting in every test that creates a canvas for the GenServer to terminate?
  • Not starting GenServer in tests that don’t need it? How? And that does not solve the issue for the tests that do need it.
  • Start the processes using ExUnit.Callbacks.start_supervised/2? How can I do that? Would that solve the ownership issue?
  • Something else? I don’t have much ideas.

Please tell me if you need any more details. The full source code is on GitHub (“canvas” => “lobby”/“image”, the GenServer is called ImageServer, I renamed here for clarity).

Thanks for the help!
Thomas

* For the curious: I am using a straightforward Rust NIF via Rustler to be able to mutate images.

Where Next?

Popular in Questions Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

Other popular topics Top

skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52673 488
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 127089 1222
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54120 245
New

We're in Beta

About us Mission Statement