Constantizer Package, thoughts around Configs

Repo: https://github.com/aaronrenner/constantizer

I want to make the Repos in some of my packages actually pluggable. Many of you probably want to control how many connections you have to the database.

So by doing this, I will move the repo to the config.

I learned about Constantizer package, and I understand what it is doing and how is doing it.

But,

I want to hear your thoughts about it and collect ideas and opinions (maybe this type of feature should live on Elixir itself by improving the Config module.

Would you use it?

Is there any performance concern by not doing this?

Since it will read from the ETS table still slower than using the compilation time for these things, but at what point becomes an issue :man_juggling: no clue.

Please share your thoughts

1 Like

https://engineering.tripping.com/blazing-fast-elixir-configuration-475aca10011d

This should give you some hints. Personally I’d not use something like constantizer (or fastglobal) until I have a proven case in production, where Application.get_env is indeed the bottle neck. The other case when I would consider it is when the data is no longer just reasonable sized configuration.

2 Likes

I just put together some benchmarks on calling Application.get_env/2 at runtime vs evaluating at compile time with defconst. Looks like calling it at run-time is about 18x slower than having it evaluated at compile-time:

Name                          ips        average  deviation         median         99th %
compile_time_lookup       87.43 M      0.0114 μs    ±17.43%      0.0110 μs      0.0130 μs
runtime_lookup             4.62 M        0.22 μs    ±41.15%        0.21 μs        0.40 μs

Comparison:
compile_time_lookup       87.43 M
runtime_lookup             4.62 M - 18.93x slower

Chances are you probably won’t notice a difference either way since ETS is so fast. But still, I’d personally take the free performance gain and lighten the load on ETS instead of trying to figure out what’s going on when there is a bottleneck in the future.

Also, one other note on this comment:

Instead of making the Ecto Repos themselves pluggable, it might be worth setting up an API boundary in front of the repo that exposes only the functions your application depends on (like find_user, list_projects) instead of allowing your app to depend on the full Ecto API. This will make swapping in whatever other backend you want to use much simpler instead of having to implement Ecto-specific APIs like get, insert that depend on Ecto queries and changeset. This mox issue further echos this point where Jose is talking about why Mox requires a behaviour instead of just letting you replace a module.

Sorry for getting things off track, but I thought I’d mention it if you were planning on doing the swapping right at the Ecto repo.

1 Like

No please, by all means, go off track.

I am confused by your suggestion. Review for example, uses a Repo (dah) so I want to actually plug the Repo just because people would like to control the number of connections, I guess (somebody asked me for this)

Right now most of the packages have its own Repo (which is actually fine for me since my underline infrastructure is counting on this, for example, a separate database for each subsystem) so, yeah this is why I am doing.

But still, I am confused with reading that thread, sorry.

If the goal is to allow swapping in another ecto Repo module, swapping at the repo makes sense.

When I first read your post, I thought the goal was to allow for an entirely different storage backend. In that case, you could define a domain level API boundary in front of your ecto storage logic (which you appear to already have :slightly_smiling_face:) that allows for a different, non-ecto storage backend to be swapped in (Here is how that would work if you were going that direction: https://youtu.be/Ue--hvFzr0o?t=16m4s). However, if this library is only ever going to be used with Ecto and if tests don’t require an overly complex database setup, then a swappable storage layer probably isn’t needed.

1 Like