jstimps

jstimps

Writing a new Ecto Adapter (not SQL), unable to use Ecto Migrations effectively

Hi,

I’m working on an Ecto Adapter for FoundationDB and one of my goals is to support migrations in the same manner that Ecto users are accustomed to (e.g. mix ecto.migrate). As I familiarized myself with the adapter behaviours, I learned that the Ecto.Adapter.Migration behaviour is a module in ecto_sql instead of the base ecto.

My first question: Is it reasonable or folly to continue developing a migration using ecto_sql as a depedency even though my database backend does not support SQL?

Of course I’m no stranger to folly, and I went ahead and tried it anyway, and got it mostly working. The rest of this post is related to the one blocker I had in getting it working seamlessly.

My desired implementation for Ecto.Adapter.Migration.lock_for_migrations/3 acquires a resource (a transaction reference from :erlfdb) that must then be telegraphed down to each execute_ddl. Ideally this could be directly supported by ecto_sql by allowing the behaviour to influence any future Repo operations within the execution of the migration.

Without this direct support from the library, a quick and dirty approach would be to store the reference in the Process dictionary, and then pick it back up later on in execute_ddl. However, this doesn’t work either because of the following Task creation:

Ecto.Migrator

    fn -> run_maybe_in_transaction(repo, dynamic_repo, module, fun_with_status, opts) end
    |> Task.async()
    |> Task.await(:infinity)

(The use of a Task here ensures that the rest of the migration steps have a fresh Process dictionary, preventing me from using it to keep track of data)

My second question: Why does ecto_sql use a Task in this manner?

Finally: Are there any other solutions that I’m missing without changing the underlying ecto_sql implementation? I’m reluctant to use global state such as an ets table unless it’s a last resort.

Thanks for your time!

Jesse

Most Liked

Schultzer

Schultzer

Since the adapter is unaware of the user setup, global is a good choice, as it works regardless of whether the node is in a cluster. The adapter is still very much in the exploration stage but is usable. With the occasional bugs here and there that I’m finding while building a different product.

I think it might be worth proposing in https://groups.google.com/g/elixir-ecto to make Ecto.Migrator a behaviour and move that and Ecto.Migration into ecto main and have adapters implementing them since we both could use the flexibility and control that would bring. And it might get some more clarity over the choice already made, which is unclear for both of us, and I’m sure there will be other people in the future running into this.

/edit
Keep in mind that some people don’t use global since AFAIK the regular Erlang distribution is a full mesh, and that can bring some pain when running a large cluster, However, I don’t believe that running one large cluster is a silver bullet, and there might be more gains by divide and conquer and running multiply full mesh clusters that can communicate.

Where Next?

Popular in Questions Top

siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36128 110
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement