ConnorRigby

ConnorRigby

Nerves Core Team

Help implementing a notification system for Sqlite ecto adapter

So i’ve landed some features

In the lower level libs that make up the Sqlite.Ecto2 adapter. Now my problem is one of the actual implementation when working with Ecto. I know Ecto itself doesn’t support any notification system, but
the Postgrex adapter does support notifications. From skimming the source, i found that Postgrex just
opens another connection to the database which is something that sqlite can not support just because how it was built.

so one idea i had was to use Elixir.Registry, so every connection in the pool sends notifications out via Registry.

My issue is that sqlite sends a message that looks like
{:insert | :update | :delete, 'tablename', rowid}
(here are the docs for that)

Basically i get a rowid not a primary key or anything that is actually exposed to the Ecto Repo,
so even if an end user subscribed and got this message, they wouldn’t be able to easily lookup the data
that was affected because rowid is kind of an internal thing.

What i was thinking of doing is:

%{columns: c, rows: [row]} = Ecto.Adapters.SQL.query!(repo, "SELECT * FROM '#{table}' where rowid = #{rowid};", [])

And then sending that message to subscribers of the Registry, so a consumer would have to do:

receive do
   {action, 'users', %{columns: c, rows: [row]}} -> MyRepo.load(User, {result.columns, c, row})
end

The only problem i have with this is that every insert will also have a query to dispatch the notification,
and having user call load seems a little strange also.

Most Liked

ConnorRigby

ConnorRigby

Nerves Core Team

unfortunately the implementation of the message dispatching is handled by a NIF, and the convention for that NIF is to use charlists, because the lib is an Erlang library, not Elixir. Charlists are also faster for NIFs if i remember correctly, since it doesn’t have to allocate, convert to term, deallocate etc.

I don’t think Ecto encourages looking up a schema module at runtime, since multiple schemas could use the same table.

I guess dispatching the rowid isn’t that bad. I could make a wrapper module that one could use by supplying a schema and translate it for the user.

ConnorRigby

ConnorRigby

Nerves Core Team

I don’t think its a private api, just part of how the Sqlite3 virtual machine works. You can read about rowid here. Basically it exists, and you can use it to index things, but you can’t look at how it works. (as far as my limited knowledge of the sqlite public api goes anyway)

In most cases the rowid is the primary key, (although this isn’t always true), the only problem is that sqlite internally doesn’t actually dispatch the callback (what sends data back to Beam) until after the row has been destroyed.

There actually is commit and rollback hooks (described here) but that would add more complexity to the NIF than i would ideally give it.

idi527

idi527

That would be perfect, at least for me, I think.

Where Next?

Popular in Questions Top

Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

Other popular topics Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43622 214
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

We're in Beta

About us Mission Statement