ConnorRigby
Help implementing a notification system for Sqlite ecto adapter
So i’ve landed some features
In the lower level libs that make up the Sqlite.Ecto2 adapter. Now my problem is one of the actual implementation when working with Ecto. I know Ecto itself doesn’t support any notification system, but
the Postgrex adapter does support notifications. From skimming the source, i found that Postgrex just
opens another connection to the database which is something that sqlite can not support just because how it was built.
so one idea i had was to use Elixir.Registry, so every connection in the pool sends notifications out via Registry.
My issue is that sqlite sends a message that looks like
{:insert | :update | :delete, 'tablename', rowid}
(here are the docs for that)
Basically i get a rowid not a primary key or anything that is actually exposed to the Ecto Repo,
so even if an end user subscribed and got this message, they wouldn’t be able to easily lookup the data
that was affected because rowid is kind of an internal thing.
What i was thinking of doing is:
%{columns: c, rows: [row]} = Ecto.Adapters.SQL.query!(repo, "SELECT * FROM '#{table}' where rowid = #{rowid};", [])
And then sending that message to subscribers of the Registry, so a consumer would have to do:
receive do
{action, 'users', %{columns: c, rows: [row]}} -> MyRepo.load(User, {result.columns, c, row})
end
The only problem i have with this is that every insert will also have a query to dispatch the notification,
and having user call load seems a little strange also.
Most Liked
ConnorRigby
unfortunately the implementation of the message dispatching is handled by a NIF, and the convention for that NIF is to use charlists, because the lib is an Erlang library, not Elixir. Charlists are also faster for NIFs if i remember correctly, since it doesn’t have to allocate, convert to term, deallocate etc.
I don’t think Ecto encourages looking up a schema module at runtime, since multiple schemas could use the same table.
I guess dispatching the rowid isn’t that bad. I could make a wrapper module that one could use by supplying a schema and translate it for the user.
ConnorRigby
I don’t think its a private api, just part of how the Sqlite3 virtual machine works. You can read about rowid here. Basically it exists, and you can use it to index things, but you can’t look at how it works. (as far as my limited knowledge of the sqlite public api goes anyway)
In most cases the rowid is the primary key, (although this isn’t always true), the only problem is that sqlite internally doesn’t actually dispatch the callback (what sends data back to Beam) until after the row has been destroyed.
There actually is commit and rollback hooks (described here) but that would add more complexity to the NIF than i would ideally give it.
idi527
That would be perfect, at least for me, I think.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #javascript
- #code-sync
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








