idi527

idi527

Filtering outgoing messages to pubsub by location

Sorry for the bullet points

I have

  • a database table for “posts” each of which has a location (postgis point)
  • users subscribing (phoenix channel) for updates in posts near their location (also postgis point, but stored in an ets table)

I want

  • whenever a new post is inserted into the database (not necessarily from elixir app), notify (via phoenix channels) every user who has joined (and is still connected to the channel) and is within some radius around new post’s location

What I’ve tried

  • created a trigger that calls NOTIFY with post data whenever a post is inserted
  • subscribed to these NOTIFY messages from postgres via Postgrex.Notifications

What I’m struggling with

  • right now I receive every post and need to “filter” and “dispatch” them on the elixir side, which is slower than I’d like (I keep subscribers’ (users) current locations and their socket ids in an ets table)

I’d like to filter posts in postgres before calling NOTIFY, but don’t know how to do that since postgres does not know the parameters by which to filter the data (currently connected users’ locations).

Should I try storing currently connected users’ locations in some table?

Should I try replacing phoenix pubsub (pg2) with postgres pubsub? Is that possible?

Postgres uses one process per connection, so it’s much harder to scale.

from One of the big differences that RethinkDB has going for it is that its connectio... | Hacker News, so probably not.

Would appreciate any advice.


It’s probably difficult to understand what it is I want. I’ll try to create a repo on github with what I have already.

Most Liked

outlog

outlog

this sounds like the best plan - to allow GIS querying/filtering at the earliest possible..

what I don’t understand is how it can be slow currently, what is the current code/flow?

this I don’t quite understand do you have multiple listeners to the NOTIFY? - I would have one listener that gets the post INSERT and then queries/filters the user lat/lng data against the post lat/lng and does the appropriate channel broadcast(s) - also do you have one channel that all users share or do you have per user channels?

outlog

outlog

one idea: I notice each user can have a different radius, so when you get to saving that to postgres, it might make sense to store that as a geo shape in pg (and not a point) - so that table has a lot of different “circles” (or rather polygon approximations I think) in it and then when the post comes in you can have a very simple (and fast!) query for those where the post geo point is within.. (and all the individual user radii etc is taken care of.)

also you probably want to process trap_exit and then mark that location as inactive in the channel terminate (or delete it)..

keep us posted..

btw: there is boltun for the postgres notifications stuff - boltun | Hex

Where Next?

Popular in Questions Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> somethi...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

Other popular topics Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47930 226
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement