There’s no set rule but I would think you’d want a total of 200 or so items over those 5 keys before it makes sense performance wise to move to a different data structure. But actually analyzing the performance is always best.
axelson
Scenic Core Team
JoeZMar
The GenServer holding onto the ids has a few other jobs related to its purpose and the ids are very temporary and have no chance of changing in the future. If I am using two GenServers, mostly for the purpose of organization (one is responsible for the browser actions and the other is for delegating tasks needed to specific users). I have a Postgres db that ultimately stores the items that get picked up and by which users.
I am trying to move my state away from the boundaries of my app (funneling everything into a db) and using the BEAM more with things like GenServers, :ets, and even mnesia. So what would the difference between storing ids of users searching for items within a GenServer versus storing it within an :ets table?
I was just breaching a more advanced rails position when I left and began learning Elixir for my own projects. I do wish that I had more experience when I made the transition because building Rails apps finally started to seem easy then I abruptly changed to Elixir. I feel like l can see more of the bigger picture this time, but I’m still working out the kinks that really slow me down. I love OTP, I just have to figure it out. 
Edit: The other thing I want to note, there will be a user scraping the page and checking against the state in the GS once to several times a second.
I also chose Hound because the site is 100% javascript. It’s a tiny app, but does not work without it. I’ve only deconstructed one other site using MITM once before and didn’t know how I could simulate the same things.
I would also be willing to pay someone to help me build the rest of an app to further understand Elixir. I really want to start contributing to the community, but I’m just not quite there yet.
0
idi527
JoeZMar
This is what I just realized. It was the struct that put a kink in it for me. When I start my GS I initialize it with a map with the 5 values with empty lists now and it seems to work.
0
Eiji
It explains everything. You have all you need to create a relation like I described already.
If you have PostgreSQL database already then you do not need extra database. It only complicates your code. I was thinking about SQLite 3 (single file database) or :mnesia (which stores data in directory) for simplicity i.e. no need to setup database somewhere in /var/ or something like that. For example :mnesia is builtin in Erlang, so there is no need to install something like PostgreSQL. Everything changes if you already have database. Relation like I described is good, because it’s easy to extend (just by adding some extra fields).
This does not matters if you know how to write scrapers well. Basically you are doing something on site and inspecting it, so you see what request is send when you are registering or logging in etc. Having information about parameters, request and response content types and headers you can write any scraper without doing any JavaScript work. For some sites it’s easier, for some harder, but whole process looks really similar i.e. do something, observe requests/responses and try to reproduce them. Sometimes you need to read cookies from headers, sometimes token from response body, but scraper is scraper.
Look it’s pretty simple. For doing X scenario JavaScript code needs to send some requests and receive their responses. Developer need to inspect all of them and write Elixir version based on inspected data. Of course instead of static data like tokens you just need a simple function to fetch them from response dynamically.
As said I have experience writing scrapers. I can even write one for you/your company if you want. I just need to know url, full scenario (where to click, what to type etc.) and example data (like test account login data).
Feel free to reach me. 
0
Eiji
What do you mean?
defmodule Example do
def sample(data, value, keys \\ [:list_a, :list_b, :list_c]),
do: data |> Map.take(keys) |> Enum.find(&do_sample(&1, value))
defp do_sample({_key, list}, value) when is_list(list), do: value in list
end
defmodule ListStruct do
defstruct [:list_a, :list_b, :list_c]
end
Example.sample(%ListStruct{list_a: [1, 2, 3], list_b: [4, 5, 6], list_c: [7, 8, 9]}, 5)
{:list_b, [4, 5, 6]}
For me it’s working well.
Sorry, did you mean that you tried to do:
Example.sample(%ListStruct{list_a: nil, list_b: [4, 5, 6], list_c: nil}, 5)
If so then there are 3 solutions (depends on use case):
# 1. change struct definition:
defstruct [:list_a, :list_b, :list_c]
# to
defstruct [list_a: [], list_b: [], list_c: []]
# 2. use keys enforcing which prevents to create struct with nil values
@enforce_keys [:list_a, :list_b, :list_c]
# in such case this would fail at compile time:
%ListStruct{list_a: nil, list_b: [4, 5, 6], list_c: nil}
# 3. add extra guard:
defmodule Example do
def sample(data, value, keys \\ [:list_a, :list_b, :list_c]),
do: data |> Map.take(keys) |> Enum.find(&do_sample(&1, value))
defp do_sample({_key, nil}, value), do: false
defp do_sample({_key, list}, value) when is_list(list), do: value in list
end
0