webuhu
I have the honor to program a plant filter search.
Means you can search plants meeting certain conditions (attributes).
But I need some advice for counting number of occurring attributes.
Over simplified records:
- id: 1; attributes: [color: [red, green], size: 10];
- id: 2; attributes: [color: [red], light: [moderate];
The attributes are varying!
The filter search functionality should provide a counter for each attribute:
color: red (2), green (1)
size: 10 (1)
light: moderate (1)
Now if you’ll select moderate - green & size will be (0), red (1).
Sadly PostgreSQL is getting to its limit with this requirement.
Counting attributes is too slow.
So I’m in search of an alternative approach. → Advice needed!
More details:
- It concerns about 2,5 k records
- Attributes of all records will have a size of “just” about 10 - 15 MB
- The first most common filters will reduce the record amount just by about 20%
- Update of records / attributes won’t be very frequent
Ideas:
Should I just cache a list of all records + attributes in for example ETS?
And than do the counting / filtering with some Enum operations?
Or are there any other good approaches I’m currently blind for?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Error tracking, logs, APM, uptime and cron monitoring in one dev-friendly platform. Find and fix problems before your users notice.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 4- Show Best Posts
- Show All Posts (oldest first)
- Show All Posts (newest first)
joey_the_snake
It sounds like you are trying to do faceted search. There are some tips in this article that might be helpful for you: PostgreSQL Faceted navigation and search
It is focused on how to do it in Postgres. With 2.5K records I believe you should be able to make this fast just using Postgres.
webuhu
Thanks Joey. Ok, it definitely sound like it’s called faceted search, what I’m trying to do.
The article is very promising. I myself would never have had the idea to solve this via full text search and
ts_stat. I will definitely try this way. It will get interesting if some of my deep nested attributes are compatible with this concept. I will see…Of course I’ll be happy if it works with just Postgres. Because the record set isn’t really that large.
webuhu
So this works perfectly fine. Thanks a lot.
joey_the_snake
I’m glad it works :). Thanks for letting me know. I have plans to try it in my own project but haven’t had the time yet.