JILL24
Sharding of ets table
suppose a case where single ets is storing users and users start to grow too much in ets table. now i want to shard ets table in the same node means local node.
is it possible? and if it is possible, how?
note - Don’t use 3rd party library. just custom built.
Most Liked
jhogberg
I think you should take a step back and ask yourself why you’re having this problem to begin with.
Do you require that both reads and updates are executed sequentially?
If you don’t require that at all, then the server process is pointless and you’re much better off letting the clients access the table directly, which will drastically improve performance.
If you only require that updates run sequentially, then the server process is only necessary for updates and reads can access the table directly, which will greatly improve performance.
If you require that both reads and updates run sequentially, then I’d love to hear why. Either way you need to take a long hard look at your application’s architecture as routing this much traffic through a single process is not normal.
The concurrency options generally make the table faster for concurrent operations at the cost of making each individual operation slower. If there are no concurrent operations, such as in your case with a single server process being the only one to touch the table, then they make things slower for nothing.
LostKobrakai
hst337
ets sharding is used not to increase size, it is used to increase speed of parallel lookups/updates to reduce lock contention. Do not use ets sharding unless you have a reason to do so.
Last Post!
Nicoo
You received to much XY responses. Sounds like you just desperate to get the answer to your question.
One solution (maybe not ideal) is to make a logical separation.
Example:
Define a field “data_table” in the “user” table.
Let’s imagine that the table you want to shard is “user_data”.
You have 4 tables (shards) of “user_data”: user_data1, user_data2, user_data3, user_data4.
For each user, you can assign a shard to user.data_table field (user.data_table = "user_data1").
This approach will improve some cases, surely make others worse. It all depends on your case.
In a multi-tenant logic, it can do it.
This is not the ideal answer to your question but just to advance in the reflection for sharding
Popular in Discussions
Other popular topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Sponsor Spotlight
Deploy Phoenix to your own VPS with PaaS ease: push-to-deploy, zero downtime, auto SSL, managed Postgres. 14-day free trial at potions.io
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"