Matches my understanding. Thank you for clarification.
I appreciate your work on the crate, and I had successfully used it just by looking at the signatures. Thank you.
I am willing to improve the docs once I find some free time. Is it ok to send a draft PR, so you can review for correctness?
Do you have an idea what is your current bottleneck? There are a several things you can do improve performance.
First, you can move business logic from GenServer into the callers. The idea is to serialise your HashSet into binary, and on request send relevant HashSet into caller. This is free since (large?) binaries are shared between processes. You can optimise even further and read ets directly from the caller, not sure though how well it performs. Computing in the caller processes will help utilising multiple cores of the cpu and reduce contention.
Second, you can optimise speed of your business logic. Either in elixir land, or by moving parts/all business logic into rust routines.
And lastly, if you still have a lot of contention on GenServer mailbox (after first idea), you can replace the whole ets table with rust object. You then wrap the rust object into a resource and just pass this reference around instead of GenServer. But my hunch is that ets is already doing sensible things, so you will not gain much without fancy concurrent data structures on the rust side.
Can you batch those?
Also, how large are your HashSets? 10k-100k, or more?
. DashMap is just a bunch of shards with RwLocks. I assume ets is doing something kinda like that (it is an embedded db after all - it should have locks). But to be honest, I don’t know much about ets internals.
