ETS for TCP connections

smanza

Hello.

I saw on many posts and implementations for TCP connection management managed using GenServer or GenStatem, which is great with all the message passing, reactive, etc…
However, it can have strong risk of bottlenecks and overloading of the mailbox;

I ran some benchmarks, and tested an implementation of a TCP client using ETS to manage the socket.
Even if it’s not use message passing and synchronous reading of the socket, it still way faster, and even more when we add load or parallelization. So I don’t get timeouts easily.
It seems also a win/win as there is no need or less for process pooling, which tends to reduce the overhead.

What do you think about managing TCP connection by ETS ?

Thanks

15 comments

#ets #genserver #gen_tcp

0 2564 15

2021-10-21 09:56:30 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

Matsa59

Hello

I’m not sure to understand what/how/why you use ETS for TCP connection but have take a look on Registry ? Pretty sure it could help you a lot

I don’t know how you manage the TCP state in ETS (what happen if a TCP tunel is closed etc). In registry, you’ll follow the erlang mind of wrap everything in a process to let it crash.

Edit : I’m interesting to see some code if you can ^^

Post #1

smanza

The reason is the overload of the mailbox of the GenServer. I also tried GenStage but not better than GenServer.

For connection closed and family , I ended up checking on each message the tuple return , to determine if I need to reconnect or not. It’s more like as demand.

A better version could be to use a set of
socket (like a pool but without server) and switch one of them if the socket is closed, while reconnecting in background .

Post #2

Matsa59

It’s about connections from your server or to your server ?

If it’s from your server, GenServer is not a good solution (and registry will not help you). GenStage is not a big game changer (it will only help you to manage data flow).

But you can spawn process and monitor them from a GenServer. So the GenServer will be aware if a process is down or w.e (take a look at spawn_link, spawn, monitor, process etc)

I suppose you create x connections through x endpoints. 1 connection per endpoint should be enough. Maybe a lib like gun could also be a good resource

Post #3

smanza

It is to my server, as a client. In fact I used GenServer for the connection acceptor on the server side.

I did use GenServer for client with active: :once and noreoply while keeping the requesters during but as some point there are some bottleneck or mailbox overload (issue with unbounded queue).

So I did try multiple solutions semaphore, ets …

Post #4

dom

Have you looked into how Ecto manages database connection pools? DBConnection — db_connection v2.10.1

Each connection is a process, but once you checkout a connection, you can use the socket directly, without going through the process that owns it. I’d be curious if this library work for your use case or if you have different constraints.

Post #5

smanza

Indeed it uses ETS to store the socket and pass it to the client during a given time internal.

Post #6

marcin

How about using N gen servers, and shard connections between them?
To find the processes pid, use a Register

Post #7

rvirding

Creator of Erlang

Have you tried using {:active,:once} to limit when incoming messages are sent to your TCP process?

Post #10

smanza

Yes I did try with both client and server , but the issue I’m facing it is not from the TCP stack but from the clients wanting to send message and wait response from this socket. (Typically from the GenServer call and no GenServer info callback). For example to have hundred of parallel requests in the same time.

Post #11