Gen:tcp socket maximum number of bytes/characters possible to receive

Isn’t it the maximum segment size? I would guess there is no way around that.

Damn, ok …

Wikipedia says [0]

For most computer users, the MSS option is established by the operating system.

So maybe you can change it on your OS?

[0] Maximum segment size - Wikipedia

Use UDP, that will automatically give you packets as large as 65,507 bytes. You can also try IPC or unix domain socket, which has even greater message limit.
If you use TCP, you should implement message framing logic, which complicates logic on the both ends.
Finally, you can always use Redis pub/sub or similar.

thanks a lot for your answer, i will look into these

Uhh, I’m curious about a lot of the responses here…

TCP is a streaming protocol, it already implements everything suggested here (not always efficiently, but it works), and your use case as you described it in your first post should absolutely work.

Can you make a minimal reproducable test-case for your php sender and elixir receiver, I can take a look at the code and test it and see what is going on.

From an initial guess (due to the complete lack of code in the post, hint hint ^.^) I’m guessing the receive options are not set as you expect and are processing the information as it arrives instead of as complete chunks.

defmodule TaskyRabbit.Socket do
  use GenServer


  defmodule State do
     defstruct port: nil, lsock: nil, socket: nil
  end
  #######
  # API #
  #######

  def start_link(port) do
    GenServer.start_link(__MODULE__, port, name: __MODULE__)
  end

  def init(port) do
    IO.puts(port)
    current = self()
    tcp_options = [:list, {:packet, 0}, {:active, false}, {:reuseaddr, true}, {:packet_size, 50}]
    # tcp_options = [:binary, packet: 0, active: false, reuseaddr: true]
    {:ok, l_socket} = :gen_tcp.listen(port, tcp_options)
    send(current, :accept)
    {:ok, %State{port: port, socket: l_socket}}
  end



  def handle_info(:accept, state = %{socket: socket}) do
    do_listen(socket)
    {:noreply, state}
  end


  defp do_listen(l_socket) do
    {:ok, socket} = :gen_tcp.accept(l_socket)

    spawn(fn() -> do_server(socket) end)

    do_listen(l_socket)
  end


  defp do_server(socket, array \\ []) do
    case :gen_tcp.recv(socket, 0) do
      { :ok, data } ->
        IO.inspect (data), label: "received"
        # TaskyRabbit.tcp_rbmq(data)
      { :error, :closed } ->
        IO.inspect "closed"
        :ok
    end
  end
end

$json = json_encode($response);
$host = "127.0.0.1";
$port = 4040;
$socket = socket_create(AF_INET6, SOCK_STREAM, SOL_TCP);
$connection = socket_connect($socket, $host, $port);
socket_write($socket, $json);

the PHP code

I’m surprised you don’t want to receive it as binaries. ^.^

So there is no format to the packets at all? This sounds like your problem. Remember that TCP is a streaming protocol, you need to have something formatted on the line. Traditionally you have a 1/2/4 byte header of an integer specifing the size of this ‘chunk’ of data, but by putting ‘0’ here that means you will have to do all of the filtering and chunking yourself manually.

Here you are encoding json and sending it to the socket, but you are not saying what ‘size’ it is or anything of the sort. I’d recommend either putting a 4-byte integral size header at the front of it or making sure the json is a single line only (replace newlines in string with \n as per the json standard and remove all formatting newlines) and separate chunks with newlines, the {:packet, :line} option to :gen_tcp can parse those out for you.

If however you want to just dump json to the socket then know that you will receive it in MTU chunks and you will need to combine them and do your own chunking manually, not hard to do though:

You would change this up, receive the data and try to parse the json, if the json fails to parse then put it in an argument and recv again, then append that to the first and try to parse again, repeat until the parse does not fail. Once the parse passes then take any remaining data and put it in that holding argument (remember, you might get multiple json’s in a single packet!!).

However, none of that is necessary if you delinate the tcp chunks properly, like either via an integer header (always the easiest in my opinion) or breaking up on ‘lines’.

You can do the ‘header’ via changing the packet argument in elixir to {:packet, 4} and doing something like this in PHP (I don’t remember the actual functions so replace them with what they actually are):

$json = json_encode($response);
$json_size = str_bytelength($json);
$host = "127.0.0.1";
$port = 4040;
$socket = socket_create(AF_INET6, SOCK_STREAM, SOL_TCP);
$connection = socket_connect($socket, $host, $port);
socket_write_integer_32bit($socket, $json_size);
socket_write($socket, $json);

Or whatever the right php commands for that is…

Or use {:packet, :line} and make sure the json is a single line and append a newline at the end of each write of the json.

Ok so on elixir my options must be tcp_options = [:binary, {:packet, 4}, :line, active: false, reuseaddr: true] ?