Hey Elixir community,
I have a question, i’m building a communication between php and elixir via socket. I can’t manage to received string bigger than 1460 characters… Every string bigger than this is troncated and send after. Are-there any options to get over this ? thank you very much.
I’m using socket function on the PHP side. Elixir is the socket server and php, the client.
Isn’t it the maximum segment size? I would guess there is no way around that.
Damn, ok …
Wikipedia says [0]
For most computer users, the MSS option is established by the operating system.
So maybe you can change it on your OS?
Use UDP, that will automatically give you packets as large as 65,507 bytes. You can also try IPC or unix domain socket, which has even greater message limit.
If you use TCP, you should implement message framing logic, which complicates logic on the both ends.
Finally, you can always use Redis pub/sub or similar.
thanks a lot for your answer, i will look into these
Uhh, I’m curious about a lot of the responses here…
TCP is a streaming protocol, it already implements everything suggested here (not always efficiently, but it works), and your use case as you described it in your first post should absolutely work.
Can you make a minimal reproducable test-case for your php sender and elixir receiver, I can take a look at the code and test it and see what is going on.
From an initial guess (due to the complete lack of code in the post, hint hint ^.^) I’m guessing the receive options are not set as you expect and are processing the information as it arrives instead of as complete chunks. 
defmodule TaskyRabbit.Socket do
use GenServer
defmodule State do
defstruct port: nil, lsock: nil, socket: nil
end
#######
# API #
#######
def start_link(port) do
GenServer.start_link(__MODULE__, port, name: __MODULE__)
end
def init(port) do
IO.puts(port)
current = self()
tcp_options = [:list, {:packet, 0}, {:active, false}, {:reuseaddr, true}, {:packet_size, 50}]
# tcp_options = [:binary, packet: 0, active: false, reuseaddr: true]
{:ok, l_socket} = :gen_tcp.listen(port, tcp_options)
send(current, :accept)
{:ok, %State{port: port, socket: l_socket}}
end
def handle_info(:accept, state = %{socket: socket}) do
do_listen(socket)
{:noreply, state}
end
defp do_listen(l_socket) do
{:ok, socket} = :gen_tcp.accept(l_socket)
spawn(fn() -> do_server(socket) end)
do_listen(l_socket)
end
defp do_server(socket, array \\ []) do
case :gen_tcp.recv(socket, 0) do
{ :ok, data } ->
IO.inspect (data), label: "received"
# TaskyRabbit.tcp_rbmq(data)
{ :error, :closed } ->
IO.inspect "closed"
:ok
end
end
end$json = json_encode($response);
$host = "127.0.0.1";
$port = 4040;
$socket = socket_create(AF_INET6, SOCK_STREAM, SOL_TCP);
$connection = socket_connect($socket, $host, $port);
socket_write($socket, $json);
the PHP code
I’m surprised you don’t want to receive it as binaries. ^.^
So there is no format to the packets at all? This sounds like your problem. Remember that TCP is a streaming protocol, you need to have something formatted on the line. Traditionally you have a 1/2/4 byte header of an integer specifing the size of this ‘chunk’ of data, but by putting ‘0’ here that means you will have to do all of the filtering and chunking yourself manually.
Here you are encoding json and sending it to the socket, but you are not saying what ‘size’ it is or anything of the sort. I’d recommend either putting a 4-byte integral size header at the front of it or making sure the json is a single line only (replace newlines in string with \n as per the json standard and remove all formatting newlines) and separate chunks with newlines, the {:packet, :line} option to :gen_tcp can parse those out for you.
If however you want to just dump json to the socket then know that you will receive it in MTU chunks and you will need to combine them and do your own chunking manually, not hard to do though:
You would change this up, receive the data and try to parse the json, if the json fails to parse then put it in an argument and recv again, then append that to the first and try to parse again, repeat until the parse does not fail. Once the parse passes then take any remaining data and put it in that holding argument (remember, you might get multiple json’s in a single packet!!).
However, none of that is necessary if you delinate the tcp chunks properly, like either via an integer header (always the easiest in my opinion) or breaking up on ‘lines’. 
You can do the ‘header’ via changing the packet argument in elixir to {:packet, 4} and doing something like this in PHP (I don’t remember the actual functions so replace them with what they actually are):
$json = json_encode($response);
$json_size = str_bytelength($json);
$host = "127.0.0.1";
$port = 4040;
$socket = socket_create(AF_INET6, SOCK_STREAM, SOL_TCP);
$connection = socket_connect($socket, $host, $port);
socket_write_integer_32bit($socket, $json_size);
socket_write($socket, $json);
Or whatever the right php commands for that is…
Or use {:packet, :line} and make sure the json is a single line and append a newline at the end of each write of the json.
Ok so on elixir my options must be tcp_options = [:binary, {:packet, 4}, :line, active: false, reuseaddr: true] ?
That will be perfect ‘if’ you prepend the 4 integer bytes of the size of the content to your json data.
I found this to help: php - Sending sockets data with a leading length value - Stack Overflow
Do note it must be in network-byte order, so if the length seems wrong (as in you may not receive anything) swap the bytes, it must be in network-byte order.
But according to the link, you’ll use the php pack function/2 (probably the ‘N’ option I think).
i have response like this
<<0, 0, 26, 53, 123, 34, 97, 99, 116, 105, 111, 110, 34, 58, 34, 68, 101, 99,
105, 115, 105, 111, 110, 67, 111, 109, 112, 108, 101, 116, 101, 100, 34, 44,
34, 117, 117, 105, 100, 34, 58, 34, 56, 54, 52, 98, 54, 99, 102, 99, …>>
received: “b3815b922fee5f88ced775c28”,“position”:"+0|0+8|0",“action”:“ScheduleTask”,“input”:"{\“data\”:{\“n\”:\"{\\\“data\\\”:4}\"}}"},{“name”:“EchoDuration”,“hash”:“83e93c3b3815b922fee5f88ced775c28”,“position”:"+0|0+9|0",“action”:“ScheduleTask”,“input”:"{\“data\”:{\“n\”:\"{\\\“data\\\”:4}\"}}"},{“name”:“EchoDuration”,“hash”:"83e93c3b3815b922fee5f88ced775c28\
I’ve done everything you tell me and i have response like this, I don’t really understand how can i manage it after ? Sorry but i’m not used to socket protocol
Can you show your code? That is not looking like {:packet, 4} is set? Though the data looks good now.
defmodule TaskyRabbit.Socket do
use GenServer
defmodule State do
defstruct port: nil, lsock: nil, socket: nil
end
#######
# API #
#######
def start_link(port) do
GenServer.start_link(__MODULE__, port, name: __MODULE__)
end
def init(port) do
IO.puts(port)
current = self()
# tcp_options = [:binaries, {:packet, 0}, {:active, false}, {:reuseaddr, true}]
# tcp_options = [:binary, packet: 0, active: false, reuseaddr: true]
{:ok, l_socket} = :gen_tcp.listen(port, [:binary, {:packet, 4}, {:packet, :line}, active: false, reuseaddr: true])
send(current, :accept)
{:ok, %State{port: port, socket: l_socket}}
end
def handle_info(:accept, state = %{socket: socket}) do
do_listen(socket)
{:noreply, state}
end
defp do_listen(l_socket) do
{:ok, socket} = :gen_tcp.accept(l_socket)
spawn(fn() -> do_server(socket) end)
do_listen(l_socket)
end
defp do_server(socket, string \\ "") do
case :gen_tcp.recv(socket, 0) do
{ :ok, data } ->
case message = Poison.Parser.parse(data) do
{:ok, json} ->
IO.puts "good"
{:error, :invalid, t} ->
new_string = "#{string}#{data}"
do_server(socket, data)
{:error, {:invalid, t, _}} ->
do_server(socket, data)
end
IO.inspect (data), label: "received"
# TaskyRabbit.tcp_rbmq(data)
{ :error, :closed } ->
IO.inspect "closed"
:ok
end
end
end$host = "127.0.0.1";
$port = 4040;
$json = json_encode($response);
$length=pack("N",strlen($json));
$socket = socket_create(AF_INET6, SOCK_STREAM, SOL_TCP);
$connection = socket_connect($socket, $host, $port);
socket_write($socket, $length.$json, 4+strlen($json));Duplicates detected! ^.^ Remove the , {:packet, :line} part, that is overriding the {:packet, 4} option.
So no :line options though ?
Wow way better ! you’re so nice it seems to be good.
Correct, that is only if you are doing a line-oriented protocol. By putting a prefix-length you are doing a length-oriented protocol (which is always superior in my opinion as you can represent *anything* in it).
Only one or the other.
© Copyright Elixir Forum | Terms | Privacy & Cookies
