I can connect using :gen_tcp, but I can't send/receive data - what is wrong with this implementation?

All I want it to do is emulate the following netcat TCP command, which is working properly from my command line:

$ echo "|c country_US" | nc 10.247.4.104 26542
0.500000        <-- response

To accomplish this I have an Elixir module that uses GenServer and :gen_tcp as follows (it has a few extra IO.puts and IO.inspect calls so I can watch progress):

defmodule VowpalWabbex do
    use GenServer

    def predict(pid, client_data) do
        GenServer.call(pid, {:predict, client_data})
    end

    @initial_state %{socket: nil}

    def start_link do
        GenServer.start_link(__MODULE__, @initial_state)
    end

    def init(state) do
        IO.puts "VowpalWabbex - init..."
        opts = [:binary, :inet, active: false, packet: :line]
        {:ok, socket} = :gen_tcp.connect({10, 247, 4, 104}, 26542, opts)
        IO.inspect socket
        {:ok, %{state | socket: socket}}
    end

    def handle_call({:predict_adx_inflation, client_data}, _from, %{socket: socket} = state) do
        IO.inspect client_data
        IO.inspect socket
        :ok = :gen_tcp.send(socket, client_data)
        IO.puts :ok
        {:ok, msg} = :gen_tcp.recv(socket, 0)
        {:reply, msg, state}
    end

end

I would use this module from within IEX as follows:

$ iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Compiling 1 file (.ex)

iex(1)> {:ok, pid} = VowpalWabbex.start_link
VowpalWabbex - init...
#Port<0.6774>
{:ok, #PID<0.238.0>}

iex(2)> VowpalWabbex.predict(pid, "|c country_US")
"|c country_US"
#Port<0.6774>
ok

** (exit) exited in: GenServer.call(#PID<0.238.0>, {:predict, "data"}, 5000)
** (EXIT) time out

So apparently I get an ā€œokā€ response when I send the data, but then the I never get a response and instead I time out in a couple of seconds.

Am I doing something wrong in how Iā€™m trying to receive the response?

Thanks much!

1 Like

Does your server send a \n at the end of the package or is it just a fixed with string? That bunch of zeros at the end makes me suspiciousā€¦

And again, have you checked using tcpdump and friends what is really sent over the wire?

2 Likes

Especially this because you are doing packet: :line so every single ā€˜statementā€™ needs to be terminated with a \n from the remote connection.

2 Likes

@OvermindDL1 I just added packet: :line on a whim to see if that would fix my issue and it did not ā€“ I see the same behavior when I take it out and have just opts = [:binary, active: false]

I have confirmed that there is a newline at the end of the number that is returned, though.

@NobbZ I did watch tcpdump but Iā€™ll have to become much more expert with it to really understand what Iā€™m looking at.

In brief, though: VowpalWabbex.start_link results in this tcpdump output:

17:42  IP 172.16.17.35.52949 > 10.247.4.104.26542: Flags [S], seq 2103055949, win 65535, options [mss 1360,nop,wscale 5,nop,nop,TS val 237205327 ecr 0,sackOK,eol], length 0
17:42  IP 10.247.4.104.26542 > 172.16.17.35.52949: Flags [S.], seq 857525256, ack 2103055950, win 28960, options [mss 1379,sackOK,TS val 1819819275 ecr 237205327,nop,wscale 9], length 0
17:42  IP 172.16.17.35.52949 > 10.247.4.104.26542: Flags [.], ack 1, win 4128, options [nop,nop,TS val 237205434 ecr 1819819275], length 0

To me that looks like a handshake ā€“ three message: first from me to him, second an ā€œackā€ from him to me, and then third an ā€œackā€ from me to him.

After I try to make the send/2 call, tcpdump shows:

17:44  IP 172.16.17.35.52949 > 10.247.4.104.26542: Flags [P.], seq 1:14, ack 1, win 4128, options [nop,nop,TS val 237323302 ecr 1819819275], length 13
17:44  IP 10.247.4.104.26542 > 172.16.17.35.52949: Flags [.], ack 14, win 57, options [nop,nop,TS val 1819848826 ecr 237323302], length 0

I donā€™t see my actual data there, but the length 13 in the first line is correct, since Iā€™m sending |c country_US which has exactly 13 length.

However, when I observer tcpdump for echo "|c country_US" | nc 10.247.4.104 26542, I get this output:

18:04  IP 172.16.17.35.53085 > 10.247.4.104.26542: Flags [S], seq 1747162267, win 65535, options [mss 1360,nop,wscale 5,nop,nop,TS val 238489788 ecr 0,sackOK,eol], length 0
18:04  IP 10.247.4.104.26542 > 172.16.17.35.53085: Flags [S.], seq 3797483678, ack 1747162268, win 28960, options [mss 1379,sackOK,TS val 1820141152 ecr 238489788,nop,wscale 9], length 0
18:04  IP 172.16.17.35.53085 > 10.247.4.104.26542: Flags [.], ack 1, win 4128, options [nop,nop,TS val 238489900 ecr 1820141152], length 0
18:04  IP 172.16.17.35.53085 > 10.247.4.104.26542: Flags [P.], seq 1:15, ack 1, win 4128, options [nop,nop,TS val 238489900 ecr 1820141152], length 14
18:04  IP 172.16.17.35.53085 > 10.247.4.104.26542: Flags [F.], seq 15, ack 1, win 4128, options [nop,nop,TS val 238489900 ecr 1820141152], length 0
18:04  IP 10.247.4.104.26542 > 172.16.17.35.53085: Flags [.], ack 15, win 57, options [nop,nop,TS val 1820141179 ecr 238489900], length 0
18:04  IP 10.247.4.104.26542 > 172.16.17.35.53085: Flags [F.], seq 10, ack 16, win 57, options [nop,nop,TS val 1820141179 ecr 238489900], length 0
18:04  IP 172.16.17.35.53085 > 10.247.4.104.26542: Flags [.], ack 1, win 4128, options [nop,nop,TS val 238490007 ecr 1820141179,nop,nop,sack 1 {10:11}], length 0
18:04  IP 10.247.4.104.26542 > 172.16.17.35.53085: Flags [P.], seq 1:10, ack 16, win 57, options [nop,nop,TS val 1820141179 ecr 238489900], length 9
18:04  IP 172.16.17.35.53085 > 10.247.4.104.26542: Flags [.], ack 11, win 4127, options [nop,nop,TS val 238490007 ecr 1820141179], length 0

One difference I see is that length 14 is sent from netcat, while length 13 is sent from Elixir ā€“ maybe I should send a newline at the end as well?

Finally, would you recommend I use https://github.com/meh/elixir-socket in order to abstract away some of the low-level stuff?

edit

Please read this before my original post below:

echo in bash always appends a \n (unless told not to do), so, yes, you might need to send it as well.

If that does not work, please consider my original post below.


original post

Your dump only shows header information. I usually dump into a file using -w option. After that I look at the dumps using wireshark. You can inspect package contents then.

Also just removing packet: :line will make things about worse, since the default is packet: 4 as far as I remember, which means that erlang will parse the first 4 byte as unsigned integer and wait for that amount of bytes until it hands the message over to you.

If I were you, Iā€™d try to use packet: :raw for at least a try.

5 Likes

Ok, I think the problem is solved!

Iā€™ll leave packet: :raw, since it isnā€™t breaking anything, but it looks like everything works as expected when I appended a newline to the data that I sent.

You guys have both been helpful beyond imagination to me. Iā€™m sincerely grateful for your thoughtful engagement with my problem.

1 Like

If the server is expecting a newline then you should send a newline yep. :slight_smile:

1 Like

Do not keep :raw unless you have to! It will break your neck if you have a linebased protocol and many concurrent packages to receive!

3 Likes

ok, itā€™s back to packet: :line ā€“ your wish is my command!

Itā€™s not a wish, its just a warning :wink:

I had to go through similar behaviour as :raw has in a language that didnā€™t gave me a choice. Even worse I had to make sure I have already allocated buffers large enough to hold the data. And I had to chunk it by my self when I knew previously that my RAM (4 MiB) wonā€™t be enough to process everything at onceā€¦

But I do hope, that I do not need to do embedded networking again :wink:

4 Likes