Elixir call python by ports

Questions / Help
,
1

Is there a way to call python using ports, and get the result by receive block?
Example code is preferred, thanks in advance.

2

Well, there is ErlPort. Perhaps it does already do what you need? There is also an elixir wrapper available: export

3

Here’s a great article on ports: http://theerlangelist.com/article/outside_elixir

It uses Ruby, not Python, but the general approach should be exactly the same.

3 Likes
4

I coded a elixir port to julia, but it doesn’t work for python, I don’t know why, will dig into the ruby version. Thanks!

5

Thanks! I will find out if and how ErlPort works with respect to Python, I just don’t understand why I need use a library instead of plain elixir.

6

There is no need in using the library, you are free to do all the work again and on your own, that has already been done by the ErlPorts team.

Perhaps you can show us what you have, and what you expect it to do and describe what it does instead. Maybe we can help you to fix it?

1 Like
7

Sure, I will write a script.

8

I am not understanding something about ports, and it could be that I do not understand Python and NodeJS.

First my setup:
Mac OSX
Elixir 1.8.1
Erlang 21
Python 3.7.2, also tried 2.7.2.

In addition I have read Outside Elixir too.

I wanted to communicate with a python script via an Elixir port. For learning, I wanted to do this directly rather than using some other library, e.g. ErlPort. After some head banging I am here :slight_smile:

In Ruby, Python, and Node I have a very simple script that only prints out to stdout

Ruby

STDOUT.sync = truepyth
print(["asdf".bytesize].pack("N"))
puts("asdf")

Python

import sys

sys.stdout.write("4\n")
sys.stdout.flush()
sys.stdout.write("asdf\n")
sys.stdout.flush()

Node

process.stdout.write("4" + "\n");
process.stdout.write("asdf" + "\n");

From Elixir repl I am running

iex> Port.open({:spawn, cmd}, [:binary, {:packet, 4}, :use_stdio, :exit_status])

Where “cmd” is something like:

python <path_to_script>

and the like for Ruby and Node.

Now when I do that for the Ruby script and the run flush() I get the following:

iex> Port.open({:spawn, ruby_cmd}, [:binary, {:packet, 4}, :use_stdio, :exit_status])
#Port<0.11>
iex> flush()
{#Port<0.11>, {:data, "asdf"}}
{#Port<0.11>, {:exit_status, 0}}
:ok

Where “asdf” is the expected data.

For Python and Node I get:

iex> Port.open({:spawn, python_cmd}, [:binary, {:packet, 4}, :use_stdio, :exit_status])
#Port<0.12>
iex> flush()
{#Port<0.12>, {:exit_status, 0}}
:ok

As you can see there is no {:data, “asdf”} msg as I would expect when writing to stdout from python (similar for node).

I have tried a lot of variations on writing to stdout/stderr etc. to no effect.

9

{:packet, 4} means, that you need 4 bytes denoting the lenght of the message and no newline after that (unless its part of the message).

In elixir syntax the message has to look like this:

<<4::integer-size(4)-unit(8), "asdf">>
# or
<<0, 0, 0, 4, 97, 115, 100, 102>>
1 Like
10

I will update more later, but I seem to have found the issue. Above I had been creating the command as python <path_to_script> where the path was the absolute path.

When I changed the cmd to python ./src/py_port_test.py I get the response as i would expect.

I still do not understand why the absolute path matters for Python and Node, but not Ruby. I will look a bit more later.

11

I’m really wondering why this works, as the message format you use in in python and node is plain wrong.

To debug this, you should consider also using stderr_to_stdout and exit_status options when opening the port.

12

I will give it more looking.

13

Here’s the simplest example I can think of which works on my machine:

iex(1)> Port.open({:spawn, ~s/python -c 'print("Hello World!")'/}, [:binary])
#Port<0.5>

iex(2)> flush
{#Port<0.5>, {:data, "Hello World!\n"}}
14

Yes you are correct, earlier I had made the mistake to use struct.pack("i", 4) in python which returned byte representation in little-endian (my native), but Elixir defaults to big-endian, which I never knew :slight_smile:

Add that to the fact I do not really know Ruby and did not realize, or read closely, that [msg.bytesize].pack(“N”) encodes as an unsigned big-endian integer and I floated off into the confused.

15

Big Endianness is default on TCP I/O traffic, and since the BEAM is optimized and designed for I/O then that is why it defaults to that. :slight_smile:

16

Thanks @NobbZ, @sasajuric, @OvermindDL1, community always rooks!

1 Like