Maxximiliann

Maxximiliann

Passing JSON from Elixir to Python with Ports

JSON is serialized into Erlang unicode and then passed to a Python process:

defmodule Sandbox do
  def foo(json_data) do
    port =
      :erlang.open_port(
        {:spawn, 'python3 -u lib/Python/Sandbox.py'},
        [:binary, packet: 4]
      )

    execute = fn data ->
      send(port, {self(), {:command, :erlang.term_to_binary({data})}})

      receive do
        {_, {:data, response}} -> :erlang.binary_to_term(response)
      after
        60000 ->
          {:error, :timeout}
      end
    end

    {_, results_string} = execute.(json_data)

    Print.text("sending JSON data, python result: #{results_string}")
  end
end

Example of what the Python process receives:

json_data = "<<131, 109, 0, 0, 0, 212, 123, 34, 97, 109, 111, 117, 110, 116, 95, 97, 115, 115, 101, 116, 34, 58, 34, 87, 65, 86, 69, 83, 34, 44, 34, 110, 111, 114, 109, 97, 108, 105, 122, 101, 100, 95, 97, 109, 111, 117, 110, 116, 34, 58, 52, 54, 54, 57, 55, 49, 51, 55, 56, 44, 34, 111, 114, 100, 101, 114, 95, 108, 105, 102, 101, 116, 105, 109, 101, 34, 58, 54, 49, 44, 34, 112, 114, 105, 99, 101, 34, 58, 48, 46, 48, 56, 56, 54, 51, 48, 48, 55, 44, 34, 112, 114, 105, 99, 101, 95, 97, 115, 115, 101, 116, 34, 58, 34, 72, 90, 107, 49, 109, 98, 102, 117, 74, 112, 109, 120, 85, 49, 70, 115, 52, 65, 88, 53, 77, 87, 76, 86, 89, 116, 99, 116, 115, 78, 99, 103, 54, 101, 50, 67, 54, 86, 75, 113, 75, 56, 122, 107, 34, 44, 34, 117, 105, 100, 34, 58, 34, 50, 48, 50, 48, 45, 49, 49, 45, 50, 51, 32, 50, 48, 58, 48, 51, 58, 53, 49, 46, 48, 48, 53, 50, 49, 53, 90, 116, 55, 71, 79, 79, 54, 49, 71, 89, 117, 112, 119, 78, 117, 122, 115, 109, 102, 52, 119, 83, 119, 34, 125>>"

To deserialize this into a Python dictionary the “<<” and “>>” characters need to be stripped off, all of the numbers need to be converted from strs to ints, and then the new list of ints needs to be iterated over to decode it into UTF-8 characters.

Is there a more efficient way to pass JSON from Elixir to Python such that it’s readily parsed by a Python process?

Most Liked

al2o3cr

al2o3cr

As far as the BEAM’s concerned, a port is just a process where you can send bytes with {:command, io_list} and get bytes back by receiving {:data, io_list}. If it’s easier for your program, you can just send those JSON bytes instead of encoding term → binary.

The term conversion process is handy if you’re sending complicated structures that aren’t already encoded. Take a look at erlport, which already handles decoding ETF and has a Protocol abstraction for dispatching to multiple commands through a single port.

Where Next?

Popular in Questions Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 44139 214
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New

We're in Beta

About us Mission Statement