Maxximiliann

Maxximiliann

Passing JSON from Elixir to Python with Ports

JSON is serialized into Erlang unicode and then passed to a Python process:

defmodule Sandbox do
  def foo(json_data) do
    port =
      :erlang.open_port(
        {:spawn, 'python3 -u lib/Python/Sandbox.py'},
        [:binary, packet: 4]
      )

    execute = fn data ->
      send(port, {self(), {:command, :erlang.term_to_binary({data})}})

      receive do
        {_, {:data, response}} -> :erlang.binary_to_term(response)
      after
        60000 ->
          {:error, :timeout}
      end
    end

    {_, results_string} = execute.(json_data)

    Print.text("sending JSON data, python result: #{results_string}")
  end
end

Example of what the Python process receives:

json_data = "<<131, 109, 0, 0, 0, 212, 123, 34, 97, 109, 111, 117, 110, 116, 95, 97, 115, 115, 101, 116, 34, 58, 34, 87, 65, 86, 69, 83, 34, 44, 34, 110, 111, 114, 109, 97, 108, 105, 122, 101, 100, 95, 97, 109, 111, 117, 110, 116, 34, 58, 52, 54, 54, 57, 55, 49, 51, 55, 56, 44, 34, 111, 114, 100, 101, 114, 95, 108, 105, 102, 101, 116, 105, 109, 101, 34, 58, 54, 49, 44, 34, 112, 114, 105, 99, 101, 34, 58, 48, 46, 48, 56, 56, 54, 51, 48, 48, 55, 44, 34, 112, 114, 105, 99, 101, 95, 97, 115, 115, 101, 116, 34, 58, 34, 72, 90, 107, 49, 109, 98, 102, 117, 74, 112, 109, 120, 85, 49, 70, 115, 52, 65, 88, 53, 77, 87, 76, 86, 89, 116, 99, 116, 115, 78, 99, 103, 54, 101, 50, 67, 54, 86, 75, 113, 75, 56, 122, 107, 34, 44, 34, 117, 105, 100, 34, 58, 34, 50, 48, 50, 48, 45, 49, 49, 45, 50, 51, 32, 50, 48, 58, 48, 51, 58, 53, 49, 46, 48, 48, 53, 50, 49, 53, 90, 116, 55, 71, 79, 79, 54, 49, 71, 89, 117, 112, 119, 78, 117, 122, 115, 109, 102, 52, 119, 83, 119, 34, 125>>"

To deserialize this into a Python dictionary the “<<” and “>>” characters need to be stripped off, all of the numbers need to be converted from strs to ints, and then the new list of ints needs to be iterated over to decode it into UTF-8 characters.

Is there a more efficient way to pass JSON from Elixir to Python such that it’s readily parsed by a Python process?

Most Liked

al2o3cr

al2o3cr

As far as the BEAM’s concerned, a port is just a process where you can send bytes with {:command, io_list} and get bytes back by receiving {:data, io_list}. If it’s easier for your program, you can just send those JSON bytes instead of encoding term -> binary.

The term conversion process is handy if you’re sending complicated structures that aren’t already encoded. Take a look at erlport, which already handles decoding ETF and has a Protocol abstraction for dispatching to multiple commands through a single port.

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
nobody
Hi! In PHP: $SERVER['SERVERADDR'] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I'm a nov...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

Other popular topics Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I'm a nov...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New

We're in Beta

About us Mission Statement