peppy

peppy

How to serialize/unserialize elixir map data for RabbitMQ?

I’ve been working on a RabbitMQ publisher and discovered that you can only publish strings to a RabbitMQ queue. The problem is I’m trying to publish a map to the RabbitMQ queue and it won’t let me.

For example:

my_map = %{message: "test", userid: 234}

I tried using Jason.encode!(my_map) to serialize the map before publishing it to the queue, which seems to work. But when I receive the message out of the queue in a later part of the app, I use Jason.decode!(my_map_from_queue), and I get this:

my_map = %{"message" => "test", "userid" => 234}

I also considered:

:erlang.term_to_binary(my_map)
:erlang.binary_to_term(my_map)

However, this thread mentioned it is unsafe to do on untrusted data: Serialize and unserialize structs, tuples, keyword lists, atoms? - #5 by sezaru

My intention is to save messages from a chat room to the database, so the message text being submitted could be anything and untrusted input coming from the chat room.

The original map being submitted is pre-formatted to a specific database schema. The goal was to gather all chat messages to RabbitMQ and use a producer to collect these messages in batches and use Repo.insert_all([list of maps]). The hope was that anything going into RabbitMQ would come out exactly the same.

What is a good solution for the serialization or “stringifying” and “destringifying” things going into and out of RabbitMQ queues?

Most Liked

SirWerto

SirWerto

I think you can use binary_to_term/2 with the safe option.

:erlang.binary_to_term(my_map, [:safe]).

this only use atoms that have been created earlier, but you have to deal with the possible exception raising.

dimitarvp

dimitarvp

What we do at my current employer is using MessagePack through the Msgpax library. Works well and keeps the door open for having a multi-language stack, if we ever need it.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hey @myefomit welcome. Unfortunately I don’t think this is good advice here. Decoding to atoms will create new atoms if the message keys change. Unless the shape of the RabbitMQ messages never changes this will eventually lead to excessive memory use and crash the application.

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
Kurisu
For example for a current url like http://localhost:4000/cosmetic/products?_utf8=✓&query=perfume&page=2, I would like to get: ...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 127089 1222
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement