krisleech

krisleech

Deserialized Avro, using AvroEx, contains incorrect data

When deserializing Avro, using AvroEx, I was getting an error, so I decided to start with a simple example to see how thing work.

However while I don’t get an error in this case I do get the wrong data back.

The data is serialized in a Ruby application as follows:

id = 1

encoder = AvroTurf.new(schemas_path: Rails.root.join("kafka_schemas"))
message = encoder.encode({ "a" => id}, schema_name: "test")
File.write("/tmp/message.avro", message)

and then deserialized as follows in Elixir:

schema = AvroEx.decode_schema!(File.read!("#{:code.priv_dir(:kafka)}/schemas/test.avsc"))
p = File.read!("/tmp/message.avro")
%{ "a" => id } = AvroEx.decode!(schema, p)

id # => -40

As you can see I get a different id back, -40 versus 1.

The schema both are reading is as follows:

{
  "type": "record",
    "name": "test",
    "doc": "Test",
    "fields": [
    {
      "name": "a",
      "type": "int"
    }
  ]
}

Is it possible this is due to the encoding of the file, versus raw bytes… I tried using File.writebin on the Ruby side and it creates the same file (same checksum).

Many thanks!

Marked As Solved

krisleech

krisleech

I tried using Avrora and it works just fine. I’m not sure but I think the issue might have been because AvroEx works with plain Avro without headers or embedded schema.

Where Next?

Popular in Questions Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New

We're in Beta

About us Mission Statement