ahamez

ahamez

Protox - A 100% conformant protobuf library

Hi everyone,

I’ve been working on this protobuf library for 3 years. We use it in the company I work for, EasyMile, to communicate with our autonomous vehicles.

As you can imagine, reliability is very important! Therefore, a lot of efforts have been put into making sure that the library is reliable. To this end, I committed to make it pass all the 1302 conformance tests that Google provides for binary serialization. This is on top of having a 100% code coverage, plus property based testing and dialyzer.

In order to assess the performance of protox, I benchmarked it with random and handmade inputs against the other available libraries, exprotobuf and protobuf-elixir. You can find these benchmarks here (zip archive of a benchee HTML export). They show that protox is always faster than exprotobuf and most of the time on par or faster than protobuf-elixir. Of course, the result might be different with another computer/OS and with different inputs, I’ll be happy to include new ones.

Also, I think that protox is easier to use as it let you work directly with Elixir structures (for instance, there’s no need to use a new function to create a protobuf message, which means that you’ll have a nice error if you use a wrong field name). Furthermore, there’s no need to generate any file.
But of course, I may be a little biased here :wink:.
[Edit] It seems that file generation is important for some users, so I guess I should find a way to make it possible after all!

You can find it here:

https://github.com/ahamez/protox

Let me know what you think!

Most Liked

ahamez

ahamez

:speaking_head: protox 1.0.0 has been released! As no new features have been added and the API has been stable for a long time, I thought it was finally the time to mark it stable.

So, as a recall: why would you want to use protox rather than the others libraries like protobuf-elixir which has more features (and users :wink:)? Well, It really depends on how much do you need a reliable implementation. As protox has been written for the case of autonomous vehicles, reliability was more important than features. Thus, it has been has been thoroughly tested using the conformance checker provided by Google, has a 100% test coverage and uses property based testing.

Don’t hesitate to give it a try :slightly_smiling_face:.

12
Post #8
ahamez

ahamez

:speaking_head: protox 1.2.1 is now available!
The main changes since 1.0.0 are:

And of course, it’s still 100% compliant with the Google conformance checker :wink: .

ahamez

ahamez

:megaphone: protox 2.0 is out!

What it brings:

  • encoding speed boost of around 50%
  • significant internal restructuring for easier maintenance
  • consistent way of accessing a message’s schema

Being a major release, there are breaking changes:

  • drop support of JSON encoding/decoding (the implementation was half-baked and even Google doesn’t recommend using the JSON encoding). I hope people won’t be disappointed by this, but looking at the projects that use protox, none of them are relying on this functionality.
  • encoding functions now return the size of the encoded data so you don’t need to compute it yourself (which, by the way, provided the encoding speedup)
  • many generated functions to get a message schema information (like the syntax) are removed and the corresponding information are available in the structure returned by schema/0.

:backhand_index_pointing_right: The complete changelog.

Where Next?

Popular in Announcing Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 43487 311
New
ityonemo
Currently just starting out on a new mini-project - getting zig NIFs to run in elixir. https://github.com/ityonemo/zigler The idea here...
New
josevalim
Hi everyone, We would like to announce that Plataformatec is working on a new MySQL driver called MyXQL. Our goal is to eventually integ...
New
josevalim
Yes, yet another parser combinator library! Most of the parser combinators in the ecosystem are either compile-time, often using AST tra...
159 19417 141
New
dominicletz
Hi, I thought I had posted my library before but seems I hadn’t. The project is still in early stages but it’s growing and so I think it...
New
sbs
Only 650 LOC, wrote for fun :slight_smile: https://github.com/sunboshan/qrcode
New
treble37
Just looking for a little feedback on a tiny helper library I built - Sometimes I find the need to convert maps with atom keys to maps w...
New
Qqwy
TypeCheck: Fast and flexible runtime type-checking for your Elixir projects. Core ideas Type- and function specifications are const...
336 14482 100
New
handnot2
Samly can be used to enable SAML 2.0 Single Sign On in a Plug/Phoenix application. This library uses Erlang esaml to provide plug enabl...
New
kevinlang
Hey all, We have made an Ecto3 Adapter for SQLite3, ecto_sqlite3! We have successfully on-boarded the full suite of integration tests (...
New

Other popular topics Top

axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48342 226
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52673 488
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New

We're in Beta

About us Mission Statement