BinStruct - library for writing declarative code to get rich generated set of binary parsing/encoding features

Hex: bin_struct | Hex
Git: GitHub - 4ait/bin_struct
Docs: bin_struct v0.2.8 — Documentation

BinStruct is a library which main function is to convert readable as possible declarations to robust and performant implementations. It will generate set of functions to make parse → decode → create_new → send process trivial.

I have created small intro post few days ago about it: What is the Elixir way of decoding/parsing binary data? - #20 by Ridtt

I also included example implementation of png parser using BinStruct as alternative to suggested in article in this thread way using raw pattern matching.

For anyone interested in I suggest to start exploring with png example bin_struct/examples/png.exs at master · 4ait/bin_struct · GitHub

Then docs for main macros BinStruct — bin_struct v0.2.8

And then docs for types BinStruct.Types.Binary — bin_struct v0.2.8

More complex examples:

When things are hidden in integer: bin_struct/examples/extraction_from_integer.exs at master · 4ait/bin_struct · GitHub

When things are hidden in buffer: bin_struct/examples/extraction_from_buffer.exs at master · 4ait/bin_struct · GitHub

4 Likes

I would be very pleased if you could share your ideas that could be implemented using BinStruct, and I could help you with their implementation and later add them to examples. I’m too tired right now, as I’ve spent about a month preparing this library for publication. Custom types and flexible type conversions were specifically added for the community. I apologize for the messy formatting in the documentation and general disorder. I continue working on these improvements.

You can feel free to let me know anything you feel about and ask any questions.

Very interesting.

It was a bit hard to understand what the lib actually does from this post. There could be some examples use cases.

I was able to understand better by reading the docs.

Great job with the project. Very specific use case, but greatly done.

1 Like

I’m sorry again for this mess. I have a lot of examples from private products which I can’t paste. I want to grow use cases and docs from now every day. You can help me with ideas of any protocol you have worked with but don’t happy with code or you want to work this and I can add such examples. I just don’t have enough time for everything, was busy on work and implementing features/testing/writing basic docs for this lib.

Hi, good library, I’ve read the example and it looks promising! I’ve briefly peeked into the code and it looks interesting, and it has some special formatting, I love that. I also found that some modules are empty and exist only for sake of documentation. I’d suggest to just write separate .md doc files and expose them in hexdocs (absinthe documentation is a good example of how to do this).

I’ll definitely give it a deeper look in some near future!

1 Like

Hello there, thank you. I just have added more complex examples as I promised in intro post.

When things are hidden in integer: bin_struct/examples/extraction_from_integer.exs at master · 4ait/bin_struct · GitHub

When things are hidden in buffer: bin_struct/examples/extraction_from_buffer.exs at master · 4ait/bin_struct · GitHub

I like current way of how automatic doc tests are works. Can I achieve same with separate .md’s?

What is actually special formatting?

Some reasoning:

My main goal is to keep things as they are. Simple things will be simple anyway. Hard things will be hard anyway. I don’t see reason in any shortcuts for example for situation where some length is may be directly inferred from other field like most libraries I saw do. Every attempt of introducing multi-field writer or any other complex behavior has failed.

I ended up with idea of dynamic behavior is always a callback, there is a single way of creating virtual field during parse/decode using read_by and single way of creating anything in new context automatically using builder. I got so far with idea not mixing things and I liked it. Once you have finished writing your declaration and tested it you anyway can forget about what is inside. You have guarantees if it is working alone it will be working same inside any tree. Good way to reuse parts is to compose it into higher order macro, which again don’t care of presence of shortcuts and so on.