Is there anybody out there with good skills in metaprogramming? I definitely have to read Chris McCord’s book at some point because I feel like I’m doing too much of what I don’t understand lately.
I am one of the maintainers of YMLR - A Yaml Encoder for Elixir and have recently refactored the library to work with a protocol for the encoder. I took the inspiration from Jason but they have implemented their __deriving__ macro in the Jason.Encoder module in a much more performant way. If anybody out there feels up for the job and has some spare time, I’d really appreciate a pull request for the improvement of the __deriving__ macro in Ymlr.Encoder
I’ve taken a look and while I don’t have the time to dedicate to a PR I can offer the following observations (which may already be totally obvious to you).
Since the :only and :except lists need to be known at compile time you can process them in the macro, outside the quote block. That will certainly give you some performance improvement and make it easier to maintain as well.
Most encoding tasks that result in a binary (string) gain material performance benefits in encoding first to iodata and then calling :erlang.iodata_to_binary/1 at the very end. That function uses very efficient kernel system calls to convert io vectors to binaries. File.write/2 and friends support iodata as a parameter do you don’t even have to do the last call yourself and in effect the File.write/2 call will stream the iodata to disc (not exactly how it works, but close enough).
Those are the two primary differences I see with the Jason implementation being very idiomatic macros techniques.
Thanks a lot for taking the time to look at this and answer.
Since the :only and :except lists need to be known at compile time you can process them in the macro, outside the quote block. That will certainly give you some performance improvement and make it easier to maintain as well.
I am actually aware of that. It is the processing outside the quote block that I struggle with. I find it hard to learn and debug let alone to write a function that produces some kind of AST. Hence reading the book I guess.
Most encoding tasks that result in a binary (string) gain material performance benefits in encoding first to iodata and then calling :erlang.iodata_to_binary/1 at the very end.
ymlr actually already does that. If you look at the functions in Ymlr.Encode you see that all of them return iodata which is turned into a string in to_s!/2. The only optimisation I have been thinking about: Since most of the time the data is written to a File or other IO after encoding, it would be nice to provide an API on the YMLR module which returns iodata to the caller. But that’s a bit off topic here. And also something I could just implement.
Since that narrows the scope I’ll take a first pass at a PR for point (1) tomorrow my time. Don’t expect a finished product but at least enough for you to move forward. Hopefully gets you on your way at least.
