Cool. How is an LLM used in this situation? Aren’t we talking about serialisation?
cmo
manhvu
Yes, I’m thinking about TOON not only for saving resources, but also for integrating directly with LLMs to process a new type of API.
Concept:
Support voice commands and expose them through my current APIs (based on Phoenix Channel). The client will convert voice to text and send it to the server. On the server side, I will use an LLM to process the text like a normal API request.
I also plan to develop a new engine that processes business logic using an LLM (similar to an AI agent) instead of traditional code.
A little more about my current system: I’m building a low-latency system based on Phoenix Channel. One part of it is PhoenixGenApi, which I have published. After workload testing, I found that JSON consumes more resources than I expected (I already replaced Jason with the native Erlang JSON implementation).
0
xu-chris
I’m not sure if TOON is suitable for your case and I do not see me capable to judge this. I have worked with TOON to return smaller object based information to LLMs that saved some tokens over time and also changed a bit the operation style of the LLM.
I’ve used this: GitHub - kentaro/toon_ex: TOON (Token-Oriented Object Notation) encoder/decoder for Elixir - Optimized for LLM token efficiency · GitHub
and updated the TOON standard to the latest version in a fork: GitHub - xu-chris/toon_ex: TOON (Token-Oriented Object Notation) encoder/decoder for Elixir - Optimized for LLM token efficiency · GitHub
It can be used very similar like Jason by using @derive and feels like JSON therefore.
1
manhvu
Thanks for sharing. I checked the documentation and it looks like it won’t take much time for me to integrate this library.
I have a small question: Is there an option to decode keys into existing atoms?
0
manhvu
I tried integrating with Phoenix Channel today.
Since Toon is compatible with JSON, I just replaced the json_parser option in Phoenix and on the client.
However, I found a small issue in the toon_ex library:
data = [1, 5, 3, "a", %{}] |> Toon.encode!()
"[5]:\n - 1\n - 5\n - 3\n - a" # missing the empty object as the last item
data |> Toon.decode!()
[1, 5, 3, "a"]
This issue will cause errors when decoding with Toon.decode in server side (by Phoenix serializer).
I’ll fix this in toon_ex lib and test it in a simple app before moving to a larger app with more complex objects.
0
manhvu
This YAML reminds me of my experience with Python—one tiny whitespace issue once cost me an entire day of troubleshooting 
0
manhvu
Thank you all for your support.
Based on the suggested library from @xu-chris, I created a PoC using TOON with Phoenix Channels.
My approach:
- Updated the
toon_exlibrary by adding a missing function so Phoenix can encode terms toiodata, and fixed some bugs. - Updated the Phoenix config:
# Use Jason for JSON parsing in Phoenix
# config :phoenix, :json_library, Jason
config :phoenix, :json_library, ToonApp.MyToon
- Created a simple client in Elixir to verify it. I used the
phoenix_clientlibrary.
0
manhvu
I found two ways to using TOON (or other formats). One is update config json_library for Phoenix, in this case you cannot use Phoenix Channel with JSON Apis, LiveView. Another is add option to replace default serializer by custom module, in this case you can mix Phoenix Channel with other Channel or LiveView.
More details in this link
0
LostKobrakai
I’m not sure this is two ways: I’d call the first one a hack. If you encode to toon with the json_library, that’s just plain wrong and will affect a lot of unrelated places. Replacing the serializer is the way to go. That’s there exactly so you can replace how data is encoded in transit.
0