Legion - Elixir runtime for AI agents

This actually looks pretty cool! I think one of the big challenges with projects like this is coming up with good examples. So I would love to explore this a little if you don’t mind!

Let’s say I have a typical SaaS with a user settings page that grew quite a bit over time. The settings are all over the place: light/dark/system mode switch, 2FA, 12/24 time format, very elaborate notifications settings etc. I want to add a single input bar at the top of the page that would basically go something like “Describe what you want to change and our Agent will figure it out”. And then if user describes what they want, I naturally want the agent to either:

1. Edit user settings to achieve what they want
2. Tell user it isn’t possible, suggest a workaround and create a feature request for the team (or upvote an existing one)

Would this be possible to build with Legion? How would I go about this?

Hi! Thanks!

Let’s say I have a typical SaaS with a user settings page that grew quite a bit over time. The settings are all over the place: light/dark/system mode switch, 2FA, 12/24 time format, very elaborate notification settings, etc. I want to add a single input bar at the top of the page that would basically go something like “Describe what you want to change and our Agent will figure it out”. And then if the user describes what they want, I naturally want the agent to either:

1. Edit user settings to achieve what they want

2. Tell the user it isn’t possible, suggest a workaround, and create a feature request for the team (or upvote an existing one)

Would this be possible to build with Legion? How would I go about this?

Yes, and it’ll be straightforward to do. I assume that your SaaS already has some kind of context module (or modules) that actually does the thing (toggles the setting, updates the field, etc.). To add Legion, you’d need to:

1. Add your user_id to Vault via Vault.init(current_user: user) in your router. You’re probably already doing something along these lines - it’s a single line of change most of the time.

2. Add a tiny module (or modules) on top of your context (or contexts) that would define all operations needed for your agent to see as normal Elixir functions, a.k.a. Tools. A @doc on a function might be helpful so the agent knows what exactly this function does. Inside each function call you’d authorize the caller by fetching the data from Vault - which Agent (LLM) itself doesn’t have access to and delegate the work further to the underlying real context.

3. Create an agent module (use Legion.Agent), add a @moduledoc explaining what its purpose is, and mention that if there’s no tool to perform an action, it should be explicit about that.

And that’s it. Start an agent per user when needed, use built-in Postgres storage to persist your conversations, and to prevent abuse, use the built-in rate limiter. The more flexible the tools and the better the model - the more powerful Legion is.

Does this make sense?

If you’re concerned about context size, you might consider spawning sub-agents per specific task. In legion.swmansion.com example, sub-agents are spawned to search Legion docs to be able to answer specific questions precisely, but at the same time to not bloat the main agent’s context.

And ideally - you could rebuild your application contexts (modules that do business logic) entirely, so they use Vault inside, and are the same for both Legion use AND UI use. This way you might not need to describe anything at all - Legion attaches source code of the tool (unless overriden) to its system prompt.

Thanks! I admit I haven’t seen Vault before. Doesn’t it go against the general recommendation to not use process dictionary unless absolutely necessary? Tenant id, language code and logger metadata are one thing, but putting a whole user struct there is unorthodox. An average Phoenix app expects user to be passed along in scope. Is there a way to explicitly pass a scope to a tool?

Vault is immutable, and you can’t init it if one of the parents has already initialized. This adds a guarantee that this data is only added once - but accessed multiple times. This is what makes it safe and a pretty decent choice that prevents property drilling if you may ask. You could read more about Vault and discussion around it here.

Also, if adding the whole user struct is a problem - you could add just the user_id!

This looks interesting! I like how your example gets the current user from Vault inside the tool.

I’ve been working on PII protection for Elixir agents, and I’m curious about how tool results are handled. A tool may need real customer data for an authorized lookup, while the model only needs a summary with sensitive fields removed or replaced with pseudonyms.

Is there a recommended place in Legion to apply that transformation before tool results reach the agent? Or would you handle it inside each tool?

Thanks for sharing this. I’d be interested in exploring that integration.

This looks very cool, I look forward to finding some time to explore it in more depth. It is awesome to see the continued great work people are doing in Elixir.

Hey! Thanks!

I’d suggest controlling the output of a tool and doing PII masking inside the tool call so the model never sees unmasked data.

You could build a thin layer that would intercept each tool call made by the agent and replace it with a “masked” version, but I think an easier and more straightforward way would be to just explicitly implement your tools this way.

Thank you so much! This feedback means a lot to us!!! Please let us know how it went and what you think after you’ve explored it further!

Thanks, that makes sense. I’ll explore keeping it explicit in each tool, with a shared helper for pseudonymization. The tool would return only what the model needs, while the original values stay in the application.