Is anyone working on "AI Agents" in Elixir?

Hey, so I know that there are these libraries which I didn’t try yet: jido swarm_ex

And I have these blog posts open in a tab but couldn’t find the time to read them so far:

I’m sure there is more going on

https://github.com/brainlid/langchain

I have been throwing together a list of Elixir based openAI-style clients as well with some of the forementioned libraries (in addition to some others): GitHub - druyang/awesome-elixir-llm-genai: A list of LLM and GenAI Elixir Resources/Tools · GitHub

Like you said, technically all you need for an agentic library is the ability to function call (which is really just structured outputs) and feed the feedback back into the model. Many of the libraries work, the amount of extra lifting to create the loops varies. I think the aforementioned Jido seems the most promising (not to pick favorites).

Since most agentic frameworks are bottlenecked by IO requests rather than pure computation, I also believe that BEAM/Elixir is an extremely underrated choice for GenAI. This is exacerbated by the use of batching LLM calls at scale for further cost reduction.

I’m still new to Elixir, but in my free time I want to work on some OSS projects in this area

I gave them a quick scan and they seem to be making roughly the same point I was, which is encouraging to me

It seems like “agents” in general are a broad enough concept as to be effectively turing complete - i.e. there is no wake to make an “agent framework”, because agents are just code. And not only that, but at least for the time being running large models on your own servers is not cost-effective unless you have massive scale, and even then you would want as much multitenancy as possible within your own systems, so that forcing function doesn’t go away. In practice I think even large orgs will have to disaggregate their “ai compute” from their application compute, like we do with databases and storage, for the foreseeable future.

It seems like the one bit of tooling we still need is, well, tooling for tool calls. The impetus for this post was that there was another post on here about a ruby library called rubyllm which I thought looked like a fantastic model for how we could implement that functionality.

I am curious, though: is anyone on here working on an application (an agent) in this space, as opposed to libraries and frameworks? I am curious to hear people’s experiences with this.

So, I really liked the blog post, especially that it boils down to this

I hope these examples showcase how we can build sophisticated LLM workflows using simple Elixir mechanisms. While our implementation might seem basic - just pattern matching and with statements - don’t be fooled by the simplicity. This approach of composing small, focused functions and passing message chains through them can take us surprisingly far.

The beauty of this approach lies in its transparency - we can see exactly how our LLM interactions flow, what context is being maintained, and where we might want to add error handling or logging. No magic, no hidden state, just clear functional transformations.

Libraries can still be useful when they add convenience and structure.

I also like that this article follows the structure of Anthropic’s article as it’s a very good reminder that you can probably solve most problems without “agents”, even if you make use of LLMs.

Yes, I agree. We have database and storage as building blocks, so I could imagine that in the same way many applications have a “reasoning” block that makes use of LLMs for certain innovative features. Depending on how performant and cheap they will be, I could also imagine that we can use LLMs to avoid implementing some features in code and use a call to an LLM as shortcut.
Recently, I’ve been thinking of software defined networking as analogy (no expert here, so could be totally off ). Before that you’d do all networking in hardware. As software performance increased and it got cheaper at the same time it suddenly made sense to do networking in software as you could work way more flexible.
In the same way, I could see LLMs enabling “flexible” code. I also like this analogy because there is still a lot of hardware involved in networking, it’s just that you can do innovative things with software defined networking.

Not an application in that sense but I want to try to build an LLM based program that converts models from transformers written in pytorch to Elixir and bumblebee.
Partially as a learning project for building such projects, and I’m just curious how far I can get.
I also want to take it step by step following Anthropic’s categories and only move to agents if I really have to.
As a sidenote, here are people from huggingface saying that it’s more effective to give LLMs the ability to write and execute code to do something instead of JSON tools.
I also think that when your problem allows it it’s in general more effective to let LLMs write code instead of asking the LLM to perform the operation directly.
In my case I guess there is a lot of Python/pytorch code that follows a structure that can simply be parsed and transformed to Elixir, and I don’t really need an LLM to perform that operation but rather code that parses and transforms. On the other hand, for arbitrary functions or control flow I might need an LLM and potentially a feedback loop.

I guess for real agents the core issue is that you give up control to LLMs, which might limit their use cases. Or, you’d need other ways to exercise control over the results and actions that will be performed.

Are you thinking about building any sort of agentic application?

The reason databases and storage are cheaper disaggregated is that multitenancy has inherent efficiency gains. A common example is S3, where high-storage low-bandwidth customers and high-bandwidth low-storage customers can share the same physical drives and use the capacity more effectively. LLMs currently exhibit a dramatic cost reduction in a multitenant environment because they are autoregressive.

If trends were to reverse this could just as easily end up not being the case. For example, if diffusion LLMs were to catch on and on-CPU accelerators and larger memory bandwidth catch on (e.g. amd strix halo) then you could imagine it being simpler to just run your zero-shot “AI” tasks directly on CPU (for us, this would mean Nx/Axon/Bumblebee). Models have also been shrinking, which helps.

But thus far, autoregressive models have kept winning. I have no deep technical understanding of why - maybe nobody does?

Nobody has solved prompt injection yet so giving your models the ability to run arbitrary code seems unwise. My bet would be for “real” products this will remain a very bad idea for a while.

I am not. I didn’t see any value in the paradigm at all until these “deep research” tools came out, but it’s the first use case that actually makes sense to me and I thought it was interesting that Elixir/BEAM mesh really well with that type of product.

Has anyone look at the Google A2A agent protocol (https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/) ? Initially I am thinking Elixir genserver would be a good fit to model agent to agent communication , but with these standardize protocols being pushed by big companies, what are some of the advantages of using Elixir instead of Python in this domain?

I find myself agreeing with this post by antirez (redis guy) about MCP and other “AI protocols”.

Coming up with “structured” protocols for LLMs is quite possibly the least interesting thing you can do with a paradigm which is revolutionary solely because it can interact with unstructured data. We have, for the first time in the history of our field, finally found a way to interact with computers in the way that “regular people” expect and of course the first thing programmers do is try to find a bunch of ways to get rid of the uncertainty and reassert structure.

Of course, our ability to reason about computing in a structured way is why we are programmers and everyone else is not, so this is not surprising. But I don’t think these things are going to last - they are a product of hype IMO.

Of course I cannot write this comment without linking the XKCD.

Protocols aside, the big advantage for Elixir here is that our ecosystem was practically made for this. There is no better platform for writing soft-realtime glue code between different models/services/APIs.

If you take a step back and think about this it makes perfect sense: Erlang/OTP were literally designed for telecommunications. This problem, facilitating communication between models/services/users, is telcom, it’s just that the scope has expanded far beyond phone calls. It is a testament to the wisdom and creativity of those who built these systems that they can still be so relevant today.

“Agentic” apps built with Elixir will scale better with much lower latency (especially tail latency) than anything built with Python, and (IMO) developer ergonomics are much better too, though Python is far from the worst.

If anything, our biggest “competitor” will probably be JS simply because a couple of large companies (e.g. Cloudflare) have committed to in-process multitenancy which will drive costs down significantly for those who aren’t serving enough to saturate a VM core. On the other hand, one might argue those customers aren’t very valuable.

Author of Jido here

I’m actively wrestling with these ideas. I’ve implemented several “Applications” with Jido now that … after finishing them … I really struggle to answer whether they are better with Jido or not. Long term - a few GenServers that wrap req calls to LLM API’s are better.

I have this overwhelming feeling that Jido is the right direction - but has not arrived at a sensible destination.

A few other thoughts to share:

• There’s a complexity vector here - a simple LLM wrapper doesn’t need a sophisticated agent framework - Oban works great.
• Most agent implementations are really really simplistic - any dreams of massive swarms of agents demonstrating collective intelligence are still dreams - Elixir is more suited to larger swarms of agents due to OTP
• While implementing Jido, I learned a lot about OTP - the educational journey was amazing. I found Joe Armstrong’s blog while on this journey - and realized that some of the features of Jido are simply constrained implementations of OTP - I don’t think that’s bad necessarily - but probably not the best implementation
• While it’s easy to write an run 10,000 agents with Jido - it’s not that useful - for all of the normal distributed systems problems that come from running and coordinating 10,000 GenServers

There’s more questions then answers right now - but I do think LLM’s are here to stay so it’s better to wrestle with them

This particular space in our industry is evolving a lot right now - so I’m content to just continue wrestling and playing with the ideas. A few “first principles” I’ve collected so far:

• Agents will be a new buzzword for “LLM Applications”
• Agentic workflows are just workflows - low volume ETL pipelines with more variety
• Multi-agent is simply a new variant on distributed systems problems
• Elixir is well-suited for this - but suffers from a slower iteration cycle - so I’ve been following Python and TS “agent” frameworks closely and pulling in patterns to Jido that I feel will be durable over time