How the Elixir community can survive — and thrive — in an age of LLMs.
Having spoken to Zach at ElixirConf EU and heard some of the challenges he has seen with supporting users that have completely fictional code. Coupled with his general skepticism of the AI hype. I am glad he wrote something up.
This is a pragmatic take on what we can usefully do around adoption, and being on a slimmer end of the bell curve when people start using bell-curve-machines for information and code generation.
Without trying to go to the moon or saying the cost of programming will go to zero. That stuff is speculative. This is mostly addressing current reality, whether you like the situation or not. I don’t love dealing with LLMs but I don’t particularly want to close my eyes because I don’t think it will entirely blow past.
11 Likes
I’ve only just started using AI and I have yet to use an agent. I’ve also never really chat as there is something something I find so off-putting out it. This article has some nice tips and some good insights that I hadn’t thought of. I particularly like the tip of asking it to first find the docs. This is something many people clearly take for granted.
Ultimately you should not be relying on an LLM as a source of “knowledge”.
I’ve seen several questions here, on reddit, and watching people IRL where they simply refuse to use anything other than a chatbot to figure things out. When they give up they don’t look to the docs, they ask someone. I know I’m not the only one wondering this but I haven’t seen too many satisfactory answers: what happens when people stop writing documentation? Are we going to start seeing libraries where the authors haven’t looked at a line of code? If people are producing software like this, how do the LLMs learn to use it? They’d have to start analyzing the code, of course, but how is that not going to degrade the slop even further?
Not trying to be a doomer just wondering, and wondering is all we can do.
Best case AFAIC is that we get flooded with so many quick-to-market apps that carefully crafted software with performance and security as first class citizens becomes cool again. And by “cool” I mean every day users will feel a noticeable difference.
7 Likes
I’ll tell you what my real experience is, using LLMs has allowed me to write more well tested code, as well as more well documented code. For example, we’ve been having discussions about some issues w/ Ash’s documentation.
- the search results didn’t seem ordered by the most helpful stuff
- there is so much content that “cross referencing” is important, i.e putting links to things you were probably looking for if you got here and its not what you want.
I spent about 2 hours w/ Claude working on our four most commonly used modules, Ash, Ash.Changeset, Ash.Query and Ash.ActionInput. We have a solid usage rules file and other things that helped guide its output. I tasked it with expanding the docs on the most common functions (and many of the less common ones) ensuring that we have example usage, cross links to other relevant guides, and typespecs. It wasn’t “vibe coding”, i.e I reviewed all of the work, made many tweaks of my own, etc. But it turned a multi-day project into a 2-3 hour project. And when we have other docs issues in the future I will likely feed it into Claude and have it generate some nicely formatted output. i.e “this user had trouble with X, please look at the current docs for Y and produce something that would have prevented this confusion”. It also figured out why the search results were suboptimal before I could figure it out myself. The order of modules in the sidebar controls to some degree the order of modules in the search. We reordered the sidebar, and search got that much better 
If you’re familiar with the issues around Ash docs, the improvement to this module is effectively night and day: Ash — ash v3.5.13
To me, it’s not a replacement for my brain, it’s just a way for me to exert more leverage than I would have before. When you know how to use it, it can make you faster, help you automate the boring stuff so you can focus on the important stuff, and get you past the blank-page problem.
16 Likes
It’s nice to hear that putting documentation/instructions in context helps the models write Elixir (and Ash) code at a level closer to popular languages. I am still somewhat surprised/skeptical about that result because I would expect the amount of, say, Python content to dwarf Elixir to the degree that Python code would still come out much better. The fact that these context tricks work so well indicates to me that these models are generalizing very well across languages (a point made in the article).
Obviously as an Elixir Enjoyer I do hope it works out this way in the coming years.
Maybe others have had a different experience with the “modern web” and “modern apps”, but personally I am reminded of a point I saw about AI slop on social media not having made much of a dent because so much of social media was already human slop that it was hardly noticeable.
Similarly, so many webapps are b2b-enterprise-saas-slop that I’m not sure if anything is really going to change. But there is one thing to keep in mind: if most webapps are slop, then most of the training set is also slop, and the outputs of the models will remain slop. So spending time working on your skills and building better software is not a waste, as many on the hype side are claiming, since it should be quite viable in the short-medium term to “outrun” the models. At least I hope so, anyway.
4 Likes
Absolutely, and I’m all for its usage as you describe. My comment was missing some context—I joined reddit not too long ago so I’d stop posting low quality rants here (it’s more or less worked 
). There’s a faction of people on reddit who basically call anyone claiming you still need to understand how to read and write code a gatekeeper. I’m wondering if that’s what Anthropic et al is really pushing for… sorta like how I don’t know how to write raw machine code but still call myself a programmer.
Re: Ash docs, I was familiar with it having problems but could not put my finger on it. Taking a quick look again already the sidebar so much less scary-looking! Topics look more consolidated too, for example, I don’t have to look at three different pages for different usages of the same concept. I haven’t worked on my Ash project in a week or so so extra looking forward to getting back to it now.
2 Likes
Yes, I didn’t want to dive into this because I didn’t want to write a novel-length post. But I mean generally it would be nice if it somehow triggered a trend of these things becoming important all around. Like “first/quick to market” could well become severely cheapened if people are inundated by even more garbage than they already are. Maybe we’ll start to see “Zero vibes” in marketing materials. A boy can dream 
3 Likes
There were some arguing that this was already happening even before the slop-coding boom, e.g. this Linear blog post. It does seem like people are starting to get fed up with this stuff, and I hope they continue to vote with their wallets
Unfortunately the first thing you will discover (as I have) if you even try to build better apps is that nearly all of the standard tooling and best practices are woefully inadequate and need to be thrown out, which is what I have been slowly doing. And again, this flies in the face of the “2 week MVP” strategy because even learning how to build apps right is a multi-year process for which there are scant few resources to help you along. It really does hurt.
Incidentally I do see Ash as an approach intended to solve some of those problems, though I’m not entirely sure if they’re the same problems that I have. But more to the point: AI models as they exist are not capable of the level of creativity to create such a solution, so there is still hope yet. Antirez wrote an amusingly specific post about this the other day, which I thought drove the point home well.
3 Likes
Literally the best ever summary I’ve read on how an LLM empowers already powerful programmers. I had 100% the same experience; LLMs serve me as (1) much better search engines and (2) make long and tedious work that I know how to do from the beginning, that much faster. Makes me more creative. I don’t focus on minutiae as much as before.
This last part comes with a huge red warning though; every now and then do return to the minutiae and details, lest you forget your attention to details! But even that is made much better by the fact that I can in fact be creative and detailed about things I care about, and have everything else be accelerated a little bit (or a lot).
2 Likes
Gemini being described as a “smart duck” to chat with is both super funny and super true.
1 Like
Great discussion! I wrote about this topic two years ago: On using LLMs like ChatGPT to get things done • OVERBRING Labs
I find that it’s a major windfall, iff you use LLMs correctly. And by that I mean: knowing enough of the topic at hand to spot inaccuracies, knowing how to have a discussion to explore a topic without getting caught up in “blind alleys” (where the LLM insists on e.g. giving you a solution about something you are only tangentially interested in), etc.
In the end, LLMs are a tool, though not “like any other”, since they can do things that would be considered sci-fi a few years ago. Still, knowing when and how to use a tool, matters.
1 Like
I’m early in my journey, both with actually attempting to write robust Elixir/OTP, and with LLM’s, but here are a few tips that have helped already:
- Hook up context7 MCP in Cursor. Then, in every new agent chat, instruct the agent to use context7 to read the latest Elixir docs.
- Iterate on your docs. Focus on standards. Use dialyzer heavily. Keep warnings in check. Develop a robust strategy for OTP. Use the LLM’s themselves to develop these standards for your document set, leaning on your knowledge of BEAM/OTP principles.
- Be willing to throw away an attempted implementation, using it for lessons learned in feedback loops with Claude/Gemini/Grok/etc.
I’m also focused on using this experience to enhance and accelerate learning Elixir/OTP development, rather than letting the LLM think without me.
LLM’s for coding are not going anywhere. It’s probably wise to start getting acclimated to using the latest AI tools, if you haven’t started already.
Bullish long term on Elixir for AI, both for development and as a platform. It’s still early.
3 Likes
Yes, you would think so. However, I have had chatgpt o3 and gemini 2.5 produce absolute garbage Django Rest Framework code, even though this is a stack that is/was pretty popular.
1 Like
I could be wrong, but i think LLM’s in the future will allow people to enjoy more esoteric languages, having to switch to a popular language for a library or a framework is such a downer. Future LLM’s will generate something close to the library in your programming language of choice, in which quality wise will be no different than an average engineer writing a library and sharing it on github
Right now I just use LLM’s for css stuff because I have 0 passion or patience to get good at FE design, claude spits out good enough boilerplate FE that i can tweak or ask it to tweak. I think elixir will thrive with LLM’s because of the saying “elixir is easy to learn and hard to master”. Imagine an LLM spitting idiomatic and correct elixir code with minimum effort and because the elixir syntax is simple to read, the programmer will have no issue following it
1 Like
Sadly the LLMs, even the excellent ones like Gemini Pro 2.5, still fumble with Elixir quite a lot. They think it’s a good idea to make a 20-clause with chain where each statement is an if/else block. Or cond. Hilarious to look at, and sad to check the compiler errors.
Generally you are right though, I just don’t think we are quite there yet as you also alluded to.
1 Like
Yeah it still has a long way to go, I imagine they work better for strong statically typed PL, than anything in dynamic lala land. We still have to remember that AI improves exponentially and with capitalist incentive being attached aka fortune 500 companies would rather pay 100s of million to an LLM that is good enough, than pay billions to retain workers, pay for healthcare etc etc all this means its inevitable
I think by the end of the decade it will be irresponsible not to use it to code
3 Likes
They are quite useful as rubber duckies though, and for learning. Or for thinking through a problem, such as software architecture, or refactoring.
Concrete example: Breek.gr is built with PHP (backend) and NextJS (frontend). My skill in both parts of the stack was below-par (or rusty, for PHP) last September. After I picked up the frontend and backend codebases in October, I used ChatGPT to gradually explain general concepts of NextJS based on every TSX file I was touching, and so I gradually upskilled to “mid to OK”. Way easier than wading through the NextJS docs or using Google Search. I now use Grok a lot for new feature development in NextJS, but I don’t let it code from scratch.
After scaffolding the TSX, I gradually prompt Grok to add this or change that. Way more often than not, it works. It’s hopeless to give it the full target of what a complex page should do and expect it to spew out code that you can simply copy and paste into the IDE, because in a complex app there are too many interactions with other custom hooks etc. for any LLM to take into account. It’s great for refactoring some TSX files originally written by a junior dev in JSX and then “minimally converted” to TSX by using “any” etc.; so, I also use Grok to save time when defining interfaces.
Grok is passable to good in Elixir nowadays. However, I enjoy coding in Elixir, so I don’t use it much for that purpose. After all, half the fun of developing apps in Elixir is the “in Elixir” part! Or, perhaps I’m in this phase of my Elixir journey (and growing frustration with the complexity of NextJS) where every Elixir module I write feels like a breath of fresh air and an accomplishment.
With LLMs, my 2023 hunch remains valid: you need to understand enough about the subject matter to know when you’re being given BS. And you need to know enough about the codebase to provide guidance. Most of the context is in the user’s head, the rest is provided by the user to the chatbot. You need to do a lot of hand-holding, but in the process you also understand better, and sometimes redefine the problem; sometimes, it’s an “XY problem”.
My experience with LLMs so far has been very positive. It’s a boon both to development velocity (isn’t that what the Agilists call it?) and to my understanding of two pieces of the stack that I have no intention of building expertise in. With human QA (i.e., me) we end up with robust and performant code on both backend and frontend. For $30 per month, SuperGrok is a bargain.
1 Like
Given how affordable monthly subscriptions are, I consider it irresponsible to not use LLMs already now. The value of work is not in the pain endured; and some parts of coding (especially in TypeScript, such as defining interfaces) are necessary pain that is best offloaded to an LLM. I personally don’t gain anything from defining yet another part of the code that is for “plumbing purposes” inherent to the language…
Time saved this way is invested in developing features and (even better) coding in and writing books about a language and ecosystem that I really enjoy
1 Like
I wonder about the possibility of taking a good-enough self-hostable LLM and fine-tuning it by providing it with the entire documentation of Elixir, Phoenix, Ecto and other Elixir packages often used together.
Has anyone done this and can provide some insights?
BTW, it may sound funny, but I sometimes pre-prompt it with “you are Jose Valim” 
1 Like
Apologies, when you say grok you don’t mean twitters Ai model? I assumed grok was not a serious competitor
