This is the claim I was referring to, and I’m not disputing its truth. What I’m suggesting is that there may be some dangerous second-order effects to look out for, particularly for junior programmers who do not yet have the skill set to design things themselves.
Obviously this is a spectrum, and at the far end of one side you have the most basic imaginable boilerplate. The stuff you get from mix new. Nobody needs to write that; we don’t even need LLMs to write that (we use mix new). But even then, there is probably some value to doing it at least once, just so you know where everything is.
As you move further down the spectrum, you might get something like mix phx.new. I still think these generators have some value, but even there it starts to get iffy and I know there are some who don’t like them. I don’t really like them either, to be honest. I think if I was designing a new framework (which, uh, maybe I am) I would probably not go generator-heavy. But it’s a personal choice.
However, on the far end of the other side of the spectrum you have green field designs. Stuff that you don’t actually know how to write because you’ve never done it before. Like, in my case, an LSM tree storage engine.
I cannot mix new an LSM tree. I have to figure out where the compaction goes, and where the k-way merge goes, and how the multiversion manifest is structured, and how they all fit together. This is hard because unfortunately there is no LSM instruction manual (believe me, I looked).
It’s this sort of scaffolding that I was referring to, not basic boilerplate. You have to figure out how to build these things one piece at a time. Like, you want to write the compaction, but first you need something to compact. So you write the memtable. Then you need a place to compact to, so you write the block store. Now you can write the compaction. Now you have tables, so you need the manifest, so you write that. Now you can compact tables into tables! But you need k-way merge for that. And so on.
If you’re an experienced programmer you probably think nothing of this, but the juniors (and realistically probably some “seniors”) don’t know how to do this. They think it’s magic. That’s what mitchellh’s article is about.
LLMs are much smarter than mix new, and they can actually produce somewhat plausible output if you tell them “write me an LSM tree in Elixir”. However, in doing so you will rob yourself of learning how to architect such a thing yourself. The model will make all of the most important architectural decisions for you, and it probably has no idea what it’s doing because there are not many LSMtrees in the training set. At least not real ones.
That last point is a bit funny: I have seen people ask a model to write them Paxos and have it reply with a version that has the actual implementation stubbed out. Hilarious!
But there are mistakes, deeper mistakes, that you won’t be able to spot unless you’re familiar with the problem space. And how do you become familiar, except by doing it yourself?
