Change my mind - the optimal AI prompt is the code itself

Take it you’re not a fan then

Actually, I’m using it/them on a daily basis and they are driving me nuts more often than not. For as long as they’re used as “smart” search engines (a hint book), almost everything is fine. Face them with anything deeper than that and it becomes a waste of time.

The other day I confronted two of them because they had conflicting suggestions. I gave them both a snippet of Elixir code (like 30 LOC) to double check if I did it correctly and analyze against possible race conditions.

At first I was convinced ChatGPT gave me a competent answer. The analysis sounded competent, going through different scenarios and all. Then just in case I also gave it to the newest Grok 4.1 - bam! Equally competent sounding analysis, but with a conflicting conclusion. Then I took the suggestion of one and gave it to the other.
“The other LLM is completely wrong.” was the answer, and then some. As if I hit its ego. Then I did the opposite (gave the suggestion of the second to the first one) - same result if not worse. In the end, I realized they both did the same mistake - neither took the current version of the Elixir Task module source code (until I explicitly copy-pasted it for them in the end) but they did blame each other in the meantime for not doing it.

Anyway, the above is just a drop in the ocean of my dissatisfaction and I truly cannot figure out how on Earth do some people think they can use these tools to write fully operational software.

I also just use it as a better search, although I’d have to say “different” search. I still haven’t quite figured out what my criteria are but I find myself reaching for a chatbot first or a search engine first depending on what I’m looking for. I’ve experienced it both ways where I can’t for the life of me get the answer from a search engine and then a chatbot one-shots, and the same in reverse where I can’t for the life of me get a chatbot to give me a coherent answer and then a search engine answers in the top result

I don’t really understand the appeal of AI for code gen. It can do some cool stuff, and I can see it being really useful for things like generating a bunch of hard-coded yaml or whatever, but I’m much faster at writing code than reviewing it, so it feels like a net negative in productivity to outsource the production of the code and take on a full-time reviewer role. Reviewing code is also much more mentally taxing, so it wears you down earlier in the day.

Most of the 20-30+ year veterans I talk to swear that AI is changing everything and always talk about how amazing it is, which I’m guessing is because they have 20+ years of experience in senior roles where they were mostly reviewing code and mentoring other people. I don’t have that level of experience though, and I imagine that if everyone starts with AI, they’ll never develop that experience because reviewing AI code is not the same as reviewing a human’s code. Human code review is a collaborative process that involves mentoring for more junior submitters and design discussion for peers. With AI, it’s just making sure it didn’t hard-code the API keys in the source files again.

Is it possible they have a stake in AI somehow?

Lamport begrudgingly wrote “Paxos Made Simple” despite believing that the best way to present an algorithm like Paxos is through proofs (i.e. a TLA+ spec). As it turns out, not only is the paper widely mocked for not being simple, but the explanation in the paper contains a bug which is present depending on your interpretation of the algorithm as specified in “plain English”. Vindication!

Programmers do not understand AI because it’s not for them. These models are a revolutionary technology for people who do not understand how to program computers. They are useful for doing things that we cannot do with computer code.

I remember seeing someone (I think it was antirez?) make a point that all of the hype for MCP (the protocol) was a case of everyone focusing their attention on the most boring and unremarkable part of a fundamentally revolutionary technology.

There is something so comically absurd about taking a technology which is revolutionary specifically because it can do things we cannot do with computer code and then using those models to write the same computer code we were already writing, except badly. Talk about missing the forest for the trees.

But hey, at least we finally managed to put radio on the internet!

I’m not sure I follow your point.

“writing code” is not something we used to do with computer code. At least not “writing ad-hoc, task focused code”. We could generate crud boilerplates, generate API clients based on specs, etc, but you would not have some computer program that could spit out an app that “tells if you’re in a national park when you take a picture” without that generator having most of the implementation ready somewhere and available for copy.

I think the opposite about MCP. It is great because it allows to just write some code for things that can be done with code, so the AI does not have to do those things, and just ask the MCP server to provide them.

Of course the fact that it is difficult for programmers to understand my point was in fact my point

The reason I love that XKCD so much is that it is a perfect, quintessential demonstration of what separates programmers from “regular people”. Programmers have trained themselves to view the world through a lens of rigidity that is not present in everyday life. We do this because that rigidity is a trademark of how computers function. If you have been writing code long enough you probably don’t even remember what it’s like to think normally. The first thing new programmers are taught is often “the computer will take you literally”; this is the beginning of a process of unlearning.

I suspect this is also why most open-source software is well-engineered but unusable for the average person. Overlap between good programmers and good interface designers is quite narrow because the latter must retain the ability to think like a normal human.

Machine learning is another world: one where computers actually work like non-programmers think they do. In the beginning the techniques were arcane and ineffective, but deep neural networks (nobody even says those words anymore) changed that, and pretraining on massive corpuses of language changed it further.

This is exciting because it makes trivial what was so recently impossible, as that comic nicely demonstrates. But it also means that programmers are the least equipped to understand the value-add, for the same reasons that they were most equipped to understand that GPS is easy and recognizing a bird is hard, right up until that dichotomy was erased by progress.

Attempting to coerce this technology back into the rigid box from which we have just escaped is absurd.

With respect to MCP, I do not personally have any opinion except that I do not care about it at all.

I understood it as why on earth would we write programs to identify birds when a chatbot is that program. Using an LLM to write code that… uses an LLM… isn’t very interesting at all. Am I off based on my understanding of what you’re saying here?

MCP is nice in theory, but considering there are still new products being made with SQL injection vulnerabilities, I feel like giving the user access to prompt a model that has write access to the server filesystem or production db is a disaster waiting to happen.

Except it already has happened. lol