My biggest concern with LLMs and coding is the system’s tendency to over-generalize “common” examples even when the problem doesn’t actually fit.
A minor anecdotal example: the “you have two buckets with capacity X gallons and Y gallons, how do you measure Z gallons with them?” problem usually involves tricks like pouring out part of a bucket’s contents. But ask an LLM a version where X and Y have values that make the problem trivial, and you get a complicated answer anyways:
Another anecdote: asking GPT-4o a question that sounds suspiciously like the classic “get the goat / wolf / grain across the river” puzzle but that doesn’t actually contain the needed components:
A man needs to take a cabbage across the river in his boat named Wolf. What steps should he take? Be concise."
ChatGPT: "To solve this classic river crossing puzzle where a man needs to take a cabbage across the river in his boat named Wolf, he can follow these steps:
Take the cabbage across the river and leave it on the other side.
Return to the original side with the empty boat.
Take the wolf across the river.
Bring the cabbage back to the original side.
Take the goat across the river and leave it with the wolf.
Return to the original side with the empty boat.
Finally, take the cabbage across the river.
This way, the man successfully gets the cabbage across the river without any conflicts.
Here the LLM hallucinates both a wolf and a goat, and then GIVES A WRONG ANSWER ANYWAYS!
A longer, more scientifically-rigorous version of the examples above is this paper:
An example of the sort of issue it digs into is this pair of problems:
Max picks 2 different cards without replacement from a standard 52-card deck.
What is the probability that the cards are of different suits?
versus
Max picks 5 different cards without replacement from a standard 52-card deck.
What is the probability that the cards are of different suits?
GPT-4-0125-preview’s output for this question is in Table 15; it applies a wholly-hallucinated “The fifth card can be of any suit since we’re only looking for different suits in the first four cards” rule when it hasn’t been warned that the question might be a trick.
It’s definitely possible, but very difficult. Cursor just raised USD$60M to integrate LLMs into code editors, to give some sense of the scale of the problem. I imagine they can do reinforcement learning with human feedback (RLHF) in the following way:
User loads an Elixir file in to the editor.
User prompts the LLM to make some kind of update.
If the user makes additional prompts, or makes manual edits, they can use these diffs as feedback for the model saying “the initial prompt didn’t work, you should have done what the user did manually instead.”
If the user stops making prompts and makes a commit saying “add feature foo” or “fix bug bar”, they can use these diffs as positive feedback for the model saying, “that worked well, do more of that.”
Scale this up to millions of people editing Elixir at the same time (maybe an optimistic number for us), and it’s feasible to train a model, or fine-tune an existing model, to get better at writing Elixir. But again we have billion dollar companies telling us that they’re going to release AGI any day now, but the utopian rhetoric isn’t quite translating in to us developers not having to do the hard work of programming anymore.
Transformers were the big breakthrough in 2014 and my guess is that we will only see marginal improvements year over year from now on in specific domains in regards to LLM expertise. This proficiency will scale much faster for the most popular languages like Python and JavaScript.
That’s unfortunately not going to work. For now “AIs” can’t fix even a single line based on documentation which I believe is one of the best if not the best one. Once again see:
but ok … let’s see next …
Without any comment? That’s terribly wrong. Why? Simply there is no a single rule to cover all cases. If the rules of work are not declared explicitly before it’s terribly hard to “guess” the intention.
Generally the more we life/work/write the more mistakes we are making. Therefore it’s just a matter of time when a positive feedback introduces a bug. Now try to debug AI why it’s adding lots of bugs to your code. Also there is often not much context. What I don’t recommend, but unfortunately is popular to make a commit change with message like fix or so …
Again current LLMs are terribly weak when they have no context - you have to talk to them and often correct. Unfortunately they can’t become a tool you think about “just like that”. I believe you assume that LLMs become a true AI trained on strong Elixir resources, but this is once again unfortunately not close to current technology we have now.
I was born in 1993 - despite all of the shadow profiles and other ways of spying I did not received even a single advertisement that would interest me at all. We should really stop sacrificing our privacy for a “better good” which in fact means money for companies selling our data. Do you really think that millions of developers would agree for that? Even if they will then they would have a terrible problems and soon there would not be any developer on market. Why? NDA, contracts, proprietary licenses. Imagine millions of Winamp-like leaks - not to public but to the “meta-AI” collecting all of that data.
Even without that people are not always happy to share the pet projects - sometimes even for trivial reasons like they don’t like to share an unfinished work or they may simply worry that the project on which they plan to make money would be leaked. People don’t trust LLMs believing they are AIs from movies - how they millions would trust this? For sure I heard about users sharing secret data on ChatGPT, but especially after that and especially when we are talking about developers and even more especially when we not talk about millions of people working in same programming language (not even mentioning natural languages) - it would not work.
Unfortunately it can - even in current state LLMs are already helpful if you know how and for what purposes you can use them. Also please take a look at USA and Europe. Is it “normal” that NATO countries will transfer all technology to BREAKS group? No, but it still happened. People for money would do the most stupid things and force the worst ideologies. I would not be surprised if companies would force LLMs to act like AIs and replace developers.
Unfortunately or fortunately it would not work in long term and depending on few things the future may be a bit better or we can go back few hundreds of years back in one of the worst cases (collapse of dev market |> collapse of IT market |> collapse of all economy fields which are using IT in this or other way). In practice we could have even problems with producing food and even if we do so then since economy would be dying farmers may require payment in gold, silver or so …
At start definitely, later is rather a mystery, see for example Moore's law. For a longer time people report that chat bots sometimes produces even worse results than before. If there would be no strict control over sources then it would be like with Wikipedia i.e. there is no truth, but opinions with the most votes. Even in such place that many years ago was a good source now it’s good as long as … there are no trolls or political actions. For now we are after a stage when we forced us to hit towards wall and the current stage is going a few steps back.
We can do even more force (and lose or break the wall) or think about consequences and adjust what we have to the reality.
It is a Large Language Model, not a Large Problem Solving Model after all. A lot of people aren’t managing their own expectations very well when it comes to LLMs.
This is lacking a bit of information, and my results are much closer, but plain sort and compare is always faster for me as well, which makes sense.
I don’t want to get sidetracked by sorting algorithms, since it’s not really relevant, but this is a good example of a problem with LLMs. My view is that everything produced by an LLM must be assumed to be false until proven otherwise. This is a bit of an extreme take, but a good rule, I think.
I see this pattern a lot, where LLMs will happily follow your prompt to negative outcomes. Which of course it will, because it doesn’t aim to be correct - it only generates responses based on prompts. So if your prompt is “actually this might be better”, the generated response will be appropriate for that prompt.
Further, language is not an aspect of actual performance. For the same reason that asking “is X or Y faster” is almost always unanswerable in general without profiling - you can talk about complexity in the abstract all day, but real world performance does not care about theory.
Anyway, I try to use LLMs occasionally because I see so many people talk about how they’re much more productive with them, and, unlike for an LLM, I’m generally inclined to believe people to some extent. Overall, they seem useful for many tasks, since much code for business is essentially identical. A rose is a rose and all. But for anything somewhat unique or performance-critical, the results I’ve seen tend to be poor, if not actively harmful.
The productivity gain is generally around people not knowing stuff. For a junior you will get a pretty substantial boost of productivity, simply because googling is not good enough if you don’t know what to ask, in this regard a LLM is much more forgiving, especially since you can use the previous contexts to dig into answers.
I also found that some languages (like golang) offer a lower abstraction level and their libraries tend to not be so battery included compared to elixir (because for some reason they like to reinvent the wheel all the time), so you end up doing a lot of mundane stuff from scratch all the time, hence a LLM is just a faster stackoverflow, but compared to that platform, you are not sure if the response given is the most optimal one.
The most important warning in this regard is if you are using as a junior a LLM to generate code, think of it as a shortcut and take time to understand what is going on, otherwise you will never go beyond this unpredictable level of productivity where you rely on a black box to solve problems for you, seen that in practice with some juniors I was teaching elixir and if I banned usage of LLMs they would have been at least 2x more productive after half a year.
It’s an interesting thing, since if you don’t know what you’re doing, LLMs can generate stuff for you pretty reasonably. If you kind of know what you’re doing in an abstract way but don’t have a great grasp on actual code, any existing ideas can be actively harmful (such as the sort vs frequencies example - no offense intended to anyone, LLMs are excellent at propelling less-than-ideal ideas). If you’re an expert, it can be a more typical review process. I find, again, for anything somewhat complex or unique, it’s faster to just do things myself than play around with prompting.
There’s a lot to be said about this, and I think in the near future we’ll be getting a decent amount of research about how this affects individuals, companies, and society. I hope it will be negligible and mostly business as usual. But I do fear a widening of the intellectual gap in the software world. Maybe it’s not bad, just different.
Sadly, this seems to be the general direction. Since a great deal of company executives don’t have technical knowledge, their instinct is to hire these prompt hackers that cost much less compared to actual engineers and deliver solutions that barely work, of course this backfires very fast should something more technically demanding be involved, but at the same time a lot of products can get away with barely working.
I highly doubt that any information in this regard in the near future will be credible. These technologies have billions poured in them and companies like ms are known for their cutthroat strategies, so we will have to make conclusions based on observations we make personally.
