You are giving both of these tools too much credit.

As @D4no0 mentioned, generating code just so you can get started with task X is not such a huge deal as people make it out to be. Emacs and VIM have had “snippet managers” for decades where you press a particular combo of keys and e.g. get a function with a for-loop over a list written for you.

GitHub Copilot and ChatGPT are better in that for sure – but not by much.

98% of all commercial programming is reading, understanding and modifying code. Improving the 2% that are the writing of the initial code is not impressive by any stretch of the imagination.

And finally, having something generate bigger scaffolds for you comes with the severe disadvantage that you might not understand that initial code and then you’ll lose more time first trying to understand it and then actually extending it or even using it on a basic level.

If you don’t want to work in a circus, don’t work for clowns.

I believe this was written in a confusing way. The following chart from the article is a bit clearer in that those numbers mean suggestions acceptance rate. So 46% of the suggestions were accepted but this is a quantitative measure and it makes sense they are higher for Java as it has more boilerplate. But here is what I would love to know:

1. If 54% of the suggestions are rejected, does it mean I need to parse a suggestion and then discard it? Which would mean that most of the time suggestions could be slowing me down?

2. What is the time taken to accept or reject a suggestion?

3. Does Copilot tracks what happens with a suggestion? Maybe it is accepted and then it is immediately changed or removed because it was wrong?

In any case, I believe there are two separate discussions here, and they are getting mixed.

1. Are the AI tools in a state where we can consider them trustworthy or generally acceptable? To me the answer is no. Besides a huge potential copyright issue on tools like Copilot, which has made some organizations ban certain AI tools altogether, there is still a lot to improve. For example, researchers have found that code generated by OpenAI’s Codex contained security vulnerabilities 40% of the time. However, the tools will improve as there is a large amount of techniques and ideas that still have to explored and potentially adopted (such as reinforcement learning with static and security analysis!).

2. That brings us to the second discussion: should we do what is necessary to get Elixir working with more AI tools? To me, the answer is a 100% yes, because it will only get better and it is not only about code completion.