LLM Coding Benchmark by language

I haven’t seen this paper being discussed by the community, but I think this makes Elixir more relevant than ever.

TLDR;

The paper was written by Tencent’s AI team.

6 Likes

Additionally, I believe the reasons why Elixir enables LLMs to solve problems more effectively compared to other languages are the same reasons we appreciate Elixir for humans; it’s just more challenging to illustrate.

1 Like

Feels right to me. I don’t code much in other languages, but here are two recent anecdotes:

  1. Needed to build a simple utility to package certain files in my project with repomix, to upload it with GPT-5 Pro. Asked Claude Code Opus 4.1 to write a simple bash script. After a few iterations that were going nowhere (and produced an ever growing unreadable pile of bash), asked it to write it in Elixir - with the idea that at least I can read/understand it. It got it right from the first try, the code was much cleaner and easier to read.

  2. Needed to write a small utility to automate a workflow for image creation with Google’s Nano Banana. Since Google already has a Python library, decided to ask Codex to write it in Python. It made it work at the end, but after too many iterations, and the generated code is some unreadable mess. Decided to rewrite it in Elixir (and asked Codex to create a simple library to work with Google’s API in a process) - boom, much faster development cycle, and again, a cleaner code that I can actually maintain.

That was enough to convince me that even for small utilities, I should just reach straight for Elixir - somehow modern LLMs just write it better.

2 Likes