Thanks for sharing!
I wanted to try out Claude with Elixir for a while, but other than doing standalone prompts and copy-pasting, I’m not sure how to integrate it into my workflow.
I’ve checked out Aider now, but I don’t understand what a productive workflow looks like. Starting a chat about a file and going back and forth in the terminal?
productive workflow
Here’s one example. Use @preciz recommended configuration to disable auto-commits. Then with Neovim, install an aider plugin GitHub - joshuavial/aider.nvim · GitHub . Map ‘ga’ to open an Aider console in a vertical split. With that, you can ask Aider to refactor code, add functions, tests, etc. Aider will make changes to the code. If you don’t like the edits, the changes can be reverted.
Great advice, thanks! I’m a Neovim user, so it fits perfectly. Do you also use Github Copilot then? Or just this?
I’m using it with continue.dev as someone upthread suggested, and it seems rather limited by the fact that it doesn’t have the ability to access project files. It can only analyze the snippets I copy-paste into the prompt window. Maybe this is continue.dev’s limitation?
I’m in the middle of a huge refactor, and I need to be able to say “look at the (uncommitted) changes I’ve made in this file, and make the same changes in this other file”. That would save me a huge amount of time and headache, but seems beyond the tools’ capabilities.
I have the very same test for all these new “killer AIs,” and all of them show the very same shameful result.
It also shows you how to implement it yourself… It’s common knowledge that these things hallucinate.
I agree with you that LLMs absolutely give no guarantees that they’ll always be correct and that they can be confident in their failures. However, the lesson from an example like this shouldn’t be to conclude that LLMs aren’t useful or helpful in general. They are both useful and helpful more than they’re not… so long as you’re using them correctly and come to them with proper expectations. They are not going to write your application for you or replace the programmer: in fact, I think it takes a more senior developer to really extract the benefit from these tools at this point. I look at LLM given solutions/code much the same way I might approach a Stack Overflow answer to a question I have… often times I find at least enough to set me on the right path… but I also see answers that, through the lens of my experience, I know somehow just “aren’t right” or complete, etc. If I ask the LLM to write code for me, I’m now viewing it very much as a junior developer whose code I need to carefully review. The answers or code don’t always work out or save time, but over repeated iterations, I find that overall I’m clearly more productive.
I also find that these things can be helpful in just explaining the code written by others. What turned my opinion on LLM utility was a task I was asked to do. I was given an MS SQL Server T-SQL routine to convert to PostgreSQL PL/pgSQL which was a long mess of nested loops and conditionals. Another developer on the project asked an LLM to convert the procedure to PostgreSQL and it didn’t work so I got the ask. After my failing to get a correct result a couple of times (missing a conditionally set value a couple of nesting levels away kind of things and not having an example of what actual inputs or outputs were) I turned to the LLM… but I didn’t ask it to create the code, I simply asked it to tell me what the T-SQL routine did in plain English. The LLM got that description spot on, which I could verify, and I was able to produce a correct results (by hand) in a single well structured SQL query at around a third the amount of code. At that point I was sold but also understood the role the LLM should play in the workflow.
Finally, we mustn’t forget that humans can provide wrong answers with strident confidence as well. The original posing of the “Monty Hall Problem” (Monty Hall problem - Wikipedia), and many of subsequent responses to the published solution, demonstrated that, while we may fail in different ways than an LLM, we still can fail with great certitude that we’re right. So I will continue to ask for help from colleagues and delegate tasks to other humans, too… but I’ll also consider the possibility that the answers I get will be wrong or flawed in some way and so will never suspend my judgment or critical evaluations.
For me Github Copilot was too noisy. I’ve tried 3 or 4 other AI plugins, and I like Aider.nvim best so far. People are trying different approaches and it’s good to be able to quickly test new plugins. Yesterday I found a plugin that provides a CMP-completion source.
I find the LLM’s to be most helpful with small tasks: Bash scripts, SQL queries, GenServer skeletons, etc.
