Hi! I’m thinking of building a tool to help me code using LLMs. It’s not a novel idea, but none of the implementations I tried to date ever really worked, and I saw that Elixir is a nice tool for implementing “A.I.” inference with nx.

The idea is pretty basic in itself, but I’m asking here how feasible it is to implement it with current technology as I am pretty inexperienced yet with Elixir & friends.

First of all, my program should build a comprehensive DB of my current code base, and keep it up-to-date, including ideally a model of my database (I’m using MongoDB). I’m no expert but I’m guessing this is what vector DBs are for.

Also, I should have deployed locally at least one pre-trained model, which is my next question. Which one is better suited for the task, preferably on consumer hardware? I have GPT and Bard apikeys as well, but I’m not sure how easily can they be integrated with a custom repository of knowledge. In future versions, a good idea would be to implement agents that can bounce ideas off of each other and help polish the final piece of code, including the user as one of the participants in the ‘conversation’. There can be coders, testers, linters, and agents for verifying best practices, documenting, and so on.

Lastly, there should be some sort of interface, for which I’m leaning toward Phoenix (don’t think there is any real choice here). The most basic UI should be a chatbot, but ideally, my program should be able to suggest changes, let me edit them, and write them to disk (future work)

So, my question is mainly the following: do you think this project can be done with the current state of Elixir’s ecosystem? I so, is my plan/roadmap reasonable, or would you change something?