Cool project, how much are RAGs useful in todays climate? seems like their usecase was due to early models needing over correction in the prompts, which doesnt seem like a thing for the latest frontier model. Im also on a similar timeline, started taking LLM’s seriously around early 25 and trying to get an edge without burning thousands on the latest claude model

If the frontier models are not omniscient, then you still need RAG.

How So? Ive honestly gotten lazy with my setup since gpt 5.4 and since then after a couple of prompts with the right harnesses I get good results and steering things properly without needing RAG , im daily driving deepseek latest model now because of cost , so I might need it, but so far dsh and v4 flash have felt great

What im trying to get at is RAG seems like a thing of the past for most models now

RAG is used for things besides writing code.

I apologize My world view is just code when it comes to LLM, what are some interesting applications used with RAG that is not code

Many places where specialized knowledge with citations is required.

For example, if you want a legal LLM that can give (somewhat plausible) answers for a paralegal to verify, you want it to research legal documents and generate an answer with – this is important – citations included. This is one reason there are so many startups with legal RAG.

Insurtech is another big one. You want answers about insurance policies, again, with citations to specific documents that you can click/verify. RAG goes there too.

Basically any place where there’s a body of documentation and someone is getting paid to look up documents and answer questions based thereon is at least a candidate for RAG.

That’s Interesting, now that you mention it I guess private data being uploaded and structured locally is an important use case for RAG, with legal RAG defining tool calls will also be super simple. I have not played around with RAG for quite a bit, but from your experience with it so far can it super charge a local model say qwen3.8 27b?

I’m also interested in insurtech and legal llm, can you talk more about that market and the demand, sounds super interesting, I can see the use case for legal LLM because writing up documents and court orders is a pain in the ass especially for small firms, but not Insurtech, also what other industries are looking for such tech?

That’s actually an excellent use case. A lot of places are gonna want local RAG for privacy or regulatory reasons – this will be a huge thing with medicine in the USA because of HIPAA and similar. Smaller models almost by definition have less memorized general knowledge, much less the specialized knowledge that crops up in RAG use cases.

The upshot of that is that local LLMs will be increasingly important. Local models being generally smaller, you’ll want RAG to get accurate answers on things.

yeah there are lots of open source chinese models and variants on hugginface, I think the perfect size range is between like 5b - 30b, its been awhile since I played with one, I have a llama & gemma models sitting around on my computer. I didnt know startups were seeing demand in real sectors I just assumed openAI and Anthropic swallowed up all competition, but privacy and token cost is an important variable plus I believe for most mundane tasks local models are really great when tool call and data is explicitly set.

I just wish to know what industry needs such tooling that isnt legal as its heavily crowded and elixir + the beam is an excellent language and runtime for orchestrating these models together

Medicine and insurance are two big ones.

Also, tech support in general could use RAG because lower-tier tech support reps are basically paid to be RAG applications.