This is probably the best explanation/breakdown I’ve seen so far:

If anyone is interested we’ve just started a DeepSeek portal over on Devtalk. Think it’s going to be a huge topic this year!

I have it running on MacOS already (M2 Max/32GB). Along with other models, I use the deepseek-r1:7b model with Ollama. Local deepseek is interesting in that the different versions have different bases. For example, the 7b version has a qwen base, while the 8b version has a llama base. I pulled both, but have not yet done much comparison.

As a test project, I wrote a React.js/Rust/Tauri desktop GUI to allow a SQLite stored chat conversation with the Ollama API (a micro version of ChatGPT run locally). It lets me select and use whichever LLM I have loaded locally and revisit those chat sessions later. I’m now working on a version of the app using Flutter to see if I can point a mobile version at a local Ollama API URL to have similar chats while selecting from the same loaded models. It may be cool to try the same thing with Scenic/Elixir to run as a desktop GUI app. Elixir/Phoenix could do it also, though that forces a web app for a local API; didn’t seem practical. In theory, I could also use Elixir + Tauri, but I haven’t taken the time to wrap my head around that yet.

It will become much more interesting when the AI can begin to ask us the questions we usually ask the clients or product owners, having the AI ask the developer those clarifying questions. “Given the specified set of features, it seems you are going in the direction of blah, blah, blah. Have you considered whether it should do X or Y also?” It should also be capable of verifying the build configurations and build results itself instead of just responding with suggestions to fix our build error prompts. But, I suspect it will need quite a bit larger context capacity than currently available before those type of things become possible. Imagine the context capacity that would be required for it to hold an entire decent sized project to perform that type of analysis to ask those things. But, right now, even a few larger files can exceed that analysis capacity, not to mention the additional complexity of links between those and yet more files, too. From conversations on the ollama discord, I think num_ctx is now only around 2k or 4k. That’s not a lot of space, though it is likely to keep growing over time. (Of course, those are the local context sizes I’m working with, not the larger ones possible on big servers.)

I’ve been thinking for a while that “open” models are going to win at the end and therefore “closed” providers are massively overrated at the moment.

In my mind, it’s like open source databases vs. proprietary databases. Providers of proprietary databases make good money but at the end most stuff is built with open source databases. So, I believe it’s going to be similar for LLMs as building blocks of applications. I’m not sure about the chat apps, although open models should be cheaper.

I think part of the reason is that closed model companies lost their advantage in sitting on huge amounts of data because as far as I know synthetic data is getting more important for training LLMs. And now it looks like they also lost their advantage in having huge amounts of compute available as smaller companies innovate in that area.

For me it feels like these models get more similar to each other anyways. At the end users won’t care if they score 87 or 83 on some benchmark, and they all have kind of the same capabilities, so why not pick the cheapest one.

On the other hand, I don’t know what’s going on inside OpenAI, so there might be a surprising release of something that invalidates everything I just wrote.