Thoughts on DeepSeek?

I’ve been thinking for a while that “open” models are going to win at the end and therefore “closed” providers are massively overrated at the moment.

In my mind, it’s like open source databases vs. proprietary databases. Providers of proprietary databases make good money but at the end most stuff is built with open source databases. So, I believe it’s going to be similar for LLMs as building blocks of applications. I’m not sure about the chat apps, although open models should be cheaper.

I think part of the reason is that closed model companies lost their advantage in sitting on huge amounts of data because as far as I know synthetic data is getting more important for training LLMs. And now it looks like they also lost their advantage in having huge amounts of compute available as smaller companies innovate in that area.

For me it feels like these models get more similar to each other anyways. At the end users won’t care if they score 87 or 83 on some benchmark, and they all have kind of the same capabilities, so why not pick the cheapest one.

On the other hand, I don’t know what’s going on inside OpenAI, so there might be a surprising release of something that invalidates everything I just wrote.

Listening to Sam Altman speak, I don’t think he has strong beliefs in their models having to be closed-source or open-source. He wants to make money and develop AI. What I think will happen is that they will be prompted to make them open source. More, if they open up development of these models, that may be huge benefit for them, in terms of cost reduction. They can definitely save millions or billions on the DeepSeek optimizations alone.

Well it made the headlines in the UK.. but not only that, they made it a live story on the BBC homepage and they usually only do that for big events, here’s their summary:

Summary

• US President Donald Trump warns Chinese startup DeepSeek is a wake-up call for America’s technology industry

• The emergence of DeepSeek’s low-cost Al chatbot caused shockwaves on Wall Street, with Nvidia losing more than $500bn in market value

• Shares in other major technology firms fell steeply in value when markets opened on Monday. That day DeepSeek said it was hit by a “large-scale malicious attack”

• Tech-focused shares in Japan fell on Tuesday, while stock markets in China, South Korea and Taiwan are closed for the Lunar New Year holiday

• The DeepSeek app, which was launched last week, has overtaken rivals including ChatGPT to become the most downloaded free app in the US

• DeepSeek was reportedly developed for a fraction of the cost of its rivals, raising questions about the future of America’s Al dominance and the scale of investments US firms are planning

I think many people agree (and actually want that) Joel. People are understandably sceptical and wary of this type of technology being closed source and controlled by a small number of people or companies that many do not trust or like - and this is a big advantage for companies like DeepSeek.

there’s always a better and cheaper code

I haven’t read any of his views but perhaps actions speak louder than words in this case, which does make it appear he has a preference for closed source. Many people think they should even drop ‘open’ from the name as it’s misleading.

Agree completely! If anything this feels like the real birth of AI and it will be exciting to see what happens next. I hope Apple either allow us to use DeepSeek with our Apple products (instead of ChatGPT as they’ve been saying) or use DeepSeek’s findings to create their own LLM for us to use.

Imagine if they apply the techniques used by DeepSeek to create on-device AI tools…

I think there are many things at play here, including shareholder and partner preferences (hi, MSTF). Making a model open source is also a headache, from the legal POV and also just logistics. They did not have to do it either. There was no pressure from competition. LLama was always many steps behind and it’s also “kinda” open-source, but not really a free software.

DeepSeek is both open-source and free software, with a permissive license with very few use restrictions.

I’m not saying Altman is enthusiastic for open-source models, but he will probably not be against it either if it’s something they will look into doing as a response to rise of other open source and free software models.

I haven’t really followed it, but wasn’t it why he and Musk fell-out? Am sure there were some stories on DT about it… let me find one… here’s the quote I was thinking of:

Musk wrote “OpenAI ws created as an open source… non-profit company to serve as a counterweight to Google, but now it has become a closed source, maximum-profit company effectively controlled by Microsoft. Not what I intended at all.”

From: OpenAI Isn't So 'Open' Anymore, According to Elon Musk - dot.LA

If that is how they go I imagine because they have little other choice. But for many it will be too late.

I think this is why so many people are excited - it’s brought AI to the masses

I’m going to try setting it up on my Mac later, I tried Janus Pro WebGPU - a Hugging Face Space by webml-community but that’s not working for me (clicking on the generate image prompt does nothing)

This is really cool:

They’re running the full 671 billion parameter model on a cluster of 8 Mac Minis, each with 64GB of RAM.

On the subject of Musk and his grievances - I had a brief look into it but that’s up to lawyers to decide, and I don’t know how that’s going.

Generally speaking there is a way in American law to convert non-profit org to a for-profit org, and it usually involves sale of all assets, which I believe happened, and this is why Altman is not the owner, just a CEO. If the procedure was followed properly is another question that I am unable to answer.

I also suspect if there was more of the “open” in actions of OpenAI post transitioning to for-profit org, i.e. they continued releasing their models as free and open source software, there would not be this beef between Musk and Altman. Maybe this whole DeepSeek thing resolves that dispute by forcing OpenAI to open up again.

This is probably the best explanation/breakdown I’ve seen so far:

If anyone is interested we’ve just started a DeepSeek portal over on Devtalk. Think it’s going to be a huge topic this year!