TimButterfield

TimButterfield

While DeepSeek is interesting, when it tells me 'Too busy; try again later.", that makes it easier to go back to ChatGPT, where I very rarely have to try again. Of course, if I’m using the DeepSeek-R1 locally with Ollama, then I don’t have the busy issue. But, I am working with a much smaller LLM in that case.

Speaking of AI in general, I do find it interesting that, if you can be extremely specific in what is required, AI can do a somewhat decent job of creating a starter project for you. Of course, if you’re not sure you are specific enough, you can have it review your instructions and suggest options for clarification. But, you still need to have thought through a decent starting set of features. Of course, even then, there will still be cases where you have to correct the AI hallucinations. For example, when fixing resulting errors, it often suggests a fix that, while correcting one thing, will remove other required functionality. Or, it may get in a cyclic loop of delete A and add B only to later have deleted B and added A back, resulting in the original error all over again. This is partly related to limited context lengths. It cannot always hold sufficient scope simultaneously to make correct suggestions. So, you still need to keep on top of it and know enough to write code the old fashioned way, by using your own mind and with your own fingers on the keyboard.

1
Post #18
AstonJ

AstonJ OP

I’ve been getting the same since their servers were attacked. Hopefully it won’t be long before it’s back to normal.

It’s worth trying it on your local machine btw, I posted a macOS guide here - it’s surprisingly quick, though of course a much smaller model.

I agree - I think right now it’s great for grunt work or getting you up to speed on things, or for simpler tasks. But I see DeepSeek as the real/meaningful birth of AI - it’s open source and pretty powerful, and it is only going to get better from here… imagine where we’ll be 5 years from now. It’s an exciting time for the tech world, particular because it is truly open AI.

0
Post #19
CharlesO

CharlesO

You could try https://kimi.moonshot.cn/ as an alternative.

1
Post #20
AstonJ

AstonJ OP

Thanks Charles! I’ve been meaning to try Kimi!

It’s very fast! And really nice how it integrates web search automatically! However it requires a phone number to create an account… and I’m not sure I want to give it my number :lol: (DeepSeek and Qwen let you register with an email.)

0
Post #21
TimButterfield

TimButterfield

I have it running on MacOS already (M2 Max/32GB). Along with other models, I use the deepseek-r1:7b model with Ollama. Local deepseek is interesting in that the different versions have different bases. For example, the 7b version has a qwen base, while the 8b version has a llama base. I pulled both, but have not yet done much comparison.

As a test project, I wrote a React.js/Rust/Tauri desktop GUI to allow a SQLite stored chat conversation with the Ollama API (a micro version of ChatGPT run locally). It lets me select and use whichever LLM I have loaded locally and revisit those chat sessions later. I’m now working on a version of the app using Flutter to see if I can point a mobile version at a local Ollama API URL to have similar chats while selecting from the same loaded models. It may be cool to try the same thing with Scenic/Elixir to run as a desktop GUI app. Elixir/Phoenix could do it also, though that forces a web app for a local API; didn’t seem practical. In theory, I could also use Elixir + Tauri, but I haven’t taken the time to wrap my head around that yet.

It will become much more interesting when the AI can begin to ask us the questions we usually ask the clients or product owners, having the AI ask the developer those clarifying questions. “Given the specified set of features, it seems you are going in the direction of blah, blah, blah. Have you considered whether it should do X or Y also?” It should also be capable of verifying the build configurations and build results itself instead of just responding with suggestions to fix our build error prompts. But, I suspect it will need quite a bit larger context capacity than currently available before those type of things become possible. Imagine the context capacity that would be required for it to hold an entire decent sized project to perform that type of analysis to ask those things. But, right now, even a few larger files can exceed that analysis capacity, not to mention the additional complexity of links between those and yet more files, too. From conversations on the ollama discord, I think num_ctx is now only around 2k or 4k. That’s not a lot of space, though it is likely to keep growing over time. (Of course, those are the local context sizes I’m working with, not the larger ones possible on big servers.)

3
Post #22
AstonJ

AstonJ OP

I’ve only used the Qwen models, however I did notice a difference where I felt the 7b performed better than the 32b, even though the latter took longer! I posted the results here - if you’re on DT I’d be curious what kind of results you get with the same prompt.

That’d be pretty cool! In terms of design, they could show you initial concepts and ask which you’d prefer them to pursue.

I think the next meaningful progression will be when we can run the current big models locally, with the moe that can’t be that far off I imagine..

1
Post #23
ken-kost

ken-kost

Think your guide is universal. I’m trying it out on Windows. I got an RTX 4090 and I’m downloading DeepSeek-R1-GGUF and have a warning label Likely too large for this machine :joy: I’m curious what will happen. :bug:

1
Post #24
AstonJ

AstonJ OP

Haha! Let us know how you get on but I think you’ll need enough memory for the model to load into it :upside_down_face:

Thanks… I’ve updated the title :003:

1
Post #25
TimButterfield

TimButterfield

I’m on DT also. I’m on a lot of things, too many to keep up with. :slight_smile:
I used the same prompt you had used and tested these (results posted in your linked thread):

  • local Ollama API DeepSeek-R1:7b (qwen family)
  • local Ollama API DeepSeek-R1:8b (llama family)
  • Online chat.deepseek.com
  • Online Gemini (Flutter AI Toolkit)
  • ChatGPT-4o
  • ChatGPT-o1
2
Post #26
Nefcairon

Nefcairon

I’m a total newbie, so the following question might be very stupid:
Is it possible to feed a local installation with an existing Elixir app, so that I can ask questions and it gives me answers that take into account both the programming language and the existing code and help me?

1
Post #27