DaAnalyst

DaAnalyst

Any first-hand experience with Claude model getting "nerfed" down after using it for a while?

Been using Claude for over a week now (Opus 4.6 then 4.7, max subscription). Honestly, can’t hide my joy, at some points feeling even ashamed of my past stubborn skepticism. It’s amazing how much (more) I managed to do over this last week. The productivity boost is comparable (if not bigger) than when I switched from OOP to Elixir 7+ years ago.

Just when I thought I figured it all out (how to get the most out of it - btw, I’ve managed to achieve 0 slop for what I’ve been using it for currently), yesterday I got cold showered when I called to brag to an old friend of mine who runs a small dev agency and who I knew has been using Claude for a while.

“Enjoy it while it lasts” he laughed cynically. Told me the (consistent) excellent performance I’ve been experiencing would cost way more than a max subscription, and said he felt it’s been a bait and switch - that over the 8 months he’s been using he’s experienced very serious and frequent drops in Claude’s performance to being downright dumb and messing things up (which is the exact opposite of my experience so far).

A couple of weeks ago I saw a Zerohedge retweet of an engineer from AMD ranting about Anthropic having been nerfing Claude (deliberately downgrading its performance).

Since all this can be very subjective, I need to ask if someone here had this kind of in-person experience or maybe even a more in-depth knowledge on the matter?

Most Liked

lud

lud

In my experience, wanting the model to fix everything it finds is not a good idea.

First the model will lose focus, trying to do too much at the same time, leading to worse quality of output. Secondly because if the model reports to you or has questions for you, it will start to talk about multiple topics, leading to conversation where you talk about too many different points each round.

In general I find it is best to start a session, tackle some implementation, start a new session, ask for TODO fixing (only TODOs in the same module or otherwise accross codebase but related to a single concern). You can use an agent to scan the codebase for TODO (or ask it to use credo for that), then group the todos, categorize and handoff a fixing plan for the next session on a single concert, and start a new session.

(now of course sometimes the code is small, the fixes are easy so fixing todos along the way can be ok)

oleksify

oleksify

It usually depends on how it is used. Plain direct usage of prompting (even through Plan mode) most of the time result if crap output. Could be subjective of course, but it writes duplicated code, do not reuse already written functions and module, ignore codebase standards (even if they are directly written to local CLAUDE.md).

Sometimes psychosis starts out of nowhere, and it’s starting outputting pure crap. Or it states he did something, but never actually wrote that code. It can also easily wipe hours of work but just resetting git state. Things happens, to what Claude usually say - “I as so sorry!“.

Only way. I was able to keep it in line is through having very strict orchestration framework with a lot of hooks. So, in short - your friend is correct.

Besides those “features“, service has terrible uptime - it is down very often. It’s not a feeling - their status page all red.

Lucassifoni

Lucassifoni

Something I faced often was comments like “Hmmm… I see this #TODO in the file but not related to my changes. The next operator will fix it.” which is quite infuriating behaviour, since I left the #TODO for my next passage on this file, be it manually or assisted.

When I added “Think like a boy scout, leave the place better than you encountered it” in CLAUDE.md, it started saying “Hmmm… I see this #TODO in the file but not related to my changes. – wait : boy scout rule, I must fix it” which I must admit, I have not decided if that was more or less irritating than the previous version, but it fixed this refusal to handle low hanging fruit, like restructuring types, doing a small refactor…

Generally I see a lot of behaviours that I do not tolerate in myself, and being “augmented” this way is a bit tiring.

Where Next?

Popular in AI / LLMs Top

cjbell
Hey all — Chris here, CTO at Knock. We’re an Elixir shop (Phoenix, Oban, the works) and we recently shipped an AI agent for managing cust...
New
water
interesting thread Please wake me up when I can run it locally. I prefer privacy over convenience. This thread/fork split from: AI is...
New
FedericoAlcantara
Hi guys, I’ve been running Qwen locally for the last two weeks using an opencode + Ollama + qwen3.6:35b-A3B setup with q4_k_m quantizati...
New
ken-kost
Agreed; I just want to add that IMO Ash amplifies this even further. Especially since the dawn of usage rules. :cowboy_hat_face:
New
AstonJ
What kind of impact do you think AI will have on programming/tech jobs? Have you or anyone you know been impacted directly? Does it conce...
New
DaAnalyst
Exactly. A bit off topic, but I think the issue with the machines is they don’t bear any consequences. And even if they did one day, the...
New
AndyL
Elixir has rapidly changing hex packages, both public and private. For people who use AI Coding Tools (Aider, Cursor, Windsurf, etc.), h...
New
Cheezy
I spent the month of January building and refining a tool to help with Team / AI collaboration. Several friends have provided feedback an...
New
Vidar
I’ve been trying to come up with a fairly real life representative way of evaluating code quality from AI, and by extension the functiona...
New
sergio
Smart people online are talking about loops. Frankly I don’t understand what these things are in terms of tangibles. I use opencode a...
#ai
New

Other popular topics Top

hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54120 245
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement