DaAnalyst

DaAnalyst

Any first-hand experience with Claude model getting "nerfed" down after using it for a while?

Been using Claude for over a week now (Opus 4.6 then 4.7, max subscription). Honestly, can’t hide my joy, at some points feeling even ashamed of my past stubborn skepticism. It’s amazing how much (more) I managed to do over this last week. The productivity boost is comparable (if not bigger) than when I switched from OOP to Elixir 7+ years ago.

Just when I thought I figured it all out (how to get the most out of it - btw, I’ve managed to achieve 0 slop for what I’ve been using it for currently), yesterday I got cold showered when I called to brag to an old friend of mine who runs a small dev agency and who I knew has been using Claude for a while.

“Enjoy it while it lasts” he laughed cynically. Told me the (consistent) excellent performance I’ve been experiencing would cost way more than a max subscription, and said he felt it’s been a bait and switch - that over the 8 months he’s been using he’s experienced very serious and frequent drops in Claude’s performance to being downright dumb and messing things up (which is the exact opposite of my experience so far).

A couple of weeks ago I saw a Zerohedge retweet of an engineer from AMD ranting about Anthropic having been nerfing Claude (deliberately downgrading its performance).

Since all this can be very subjective, I need to ask if someone here had this kind of in-person experience or maybe even a more in-depth knowledge on the matter?

Most Liked

lud

lud

In my experience, wanting the model to fix everything it finds is not a good idea.

First the model will lose focus, trying to do too much at the same time, leading to worse quality of output. Secondly because if the model reports to you or has questions for you, it will start to talk about multiple topics, leading to conversation where you talk about too many different points each round.

In general I find it is best to start a session, tackle some implementation, start a new session, ask for TODO fixing (only TODOs in the same module or otherwise accross codebase but related to a single concern). You can use an agent to scan the codebase for TODO (or ask it to use credo for that), then group the todos, categorize and handoff a fixing plan for the next session on a single concert, and start a new session.

(now of course sometimes the code is small, the fixes are easy so fixing todos along the way can be ok)

oleksify

oleksify

It usually depends on how it is used. Plain direct usage of prompting (even through Plan mode) most of the time result if crap output. Could be subjective of course, but it writes duplicated code, do not reuse already written functions and module, ignore codebase standards (even if they are directly written to local CLAUDE.md).

Sometimes psychosis starts out of nowhere, and it’s starting outputting pure crap. Or it states he did something, but never actually wrote that code. It can also easily wipe hours of work but just resetting git state. Things happens, to what Claude usually say - “I as so sorry!“.

Only way. I was able to keep it in line is through having very strict orchestration framework with a lot of hooks. So, in short - your friend is correct.

Besides those “features“, service has terrible uptime - it is down very often. It’s not a feeling - their status page all red.

Lucassifoni

Lucassifoni

Something I faced often was comments like “Hmmm… I see this #TODO in the file but not related to my changes. The next operator will fix it.” which is quite infuriating behaviour, since I left the #TODO for my next passage on this file, be it manually or assisted.

When I added “Think like a boy scout, leave the place better than you encountered it” in CLAUDE.md, it started saying “Hmmm… I see this #TODO in the file but not related to my changes. – wait : boy scout rule, I must fix it” which I must admit, I have not decided if that was more or less irritating than the previous version, but it fixed this refusal to handle low hanging fruit, like restructuring types, doing a small refactor…

Generally I see a lot of behaviours that I do not tolerate in myself, and being “augmented” this way is a bit tiring.

Where Next?

Popular in AI / LLMs Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
cjbell
Hey all — Chris here, CTO at Knock. We’re an Elixir shop (Phoenix, Oban, the works) and we recently shipped an AI agent for managing cust...
New
DaAnalyst
How much would you really be willing to spend (more) to keep it going with Claude should Anthropic go berserk with the rates, or put diff...
New
Cheezy
I am using Claude Remote Control for the first time this morning. AMAZING! I’m running one remote session on my iPad while running anothe...
New
calebjosue
What sort of libraries are available to integrate LLMs into your Phoenix Web Framework applications? e.g. Mistral, since these guys have ...
New
Cheezy
I spent the month of January building and refining a tool to help with Team / AI collaboration. Several friends have provided feedback an...
New
nbw
This is an open discussion to anyone using AI to help them work with Elixir in production-ready apps. I work on an Elixir app at work th...
#ai
New
kayoj
Hello I have been testing different environments for Elixir development & noticed something odd. When I run the Elixir formatter loc...
#ai
New
DaAnalyst
The new langauge/framework/library.. developers better start writing very exhaustive docs, ..and intent proving tests/examples, ..and int...
New
ricksonoliveira
Hey folks, I have been looking for an alternative to good old VsCode for a while. Have tried some vim and emacs editors with their plugi...
New

Other popular topics Top

skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52673 488
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 127089 1222
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54120 245
New

We're in Beta

About us Mission Statement