To me that’s Gemini Pro v2.5. Its quality noticeably dropped in the last month. 
dimitarvp
olivermt
The model seems to perform identical to me. Sre you sure its not just your perspective that moved?
1
Dmk
Quite certain, but hard to know for sure. I have had cases where it completely made up syntax elsif multiple times over the course of 2 days. Never did it before, haven’t seen it sense. Nothing else changed - same code base, same CLAUDE.md files, etc. That’s just one example amongst many.
0
dimitarvp
Last night Gemini gave me Java annotations as Elixir module attributes. Never happened before.
And the quite tragic thing is that I’m giving it easier tasks compared to two months ago… and it hallucinates like there’s no tomorrow.
It’s really difficult to tell whether it’s not a mix of factors of course. But the scope of tasks I’m giving it is actually smaller and it still fails.
I’m guessing it’s shenanigans over the free tier. They want the positive PR of their big context window – 1M tokens – but likely want to spend less.
It’s gotten so bad that I started considering subscribing to Claude Pro/Max.
0
olivermt
All the major vendors do batching of multiple requests on their backends, maybe they have upped the batching to cope with load and the context is leaking (more) between users?
For those who doesnt know: Your request goes into the model alongside other requests.. I am actually not quite sure how they do “sandboxing” or even separate the data enough to get distinct answers, but apparently this is why two requests with temperature set to 0 will still often give different results instead of deterministic ones.
4
te_chris
dogweather
The more I use AI for a Phoenix LiveView app, the less value I find in it. I need to re-write nearly everything. Usually the issue is around rescuing exceptions and then not notifying about them. Or using the non-raising version of a function and then not doing something reasonable when it returns :error.
It makes me think that the models were trained on miles and miles of mediocre code. After all, isn’t most code mediocre? I wonder how they appraised the code quality of their training corpora.
1
dimitarvp
They very likely did not. “Pour a few metric tons and the AI fill figure it out” is probably the most thinking they’ve ever done.
4
te_chris
I’ve ditched liveview for our app as we aren’t doing consumer facing web. Using ineritajs with react and vite, shadcn for layout. It’s just our admin, but it’s entirely vibe coded and actually works pretty well. Recommed if you’re doing internal stuff - wouldn’t be this cavalier for user-facing. It’s extremely efficient to develop.
2
zachdaniel
Creator of Ash