To me, the problem with this new paradigm isn’t trust but a whole new way of working.

Trust isn’t needed if we treat code generated by the bots like any other kind of code : it needs to be reviewed and the developer always has the responsibility of being an expert on the code they ship.

I use Claude Code + Tidewave MCP + my own documentation and examples and claude is able to generate code that is useful and correct, and test it, by following on what is already built, then suggest refactors if this “inspiration” leads to too much duplication.

The days I use it are absolutely exhausting ! The speed boost comes with so much code to read, review, think about and potentially correct… The code has to be read and reviewed thoroughly, patterns should be evaluated and tought about system-wise.

In a single day, writing part of a feature is so, so much more gentle than thoroughly reviewing N whole features and their tests… even if the model follows my style (which is even more exhausting : since it looks like code I would have written, it is harder for me to spot errors than if it was code that looked written by someone else, because it looks too familiar at first glance).

So, to me the problem of trust is maybe moved a bit further : what I do not trust is review of AI-generated code by another AI following a spec.

If we are optimistic about AI advances in the future, then it will be able to review itself way better than any human could.

And the only important metric will be the same as always: cost and accuracy. If an AI drives a car better than algorithms made by humans and causes less accidents then it will be used.

I’m not that optimistic about all of that though.

Yep, a really important topic! As a society, I think most people agree we want to avoid a not-impossible reality where AI agents are given personhood rights akin to corporations, leading to even further obfuscation and deflection of accountability / liability.

My personal feeling is agentic ai will only be acceptable in a sane future if we limit use of non-deterministic AI during the compile time stages and bar its use from runtime.

In your example, we might allow non-deterministic AI to help create code that is then verified by humans before it’s put into cars, but never allow non-deterministic AI to drive cars, detect humans, etc.