Becoming a better engineer: how to learn how complex systems work together?

Hi @tio407

CRUD apps, if done well, are not a simple thing. One could spend a lifetime exploring the ins and outs of making a “simple” application work well, and besides, it takes a lot of skills to make a system simple.

Building software that is easy to maintain and evolve is in itself all but trivial, and even simple CRUD apps involve so many skills, often provided by separate specialists: back-end, front-end, infrastructure, reliability engineering, UI/UX, performance, …

Complex systems are not a goal: the goal is to design the simplest system that does the job well. Some systems are inevitably complex for valid reasons, like the subject matter being actually complex, but these are a small subset of the real systems. Most complex systems are “accidentally complex”, as they could be simpler, but they ended up being complex for historical reasons.

In any case, each complex system is complex in its own way, so it’s not possible to “learn complexity”. One can mature experience on the job on a particular system, but the acquired knowledge is hardly valuable in the job market.

A skill that one can learn, and that turns out to be useful in any job, is to “break down” complexity. It is the ability, for example, to divide a big problem into smaller ones. Or to model a problem in a way that only focuses on what really matters. Or the ability to recognize trade offs, and decide on which complexity brings more value than it costs, and which doesn’t.

My suggestion is to stop worrying about the immediate career implications of simple vs. complex systems: CRUD applications are not going to be automated in the foreseeable future, and anyway, as the tooling improves, so will your possibilities as an engineer. Focus instead on breaking down complexity when you inevitably meet it. Learn to model problems, and understand trade-offs. The skill is to make things as simple as they can.

This strikes me as somewhat of a strange question. Probably there was more to what your mentor said, but “Understanding complex systems” is not a software thing, it is a cognition thing. All disciplines involve systems of one kind of one kind or another, and excelling in any of them requires understanding them, and a bit more than that, I think. Certainly, if the only system you’re comfortable with is the (current) stack you’re using to build your CRUD app, then there is a lot of room for growth. But even if you understand just that very well, that puts you beyond a lot of web developers, in my experience, for whom “it works” is the end of the show, and honestly, beyond the reach of any AI-generated web apps in the indefinite future, IMO. I think software involves a lot more art than cannot itself be automated than most people seem to think. And maybe that gets us closer to the truth. It’s not so much that you have to know the intimate details of OS design to be an excellent web developer, or even “software engineer,” but the more really well-designed engineered systems (including some humble web apps) you are exposed to and understand, the better engineer you become yourself.

Yeah, that’s what “they” were saying was just a few years away when I picked up Rails 10 years ago too