Becoming a better engineer: how to learn how complex systems work together?

This strikes me as somewhat of a strange question. Probably there was more to what your mentor said, but “Understanding complex systems” is not a software thing, it is a cognition thing. All disciplines involve systems of one kind of one kind or another, and excelling in any of them requires understanding them, and a bit more than that, I think. Certainly, if the only system you’re comfortable with is the (current) stack you’re using to build your CRUD app, then there is a lot of room for growth. But even if you understand just that very well, that puts you beyond a lot of web developers, in my experience, for whom “it works” is the end of the show, and honestly, beyond the reach of any AI-generated web apps in the indefinite future, IMO. I think software involves a lot more art than cannot itself be automated than most people seem to think. And maybe that gets us closer to the truth. It’s not so much that you have to know the intimate details of OS design to be an excellent web developer, or even “software engineer,” but the more really well-designed engineered systems (including some humble web apps) you are exposed to and understand, the better engineer you become yourself.

Yeah, that’s what “they” were saying was just a few years away when I picked up Rails 10 years ago too

Don’t get stressed about automation of programming just yet. Even if some organizations can actually do it they are too afraid to open-source it and will only use it to their advantage. The natural human greed will prevent such techniques being public for quite some decades in the future still.

Your question is very generic as has been said. Some of the Elixir books actually walk you through building rather complex apps (don’t have the exact titles in my head, sorry) and that likely will help you a lot.

But beyond that, you need to practice! If you ask a bit more concrete question I am sure the forum will help you.

Being “able to enter into a complex systems and make changes” can mean many things; it could mean just being able to understand a spaghetti code mess well enough to make changes. That’s a useful skill to be sure, but I think having a broader view is better here.

Complexity is an enemy of software engineers; we need to be able to deal with it to make changes to existing complex systems, to create new systems with minimum of accidental complexity, and to reduce accidental complexity in already existing systems. A few thoughts on how to develop this as a skill:

• In daily work, taking on challenging tasks dealing with multiple moving parts is good practice, especially when you don’t know those parts well yet. If you can take the time to refactor and improve the existing code and reduce its complexity, all the better.
• Try to take on larger responsibilities that require you to have a better understanding of systems you’re dealing with and to make architectural decisions.
• Improving understanding of fundamentals helps to better grasp the pros and cons of different choices that go into building systems. One book I recommend for this is Designing Data-Intensive Applications

Here is my practical advise:

• Learn touch typing and vim, and work on linux server command line and config
• Coding a lot, any lang at 50K lines of code
• Read one language spec, I read java one, so after that you will not fear any new lang
• Read Riak pagers and dig into Riak Core source code, and after that you kind of comfortable with any complex projects
• Use Wireshark

Hi @tio407

CRUD apps, if done well, are not a simple thing. One could spend a lifetime exploring the ins and outs of making a “simple” application work well, and besides, it takes a lot of skills to make a system simple.

Building software that is easy to maintain and evolve is in itself all but trivial, and even simple CRUD apps involve so many skills, often provided by separate specialists: back-end, front-end, infrastructure, reliability engineering, UI/UX, performance, …

Complex systems are not a goal: the goal is to design the simplest system that does the job well. Some systems are inevitably complex for valid reasons, like the subject matter being actually complex, but these are a small subset of the real systems. Most complex systems are “accidentally complex”, as they could be simpler, but they ended up being complex for historical reasons.

In any case, each complex system is complex in its own way, so it’s not possible to “learn complexity”. One can mature experience on the job on a particular system, but the acquired knowledge is hardly valuable in the job market.

A skill that one can learn, and that turns out to be useful in any job, is to “break down” complexity. It is the ability, for example, to divide a big problem into smaller ones. Or to model a problem in a way that only focuses on what really matters. Or the ability to recognize trade offs, and decide on which complexity brings more value than it costs, and which doesn’t.

My suggestion is to stop worrying about the immediate career implications of simple vs. complex systems: CRUD applications are not going to be automated in the foreseeable future, and anyway, as the tooling improves, so will your possibilities as an engineer. Focus instead on breaking down complexity when you inevitably meet it. Learn to model problems, and understand trade-offs. The skill is to make things as simple as they can.

If your immediate goal is to enter a complex system and make changes, you should stop worrying about understanding the complex system for a while. In stead, you should try to isolate out the part you want to change (zoom in), pretending other parts don’t exist.
Once you have zoomed in and contributed in several small areas, you can zoom back out and try to understand the big picture. I find this approach better suited for me. If I tryed too hard to figure out the big picture from the beginning, I would be overwhelmed by the amount of details and lost my confidence.

The only reason we write complex code is because the real life is complex and our job is modeling it. Understand how “entities” (the vague definition is used on purpose, because it could be literally anything) interact with each other on the real world and translate it to code on the proper abstraction level is the heart and soul of a good software engineer.

Our job is mostly turn real world systems into software systems.

For instance, if you are modeling a calculator app you have some entities:

• Digits ex: 1,2,3,4
• Operators ex: +, - , * , /
• Aggregators ex: ( , [ , ) , ]

And you can think how they interact with each other:

• Digits are the core values of your system, they are the inputs and the output it self
• Operators have semantic value on your domain, it means that are business rules (Math rules) that this operators applies, operators only make sense when used with digits 2 + 2
• Each operator have a relation with other operators, like * and / must be solved first and + and - last
• Aggregators applies business rules, that change the the default priority of operators

With this you can start modeling this calculator system in a software system:

• If you use a OO language, create a class for your Digit entitiy, which could hold information about the digit, beyond the digit value it self, or a struct could represent it in Elixir.
• Each operator could be defined as functions that expect digits as entries sum(digit1, digit2) inside their own modules
• Aggregators must be analyzed before hand on the calulation, and you will find out that use a classic data structure called stack to do it will help you a lot.

This is a very simple example that I could talk a lot more.. but the point is just to show that for modeling a complex system you have to understand the domain/business it is related to and try the best you can implement this relations as code.

I’ll probally be hated on this, but I think an Object Oriented language can teach you a lot about this, it is the most popular programming paradigm for a reason. (:

That said, to “enter a complex system and make changes” you have to understand what the system is about and how it models the reality. Knowing how and when entities interact with each other.

Entities are usually represented as a well defined document inside a project, in elixir we call them modules on OO classes, but the fact is that stuffs relates to each other.

A usefull technique is trying to identify:

• What is the input?
• What is the output?

The output must be related to the input, so as you read the code, you will identify how it transform or use the input to generate the output.

For example:

Input: Client Identifier
Output: Client Name

Predict: Reading this code you probally will find a database call using the identifier and retrive it’s name

Use this techinique of thinking about inputs and outputs, and predict an expected behaviour has helped me a lot when trying to understand new systems. Because I knew exact what I was looking for and more important why I was looking for it.

With this I can ask the right questions even when I predicted wrongly.

Real world example:

I was trying to understand a function that updates the user table, but the input of the function was not the user id, instead it was the user document.

I knew that another table has this information, linking document numbers with users ids.

But I did not found this on the code.

After this I could ask for help with the exact question: “How this functions knows which user must be updated?”

The answer was: “The user id is a hash of the document”

With this I could learn and understand the system easily, because I knew what the function should do and predicted (wrongly) how.

This technique will improve on time, because your prediction power will improve with time and knowledge! And It forces you to always think: “How I would do this?”

I’m already writing a lot, but to wrap up.

• Understand the context the software you are working with was built.
• Think about inputs and outputs, and how you would built this
• Great books to read:
  • Clean Architecture (General concepts)
  • Refactoring - Improving Design of Existing Code (General concepts)
  • Desiging Elixir Systems With OTP (Elixir oriented)
  • Elixir in Action (Introdutory book to elixir lang, but the practical advices about reasoning about elixir systems are gold)

Slightly off-topic but I wanted to link to an interesting talk based on something @oliveiragahenrique wrote:

No hate, sorry.

There is an interesting talk from Richard Feldman where he dives into the history of various mainstream languages and why they are popular. OOP being a great paradigm for modeling isn’t among them though. It’s a super interesting and entertaining talk, highly recommended!

Just for clarity, I don’t think OO is the best paradigm for modeling. You can do great models with any paradigm. But I do think that OO modeling is closer to the way humans naturally think, which could help if you are starting out with the topic.

Gonna check out the talk! Looks great! Thanks!