Do we need agile software development?

Allen Holub - The Death of Agile (Keynote from Software Architect 2014)

TL;DNW:
:-1: Scrum, Agile
:+1: agile

… and yes, being an agile consultant he has something to sell - so it’s always useful to keep that in mind.

1 Like

Uncle Bob

1 Like

@StefanHoutzager

This is a pretty old dream and I sadly never seen it working at all. Back when I started coding for a living (late 2001) stuff like IBM Rational and UML diagrams / interpreters were selling as hot cakes yet nobody in the company I was in ever managed to extract a value out of them. And believe me, they tried and tried and then tried some more, for 5 years, and were still trying by the time I left.

I agree that a business engineer must model processes in a similar fashion as you showed here, however:

  1. This is too specialized a position for most small-to-medium businesses to even know exists. Only huge corps where the mid-tier manager has to justify their spendings so as not to face budget cuts would really turn to such people.
  2. Most businesses have no idea that they might need such an expert and they haven’t realized their need for it yet (most likely never will).
  3. Most businesses would be skeptical of the real value such expert would eventually deliver and would most likely never hire them on this basis alone.
  4. It doesn’t stop bureaucracy; it merely redirects it somewhere else – namely forces the programmer(s) to know yet another methodology and be shoehorned into communicating through it instead of, you know, plain old human conversations with a paper and a pen on the table.
  5. Many managers just cannot breathe if they don’t put their finger on the process often – if you give them these cryptic (to them) diagrams they’ll dismiss you and just hand-wave their way to their familiar business lingo and you are back at square one where nobody pays attention to the diagrams. So using such tooling and processes requires tons of discipline and a realized need in the organization that these are actually needed and will contribute value.
  6. Most stakeholders could be business engineers because they hold critically important information about business rules and processes that they want encoded in a computer system but every single one I knew was hesitant or downright sabotaging to the idea of learning specialized terminology and tooling to be able to project their insights to the lower-level executors (the programmers).

I have a lot of respect for the people who manage to make all of this work – and you’re apparently one of them. So hats off to you! You’re a rare breed of a serious engineer.

Truthfully though, and back to the main topic, 2-3 day iterations and then touching base and correcting course is all that 90% of my projects ever needed. Everything else was unnecessary red tape that only annoyed everybody in the chain to no end and made productive work near impossible.

The original “Agile” meaning has been hijacked to hell and back, several times, and it’s a practically meaningless buzzword in the IT area for a while now.

And the so-called “Scrum Masters” are basically overseers that the mid/high managers hire so as to have a more intimate knowledge on the internal workings of the team and be able to scorn people more successfully. That’s all I’ve ever seen from them. They’re basically the king’s executors. Like Hela in “Thor: Ragnarok” said: “They not only executed the king’s enemies, but also executed the king’s vision… [pause] …but they mostly executed people.” – I think that captures the essence of the Scrum Masters pretty well. :smiley:

4 Likes

I find the agile manifesto not a very interesting paper. Just a couple of statements put together by a couple of guru’s (see the guru problem: http://darkagilemanifesto.org/dark-side-of-agile-janes-succi-splash-2012.pdf ). What I find important is making your software agile. When your software is not agile you can sprint and talk your heads off on endless scrum meetings, but all you do is push an elephant (hear this great presentation by Hickey: https://www.youtube.com/watch?v=rI8tNMsozo0). Another one you will like (it’s in line with your remarks about the scrum master role: https://www.quora.com/Why-do-some-developers-at-strong-companies-like-Google-consider-Agile-development-to-be-nonsense/answer/Michael-O-Church).
On your remarks about bpm as I showed (a couple of things): I do not find that processes should be modelled in this way (did I write that?) perse. It is a choice that you have. If you do not choose to use it, the many years of research and development and use in practice of bpm can learn you a couple of things. Last time I showed my small (very very incomplete) bpms to a productmgr at the company where I’m doing a contactorjob at the moment, this mgr noticed a lot that is lacking in their homegrown workflowsolution. Maybe you have not seen working solutions, but they exist, in abundance. F. e. the bpmn and dmn modelers I use are open source from camunda, you can find them on github. Zalando works with camunda bpm https://camunda.com/case-studies/zalando/. Scepticism is good, I f.e. recently put messages about external dsl’s like bpmn on this forum. But it is good to inform oneself also, even about things that nauseate sometimes, to be able to construct opinions that are well grounded and eventually learn a couple of things that might prove of use.

@StefanHoutzager

I fully agree we must all keep an open mind at all times. I’ve dug myself into such a hole in terms of career and thinking before I discovered Elixir, Go and Clojure (LISP) that I cannot even begin to describe how narrow-minded and stupid I was. What divides great programmers from mediocre ones is the ability of the former to constantly self-improve, be curious and never settle for “good enough”.

To that end, I also agree that ideas and tools like BPMN and DMN are useful and practical. And that they might open up new perspectives in front of their students. All of this is true.

What I find myself in some disagreement with you though is the necessity of these ideas and tools; the criticality of their presence in a given project. Even though we must always strive for being better, we only have so much time to work (or even live). I am currently in the process of working the depth of my new skills (before going wide again, namely learning other languages / frameworks) and maybe this is why I somewhat degrade the value of process modelling tooling. Excuse me for that, I am trying very hard not to be biased. Just trying to combine having a lot of culture plus getting work done. It’s a tough balance.

As for the links, I already read all that you pasted in this thread. Awesome reading! I am very thankful for your sharing them.

2 Likes

9 posts were split to a new topic: Agile Development Tooling

Here’s an interesting discussion btw about that old dream (the myth of the citizen dev etc) / bpm.

2 Likes

Surviving Your Inevitable Agile Transition - J.B. Rainsberger (2014)

Should have the subtitle “The economics of software development”

whyrefactorSmall

3 Likes

Even Goldman Sachs uses BPM (from Camunda).

Goldman Sachs, who are using Camunda for their global workflow automation, 
is a tech company (and I do not think they pursue a low-code app development strategy)
11.30 AM - Case Study Goldman Sachs

Goldman Sachs Workflow systems provide vital infrastructure to the operation of the firm, 
coordinating the execution and delivery of millions of business-critical tasks across the 
entire Firm (including trade facilitation, regulatory rule computation, exception processing 
coordination, personnel onboarding and thousands of other business processes).

In this presentation, you will learn how Goldman Sachs is leveraging Camunda software 
at the heart of two of its Workflow platforms:

Workflow Elements – The Camunda process execution engine is integrated with 
a task engine, task cache and entitlements cache, wrapped by REST APIs, to implement a 
“managed appliance” which integrates with Firm systems and orchestrates both human 
and system processes at enterprise scale.

Decision Services Platform – The Camunda process execution engine is leveraged in 
our in-memory Decision Flow engine. The engine integrates high-performance
rule execution with data integration in an extension of the DMN standard. The platform 
leverages GS’s own open-source jDMN execution engine.

The Twitter discussion is pretty interesting, spent 15-20 minutes on it.

The inevitable conclusion is that it seriously depends on the needs of the organization. Some of them have outgrown the need for 20x smaller apps and want something huge that can orchestrate all their needs. I guess in this case BPM and family are a good fit (if used judiciously, as the Twitter discussion states).

That’s IMO a pretty rare case however. And is a niche for expensive consultants. Most of us programmers don’t want to be put in a very specific corner and if sh*t hits the fan, we want to have good career prospects.

Elixir is a niche at the moment also. And we’re not cheap, are we? :slight_smile: About orchestration / choreography Bernd Rücker from Camunda wrote interesting things, f.e.:


edit: this is an interesting read also:

1 Like

Is it not a means to sell your company for two times it’s current value in two years. Hire agile consultants, impose scrum on all teams, put the ten agile commandments on the walls throughout all buildings of the company. I doubt this is too cynical in many cases. https://www.jbrains.ca/
The message of Rainsberger is not my cup of tea.

In working with multiple companies and highly varying social structures, these are my two cents:

  1. Doing any kind of organizational structure ‘for the sake of it’ does not work, because people don’t know why they are doing it. This is very similar to attempting to uphold ‘universal laws’ without understanding why these rules (that usually are not as universal as they are claimed to be) are actually good most of the time.
  2. It is really important to keep on talking with each-other, to ensure that all developers are constantly unblocked and everyone is on the same page as to where the project is going.

I personally don’t think there is 'one golden Agile’gah, the capitalization, but rather I view the ideas outlined in the agile manifesto and the research that has been done since then as a bag of techniques that might (but not necessarily all of them might) help your organization to keep communicating.

I’m working for multiple companies and highly varying social structures already twenty years. What I described is what I literally see at the moment at a large company. Scrum imposed by authoritarian management, the manifesto on the walls etc. It’s a shareholder-value driven (value several milliards of euro’s) company, now for sale again for around two times the value it had four years ago. The stages of the lifecycle wherein applications are in this company differs. But it’s agile everywhere. I don’t think that is wise (I agree less or more with Simon Wardley who says this: https://vimeo.com/189984496 and http://blog.gardeviance.org/2015/06/why-agile-lean-and-six-sigma-must-die.html ). I see agile failing in the maintenance of a large legacy app, I was hired as contactor to help solve those problems. Again: you can talk your heads off at endless scrum meetings, but if you do not make the software itself easy adaptable (agile) you’re pushing an elephant. So that’s what I proposed after a look at the code there: refactor that application, help them to become more truly agile. Did that for 11 months. There are much more problems with scrum, see the Michael O. Church criticism links I already sent. I experienced scrum as very inefficient. Far too much talking. That is what Erik Meijer (he has worked for some companies also) also says in his agile presentation (the link this thread started with), “did you check in any code today?” “TDD (XP) is for pussies” + “you’re f*cked” with agile. Scrum your scrum; adapt it to a process that works. Tear those commandments from the walls and throw them in the garbagecan. And fire those expensive agile consultants that have no knowlegde of writing software. But maybe it’s more practical to just leave the company and find one that is not shareholder-value driven (no long-term vision and so) and pragmatic instead of dogmatic.

2 Likes

Every effective manager of software teams I’ve worked with has used some form of “Kanban” to keep track of what each team member has on their plate, provide visibility, and make sure things don’t get forgotten. One time 10 years ago it was a simple spreadsheet sent out every week with individual discussions when something new came up.

I used this spreadsheet approach myself a few times in different places, finally discovering the various online Kanban tools which I like a lot.

When showing this to my wife she exclaimed that this is what she used 20 years ago in a manufacturing company. Some ideas are just obvious and useful.

2 Likes

I think effective management depends on the situation. One parameter could be the stage of the lifecycle of the managed application. One size fits all is not a good idea, and processes should be adaptable. See Wardley’s excellent presentation (again: https://vimeo.com/189984496).

That was a very good use of an hour, thanks for the pointer.

Even a mature commodity business that is still building any bespoke software in teams needs to co-ordinate that work in a way that’s transparent and efficient. A simple Kanban board with a prioritised backlog internal to that team is hard to beat for my tastes (which tend towards the highly pragmatic).

Identifying ‘what’ to spend time on (and what not to) is a different matter altogether. Related closely to growing a software system while keeping it adaptable. That is hard regardless of what process is followed.

I see you’re new here. Have you already watched presentations from Rich Hickey? This is one where he says interesting things about keeping software adaptable: https://www.youtube.com/watch?v=rI8tNMsozo0
He has more good presentations, google for them if you like.

As I see it,“Agile” has been used and over used to the point it has no clear meaning.

I say:

  1. Define Agile.
  2. What are the alternatives?
2 Likes

A hero of mine does not seem to like scrum much either: https://twitter.com/joeerl/status/994613703667601414

A programmers job is to understand the problem not go to 
stand-up meetings and scrum themselves to death.
3 Likes