Agile Software Development
Are you sure you want to be agile?
Many organisations do not actually want or need agility. But by claiming to be agile and, at the same time, practicing an agile “cargo cult”, they harm themselves and the idea of agility in general. Instead of sticking to a broken wanna-be culture, they should look into traditional methods, which may be better suited for their situation.
When you ask a person if they, their team or their organisation is agile, you will almost always hear, “Yes, absolutely!” These days, it’s even offensive to question someone’s agility, much like if you were asking, “Did you brush your teeth this morning?”
The truth is that many organisations just pretend to be agile. What we see in reality are top-down micromanagement, “MVP” backlogs with hundreds of must-have features, artefacts being thrown over the fence, delivery taking forever, and everybody celebrating an absurd “cargo cult”.
The essence of agility
I don’t want to write an essay about agility itself here, but in order to get everybody on the same page, I will reiterate what I believe is essential. It is still, after all these years, a good idea to have a look at the Agile Manifesto and the Agile Principles. They are the foundation of everything and they explain very well what agility is about, for example:
“ Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”
Not much more, nothing else. Agility is about trust and enablement as well as the readiness to change, experiement, reflect, learn, and improve. All the agile tools, ceremonies and “frameworks” build on these ideas. Some of them are actually useful, others are just merchandise making money from a very simple concept. And all of them are useless if they are applied without understanding the underlying philosophy.
The self-deception of “pretend-agile”
But why are organisations pretending to be agile when they, in fact, are not? Why are companies lying to themselves and to their employees? There are mainly two reasons for this:
- First, agility itself is an asset. Organsations who describe themselves as agile are more attractive to customers, investors and employees. For instance, try finding skilled software developers with a job description that highlights your “well-planned and documented software development process”. Or, as a digital agency, try pitching your “rigorous planning and effective change management” to potential clients. Not too compelling, right? This leads to the situation that even companies who don’t actually want or need to be agile, adapt agile-sounding roles and introduce agile-sounding processes.
- Second, organisations are torn between the complexity and the volatility of software development. Complexity requires planning and control, but volatility requires constant adaption. Many decision makers see agile development as a way to “get stuff done a.s.a.p.” while having at least some kind of formal process in place. But instead of fully commiting to an agile workflow and the changes this would mean for them as well, they impose a cliché of agility which only serves themselves.
Failed agile does more harm than no agile
Many organisations, small and big ones, and not only in software development, find agile development intriguing — which is good. They read articles, and hear conference speakers, praising the increase in motivation and velocity, and they believe it will solve their own problems.
The problem is, however, that they often apply what they think is agile in a way that is bound to fail.
- Often they only pick a few agile elements to begin with, which is not bad per se. However, they pick the wrong ones, namely the most apparent ones. For example, they start with daily standups and planning poker, because this is the fun stuff they heard from others. Continuous improvement through retrospection is something for later.
- Even when they want to do it properly, they are afraid to commit to the changes, risks and investments that are necessary to establish an agile mindset and workflow. For example, they assign one developer who did Scrum in their previous company to be the “part-time Scrum Master”. This could work under very special circumstances, but in most cases, it will just die away after a few months, especially when that person has neither the time nor the mandate to drive a real agile transformation.
- In the long run, this leads to a phenomenon where we see a mix of agile method with traditional approaches. My favourite is a Kanban board combined with a pushy project manager who has never heard of the pull principle. Another classical scenario is a requirements specification disguised as a backlog, containing “must-have” features for an “MVP” expected to be delivered exactly as planned within six months time. (Spot the irony.)
- The most heartbreaking constellation, however, is when you have a motivated team that is in theory capable to work with a high degree of agility. But their surrounding organisation/management/stakeholders, while “allowing” the team to work in an agile way, don’t give them the autonomy to actually take end-to-end responsiblity. Instead, they restrict, control and impede the team. To watch this is like seeing an abandoned fawn in the forest, not yet knowing that its mother has been eaten by wolves.
This is sad enough for those people and organisations. They could as well work with a traditional process and benefit from increased velocity, efficiency, motivation and quality. Instead, they chose to become an agile zombie.
But they are not only hurting themselves, they also give agility a bad name. Because when they claim to be agile towards people who know even less about agility, when they show fancy tools and throw around buzzwords, they contribute to fostering a very wrong understanding of agility and failures that are then attributed to agility itself rather than to a failed implementation.
Be honest. Be bold. Embrace the waterfall.
Agile software develoment became increasingly popular over the last two decades. It was conceived when people realised that the reality of software development is often chaotic, erratic and volatile, even where process models and methods exists on paper. But instead of applying their own processes appropriately, organisations turned to agility, desperate for a convenient solution.
But in many cases, it might be better to not apply agile practices:
- If your undertaking is more project-like than product-like, i.e. it has specified requirements and a finite end state, why does this have to be agile? You should be knowing what to do from day one. You don’t? Well, then do your requirements engineering. And, by the way, you can have daily standups, pair programming, and bi-weekly stakeholder reviews anyway — this has nothing to do with agility.
- If your organisation is very hierarchical and top-down, you will not benefit of agility at the team level. Agility implies a certain degree of autonomy and end-to-end responsibility, which in turn requires a type of leadership which is based on trust and enablement, rather than control and formalities. If your organisation promises teams agility to lure them into commitment, the disappointment will be the bigger when people find out that it’s just a charade.
- Are you optimising for minimal cost rather than maximal value? If so, you should know that agile development, when done properly, is not necessarily cheaper than the traditional approach. It comes at a significant overhead of reflection, re-evaluation, discussion, try’n’error, refactoring. This will usually result in higher value, but it’s not said that you’ll be saving time or money.
- Speaking of time: Are you in a hurry? The truth is that agile development requires people who, beneath their technical skills, bring a fair share of self-organisation, self-motivation and social skills to the party. Finding those people is not easy. And getting to speed with a newly established agile team will take some as well.
In all these cases, you might be better of with the traditonal approach, a.k.a. “the waterfall”. Plan everything as thoroughly as possible, leave some room for human error on the way (but not for new features!), and then implement and deliver the hell out of it. Don’t forget to write a big invoice and have a party afterwards.
But admitting this requires honesty as well as seriously embracing traditional software development models, with a focus on rigorous planning and project management. There’s no shame in this; in fact, many high-profile software projects could not even be built in an agile way. How would you feel about the control center software of a nuclear power plant being developed in an agile way? How about the safety assistant software of your car? Or the MRI system at the hospital?
Pick your way, agile or traditional. But understand the rationale behind it and stick to your process. Otherwise, you will fail with every methodology.
