What happens when agile teams go wrong? How is it that in long running agile programs, time and time again, I see teams that don’t build software that meets the operational requirements, that doesn’t do security properly, that fails to engage with design or user research or analytics in quite the right way.
I think that one of the main issues that we see in long running agile teams is one of lack of accountability and division between accountability and responsibility.
As you may know from some of my previous posts, I’m a fan of developers understanding business practices and working out why the organisation does what it does. If you’ve read any good books on how to run a business and how to structure an organisation, you may be aware of what we mean by accountable and responsible, but many developers I speak to don’t seem to understand. They often confuse the two and they get very upset when they see managers who claim to be accountable but don’t seem to be responsible for anything, despite the fact that this is exactly the model that we use in Agile.
If we are going to model how a team works in agile, I still hold that the basic RACI model works reasonably well. It boils down thus:
People within the organisation are Responsible, Accountable, Consulted and Informed about any given decision. These terms have specific meanings
Responsible means that you are expected to carry out the decision, whatever it is.
Accountable means that you are the owner of the decision, if it was the wrong decision, then it’s your fault
Consulted means that you probably advised the decision maker (the accountable one) but it’s not your decision
Informed means that you get told that the decision is made, but aren’t asked for your opinions or advice on the decision.
We can use these roles, combined with a matrix of decisions that happen, to model how an agile team should work together, so we might end up with a table that looks something like
|Decision||Product Owner||Tech Lead||Developer||Web Operations||User Researcher||Designer|
|Analyze research feedback and produce stories||C||I||–||–||AR||C|
|Examine tech buglist and produce stories||C||AR||C||C||–||–|
|Ensure service is monitored 24x7||C||C||C||AR||–||–|
and so on
Here’s the weird thing about Responsibility and Accountability.
Responsibility travels downwards in an organisation, and Accountability travels upwards. If the tech lead is responsible for delivering the stories, they delegate the actual writing of code to the developers. People in organisations tend to delegate their responsibilities or sub-dividable portions thereof downwards, to the next set of people down.
Equally, Accountability travels upwards, a product manager might be accountable for sprint to sprint decisions, but they may report to a Director of Product who is accountable for delivering on a portfolio of products, and they report to say the Chief Digital Office who is accountable for formulating and delivering on the company wide digital strategy [Note this organisation structure is for example, don’t copy it]
You also tend to notice a crossover between responsibility, accountability and delegated responsibility. If you are responsible for something, but you delegate it downwards, you become accountable for that decision and for decisions made by the people in the team you made responsible.
Theoretically this works fine, in an agile team, a product owner is accountable for product decisions, they get stories costed by the team and make prioritisation decisions about what work to select from the backlog. The delivery itself is the responsibility of the team and team members.
So why is it that I so often see operations or security decisions that get de-prioritised or not fully delivered on? One example is times where I hear complaints from team members, from people responsible for delivery that they know that certain things are wrong or broken, but that they don’t feel enabled to fix it. That’s an indication of where someone has responsibility but accountability lies elsewhere, they don’t feel capable of making the decision but don’t have a good line to accountable people.
The reason is that we often manage to fail to make the right people accountable for their decisions. If a product owner is making prioritisation for teams that takes up their entire team, they need to be accountable for all the consequences of that. But certain decisions consist of things called externalities. We make decisions based on cost-benefits mostly. We look at a decision about implementing feature A vs feature B, and if feature A costs more but delivers significantly more benefit we’ll go for it. If the cost of a decision is externalised, it gets taken out of the cost-benefit equation. A common example of this in real life is the decision to smoke cigarettes. Most of us know that cigarettes are bad for us, but because the threat of ill-health is a long way away it gets externalised compared to the cost of the cigarette vs the benefit of the nicotine. Another version is the cost of burning fossil fuels in the form of petrol. Because the negative effects tend to not affect either the supplier or the consumer directly, people continue to pick the most cost-effective solution rather than the better for wider society decision.
Agile has externalities, in the existence of the operational or security impact of product decisions. In most organisations, the pain of operating a service is felt not by the developers, and not by the product owner, but by the operations team. In particularly unlucky organisations, the accountability for operational uptime might travel upwards through a totally different chain of command to a board member in charge of operations rather than one responsible for delivery. This is the essence of DevOps for me, attempting to make developers feel the pain caused by decisions that they make, or more formally internalising the externality.
But the product owner is a particularly special role in agile since in almost every agile methodology I’ve seen, the product owner is accountable for the prioritisation decisions. It is them who decide how much time is spent on operational tasks, on security tasks and on product delivery. If they are accountable for delivering on a product strategy that is based on features, they are clearly in an impossible position when it comes to implementing the correct operational or security decisions.
What’s the answer to this?
Examples of attempts to solve this I’ve seen have included having dedicated team time for operational concerns, or tech debt or security stabilisation sprints. These are all examples of moving the accountability to someone else for some of the time.
I don’t know the right answer, but I suspect the right answer is to actually make the product owner accountable for all of those things. The product owner should be accountable for operating the service, for securing the service as well as delivering on a product strategy. How you make the product owner actually accountable for those decisions is something I haven’t worked out yet, but I think it’s possible, and I think it’s necessary if you want to build secure, operable software.