Agile Monitoring is not project marketing. It is about effective control at all levels and the proactive dialogue within the team and with the management.
Some time ago I had a quite intense discussion with a client of mine about the purpose of project reporting. He elaborately tried to convince me that the purpose of project reporting is to create comfort to his line manager. The manager himself should be convinced that the project was well on track and should feel well informed about the overall status of the project. I strongly disagreed and showed him the perspective I aim for in agile management.
First let me clarify that this will be the last time I use the word reporting in an agile context. The word itself is highly misleading. I am going to use the word monitoring instead. In comparison to reporting, monitoring exceeds the pure view on the current state of a project by being decision oriented for the team itself and not only for archiving purposes required by the line manager.
The client I was talking to was a project manager and has been used to work in waterfall projects for most of his career. Together with me he ran his first agile product development project. In his career he experienced that managers can strongly intervene in projects and that this might have changed the course of the project heavily and might even have ended the career of some project managers he knew. From that standpoint I could absolutely understand his perspective on the purpose of classic project reporting which was basically to create some calm – well, before the storm.
The project he accompanied in the role of project manager was fairly large and was planned for a duration of several years. The project was integrated in an even bigger program which was planned in a waterfall sense for an even longer project duration. I was the agile coach for the only project in this program which ran agile. From his perspective his job description was to gather information on the status of the different projects in the program and aggregate this information to a point where the top managers could see… what? A green traffic light. What else. You know the term the project status is „melon green“, right? It states the fact that many projects pretend to have a „green“ status but the deeper you drill in the green melon the more red it becomes.
It is about effective management not about reporting
When I look across all the projects I was able to accompany so far I make an interesting observation. Data, derived out of project circumstances, is almost always aggregated across several hierarchical levels and is then handed to a position within the organization where a certain decision is executed based on that aggregated data. The person making that decision has almost no connection to the source of the data itself anymore – the product development work.
The client I described above faced the same situation. But how effective can such an executed decision be? Does such a top management impulse really have any effective impact referring to the aimed goal of that project? Sure top management executions are powerful. But do they have a managerial impact on the operation of the project? I say no. Because the effect of that management decision is limited to the direct reports of that particular manager. From there it slowly has to trickle down. This doesn’t only consume a lot of time. Also the intent of the top management impulse might change during its course through the different levels of the hierarchy on its way down to the project team due to certain degrees of interpretation and individual motivations by people in the hierarchy.
My client experienced the same in his career. That is why he prepared his project reporting in a way that it was, as he described it, safe to be clearly understood by his own 6-year-old son and by his top-executives.
When I think of agile project monitoring I don’t think in terms of manipulation, I think in terms of effective maneuvering and management that has an impact on product development. Impact in that sense means the management action that is derived out of monitoring information is effective in regard to the vision of product development. Feedback is a foundational practice in Scrum. So think the measurement in terms of feedback, not as the traditional lever to motivate behavior.
The effectiveness of an intervention increases with its immediacy. Which leads to the conclusion that those in direct operation of the project have to make the change, have to make the bold decisions based on their own project data, which then leads to the point where those people in the operation of a project need to understand the modes of operation within their own project domain which they need to actively influence.
So when you’re looking for effective management you have to set up the monitoring „from below“, which means from the perspective of the Scrum team. With this setup you create immediacy. Every decision you make based on real development data has in return a direct impact on the product development project.
Whenever a Scrum team escalates the need for a decision based on observations in the project, that Scrum team willingly gives away the responsibility for the decision itself. This bears at minimum two quite high prices: First, you accumulate cost of delay due to the time it takes until the need for a decision has climbed up the hierarchy and comes back to the team. Secondly, the quality of the decision is very likely not as good as it could have been if the team had made the decision on its own. But at least the team doesn’t feel responsible anymore. But isn’t that a fad? The team ultimately remains responsible but gives away its chance to boldly influence the outcome of the project.
A relevant discourse about the status of the project
„Where are we?“ That was the introductory question my client was used to be faced with in his recurring project status meetings. A question repeated a million times across so many projects, so it seems to be the one vital question of managers when project team representatives report about the project. But what does that question mean in terms of agile product development? Time and budget are fixed. The scope changes as the product development team learns. So the question needs to be reversed: „When is the project doing well?“ Start with the desired outcomes. This also helps to understand what might be worth measuring.
What we are looking for in agile monitoring is a relevant discourse between the Scrum team represented by the Product Owner and its manager. The first relevant discourse is the one about the goal of the project that is broadly described in the product vision. This discourse should be repeated over time. Later, during product development the Scrum teams needs to aggregate their gathered data in a way so that it leads to a specific question that can be answered by a manager. A question that is the starting point of a fruitful dialogue. This dialogue should take place every single sprint. The Scrum team decides which data it is going to present to the manager and the scrum team defines the intent with which they present it. Don’t kill your manager by data. In an agile monitoring dialogue, you aim for transparency and not for dumping random data on your manager.
The manager shapes the environment in which the Scrum team delivers. If project monitoring is the invitation to a dialogue, it implies also the invitation for an intervention by the management. So the dialogue with the manager focuses on the improvement of the working environment of the Scrum team. Therefore, you need data that indicates which offer the manager has to make to create a beneficial impact on the project.
The monitoring dialogue: Forget about any justification
When I think of my client and our discussion that day, I can remember that he didn’t feel very well before he was supposed to go to a project status review. He had experienced a lot of blame games in his career and he knew that from his perspective his status reporting presentation was always intended to make clear that whenever something wasn’t going well at least it was not his or his project teams fault.
This might be common practice but justification doesn’t have a place in this dialogue. It is all about support. There is no need for the Scrum team, which starts the dialogue, to automatically defend itself. The manager and the Scrum team share the same goal which is manifested in the product vision. Whenever you identify a risk in your project, talk about risk mitigation and not about who’s fault this could be. This monitoring dialogue is not intended to convince your manager that your project is on track and soothe his nerves. Agile monitoring is not project marketing.
The idea of agile monitoring exceeds the pure description of the projects current state. It is always asking for contribution at all levels. What does the Scrum team do to actively improve its delivery capability? What can the contribution of the manager be?
The goal of this monitoring dialogue of course is the generation of transparency which should lead to maneuverability of the whole organizational system. You want to establish a framework for action along with your management. If your dialogue doesn’t aim for this goal it is waste.
The foundation of this dialogue obviously is honesty. You are not intended to lie to your manager. And clear enough the manager is not intended to ask for being lied to. Every form of political theatre should be sentenced because it is causing waste. Manager, who punish transparency and honesty along with it, massively harm the whole organization. This behavior is dysfunctional as well and should not be tolerated either. The rate in which you allow openness and transparency in this particular dialogue determines the size of your framework of action. If a manager doesn’t want to hear anything about the Scrum teams’ impediments he loses his option to support in removing those impediments and ultimately contributing to the success of the project. He loses the chance to shape the environment of the Scrum team which basically is his job.
Done. The most reliable monitoring of all
The heavy need for a status update in a sense of project reporting stems, as far as I can see, from times when product development teams weren’t able to constantly deliver product increments which could be considered done. The urge for a status report on whether the team will deliver all the planned features in the given time and budget can easily be understood if a project team needs to work for several months on a concept on how to implement certain product features. I guess that was also the perspective of my client that day. As I said before, his experiences were based on many years of waterfall-ish projects.
In agile product development this urge is not relevant anymore. At least every sprint you see the delivery. In Scrum we only talk about things that can be considered done. The discussion doesn’t circle around consumed hours. The focus shifts to the actual delivery.
If your Scrum team cannot deliver features in the state of done than don’t dare to talk about all the things you’ve worked on. Accept that there was simply no delivery this sprint. Don’t talk about the pretense of delivery. Instead shift the dialogue to what the Scrum team needs in order to be able to constantly deliver “done” features within one sprint. Don’t accept the state that it is “not possible”. It always is. Managers can offer help to make it possible.
I’ve used the word effectiveness above. Now you can see what that means from a Scrum perspective. It means realization of the product not realization of the effort. Effort without delivery is just waste. Your client doesn’t care how much you’ve spent as long as you cannot show him on what value for him it was spent.
Relevant monitoring artifacts
So to make it even more frankly: The only relevant aspect of your monitoring is the realization of your product. All your monitoring artifacts should have a clear relation to this realization.
In this client’s organization and in many organizations before I’ve experienced that these organizations measured how well they are on track in terms of planning. But every IT organization in the world is responsible for the realization of a product and not only for planning. Even if this IT organization buys a product from a vendor. The product is not tangible as long it is not up and running. The whole effort of planning is not relevant and it’s not worth monitoring it. A finished concept is not a delivery in terms of product realization. Whenever you talk about the work you’ve spent you just disguise the fact that your flow of delivery is dysfunctional. Doing is worthless without the state done. Doing is not performance. Whenever you value doing over done you reward dysfunctional behavior.
What in fact is worth monitoring is the recurring answer to the question: What can currently be considered as done? How does the product evolve? What did the Scrum Team learn about the user’s needs?
Practically there are some pretty straight forward monitoring artifacts to be considered at different levels. If you take a look at the team level, use
- the cumulative flow diagram and
- the sprint burndown chart
to understand the delivery flow within the team. Actually you don’t need more. Sure enough there are plenty of other metrics and diagrams and they all might be useful but if you consider the bare minimum, start with these two at team level.
If you talk to your user all that matters is the shippable increment. In other words: What is done. And for that you have a review meeting. And yes this means that you need to invite the user to the review. The line manager or the top-executive is not the user. He is allowed to attend the review but the Scrum team does not present to him but to the user.
In the dialogue with the manager or other stakeholders you can use:
- The vision of your product – reconsider it from time to time
- A burndown chart with integrated buffer consumption for more than 3 sprints
- The current lead time for your backlog items
- The impediments – distinguish what will be tackled by the team and on which the management intervention is needed
- The current achievements in terms of learning and enablement of your Scrum team
With these artifacts prepared the Scrum team initiates the dialogue with the management. Impediments which the team cannot solve by itself become a vital part in that monitoring dialogue. Ask the manager for contribution and use the monitoring artifacts as feedback device to see if you’ve pushed the right levers and if the current influence on the project environment is sufficient.
In the discussion with my client that day we agreed to change the course of his reporting over time in incremental steps. First we started to measure two lead times. One for the delivery of the backlog items and a second for the decisions which have been needed by the Scrum team to make progress. Especially the second lead time led to a very positive discussion between the project manager, who was still representing the Scrum team, and the line manager, which over time resulted in a change of the project framework that fostered the decision making by the Scrum team. To make those decisions fast and safe to fail at the same time, the Scrum team started to work with explicit working hypotheses to show that they had made decisions and they were eager to check those hypotheses over time. Moreover, the impediments of the Scrum team became a part of this monitoring discussion which helped the management to understand the situation of the Scrum team and to offer some solutions for these impediments.
2 Antworten zu “Agile project monitoring – avoiding the agile watermelon”
Thanks Stefan, this is brilliant stuff and really helpful. Just one small remark: In my opinion, escalating problems is not necessarily equivalent to giving away your responsibility (though it’s often used this way). You can escalate a problem to get the support from a different level of responsibility. But this requires being very clear about what kind of support you need.
Thank you Bernd for your nice reply. I agree with what you’ve said. I tried to be pointed to make it more clear. The way you described escalation is the one team should aim for if they cannot solve problems on their own.