Choosing right Agile Delivery KPIs for Agile Projects


Choose right Agile Delivery KPIs for your Agile Projects.

We have been familiar with the progress percentage, or percentage completion, which is usually tracked for a traditional project management system. Traditional approaches usually focus on task completion within a set schedule and cost.


However, when we use agile methodology, teams, customers and stakeholders see almost immediate results. We basically use prioritization techniques to build goals and objectives and adjust timeframes and effort to deliver a product that corresponds to schedule requirements. Thus, during agile transformation I personally lean towards KPI tracking, using which you can identify several traits. KPIs help to understand human behaviour, quality, efficiency, establish flow, set accurate work in progress limits and most importantly build transparency into the system of an organization.


First, a simple definition of the word "KPI" - This is not as per any book or literature, this is put in as simple terms as possible, which you can understand it.


KPI - A measurable value which demonstrates how effectively we are progressing towards business objectives

Now, as you know the simplest definition already, let's go a little deeper in the conversation on how to set these measurements up.


First Step - Ask your teams constructive and though provoking questions.


  • What are we measuring?

  • How are we able to measure it?

  • Are we on track with our releases?

  • How do we determine if the teams blocked?

  • How is the quality of the application?

  • Are we leaving a lot of technical debt behind?

  • What is the cost of our technical debt, how much effort will be needed to burn it to manageable limits?

  • How healthy is our backlog, do we have enough items ready for our teams?

  • Are we writing our stories too early?


These questions will help start a conversation in your teams about what is the need of the KPIs. You can also have a conversation around how you will gather data to measure these KPIs.


The tools you choose for work management will also be a limiting factor on what KPIs you can or cannot gather. Thus, it is important to keep those in mind as well. Recommended tools can be Jira, Rally, Version One etc.


KPIs and metrics help us measure certain things about our people and work that they are doing and the way how they are doing it, which help provide us with an analytical insight into what is happening around us.

I want to give you a simple guiding principle of how you can make sure that the agile metrics / KPIs that you are provisioning,


Thus, it is imperative that before a measure is used, it is

  1. Discussed with the relevant people who are impacted by this.

  2. Piloted before it is rolled out.

  3. Abandoned if its dark side creates too much adverse performance.

When an organization transitions to Agile way of working, the measurement and reporting of what we do has to also change. You can no longer just use the "progress percentage" or "red, amber, green" metrics as a measurement of success. There are several things happening around that status, which we need to measure and report on.


As Spitzer says, "People will do what management inspects, not necessarily what management expects"

As consultants, coaches or scrum masters we spend so many hours discussing these metrics, making clever Excel files which work on the extracts taken from multiple systems and give us output using graphs of various kinds. We have to understand that the metrics we gather in any Organization revolves around three major things.


  1. People - Deploying people effectively is the first step to build successful Agile teams. We have to ensure that people who are working on your projects are engaged and productive.

  2. Projects - In certain organizations we only manage one project at a time, but this is not the necessary case in other organizations. As organizations grow, you will need to manage multiple complex projects simultaneously and for each of these projects you don't necessarily need to introduce additional KPIs all the time. You have to be able to make sure that your organization Is able to maintain visibility across all projects at any stage of the overall lifecycle.

  3. Productivity - How fast we are is determined by how productive we are. Monitoring and measuring productivity is ongoing and continuous. As consultants or as an agile coach it has to be our responsibility that the tools and KPIs that we provide to the organizations help in increasing productivity over the long term.

Thus, our KPIs for Agile delivery should also allow us to measure our progress on these 3 things. More importantly on multiple levels in the organization. In any organization we can find clear 3 levels of hierarchy, line managers, senior management and project management office and executives.


Line managers are those people who operate daily with their teams to deliver projects successfully. Senior management is responsible for the roll out of the best practices and processes and tools in scaled environment. Executives basically care about the organization wide maturity, efficiency and agility. Thus, we have to make sure that the KPIs we provide and measure are matching on three levels.


Pro Tip: 
From the industry a typical KPI landscape will look something like this. 

Executive level.
Organization level aggregations.
Customer satisfaction
Operational efficiency
Workforce planning.

Senior Management and PMO
Benchmarking Teams
Release Cycle Planning
Program Velocity
Program Burn-Up
Capacity Management

Line Managers
Release Forecast
Test Coverage
Quality Debt
Defect Density
Red Build Graph
Tech Debt

Some important questions which you can ask from your teams:


Which KPIs will tell us :


Are we on track? - Deadline vs realistic delivery date.

How reliable team commitments are? - Committed vs completed, estimation accuracy

Do we apply an efficient testing strategy? - open bugs over time, technical debt, defect containment.

Where are the bottlenecks? - flow, cycle time, touch time, improperly sequenced dependencies, blocked work in a sprint / release.

How to assess productivity of different streams including even vendors? - dedicated dashboards for teams and at each level.


I want to recommend some starting steps for you and your teams.


If you are using a sprint by sprint cadence

Ensure delivery predictability via sprint-by-sprint cadence

• Delivery Forecast via Burn up

• Velocity: Committed vs. Completed

• Sprint Plan Change

• Average velocity by sprints

• Backlog Health & Ready for Development

• Commitment rate


If you are using Kanban

Monitor the input & output flows of your service to ensure that SLAs are met & bottlenecks are avoided

• Average Lead and Cycle Time

• Reaction/Resolution Target Fulfilment • Throughput & Created vs Resolved

• Cumulative Flow Diagram

• First Time Right

• Time in Status


If you are working on a fix price assignment.

Keep a forecasted delivery date on track, control the scope creep, and keep quality on a high level

• Delivery Forecast via Burn up • Cumulative Flow Diagram

• Code Quality Summary

• Open Bugs Over Time

• Defect Containment • Time in Status


To keep track of your engineering practices and also to ensure that teams are following best practices.

Make sure your dev teams are fast and reliable, and having quality gates over code and CI/CD in green

• Unit Test Coverage

• Technical Debt

• Code Violations

• Code Base Change Trend • The Biggest Commits

• Most Frequently Modified Files


To make sure that your DevOps practices are taken care of and there is collaboration across the board.


Track and Boost the performance of your teams via faster lead time & increased deployment frequency

• CI/CD Pipeline Summary • Average build time

• Average build success

• Deployment Frequency

• Pipeline Mean Time to Recovery

• Average Pipeline Lead/Cycle Time



Happy Reading!!



What is it that fuels you? For me, it’s agileincrement.com

 

 I love writing about my passions, what interests me, what interests others, and sharing all of my thoughts with my readers. I hope you enjoy browsing my site and all of the unique content I have to offer. Take a look around; perhaps you’ll discover what fuels you as well. Read on and enjoy!

©2018 Agile Increment