Three Things You Should Know About Today’s Top Three Agile Metrics

If you’ve read the latest State of Agile report, you probably noticed that there has been a distinct shift in how people are measuring success on Agile projects today. In last year’s report, the top three Agile metrics were about basic work tracking, with ‘velocity,’ ‘iteration burndown,’ and ‘release burndown’ topping the list. This year? The use of ‘customer/user satisfaction’ and ‘business value delivered’ metrics nearly doubled – leapfrogging to the top two spots. ‘Velocity’ fell to number three – sliding in popularity by 25%. From our experience, this is good news. Here’s why we think this is occurring: 

1. Agile principles are better understood throughout the organization. 

Organizations have achieved a more sophisticated understanding of Agile’s intended benefits. They know that while they may have initially adopted an agile approach for its promises of speed, an agile mindset’s greatest benefit is to deliver value in the face of unknown challenges. In fact, the first principle behind the Agile Manifesto reads, “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” Speed is an advantageous by-product, but if a solution doesn’t deliver value to the business or its customers, companies aren’t getting the most important benefits of being agile. 

The shift also indicates that the agile mindset is permeating the organization. Initially, it gained its foothold in IT – where velocity has been a useful measure for agile teams, especially when the organization is not holistically agile and views task completion as progress instead of value delivery. The fact that today’s top two agile metrics are business-focused seems to indicate a stronger business presence and influence on agile projects and teams. Today, organizations want to measure success on agile projects in terms of business value, not just technical delivery. 

2. Business value and customer/user satisfaction are intertwined and essential measures.

The doubling of both business value and satisfaction metrics over the last year seems to indicate that organizations understand they need both measures to really understand how they’re doing. The goal of measuring ‘business value delivered’ is to understand if what teams are delivering is providing the expected value to the company. Measuring customer or user satisfaction is also about value, but it’s the value customers or users perceive they’re getting from a company’s products and services. They’re both important measures of success that work well when used together, because they provide two different perspectives. 

Here’s what we’re seeing as we work with our clients: 

• Most companies are using established ways to measure business value. Revenue, time to market, and market share are prevalent. Social impact focused businesses are also looking at metrics, such as communities helped, reduction in pollution, etc. They’re quantifiable, objective, and well-understood at the organizational level. The typical challenge with measuring business value is how the organization ties business metrics back to work done at lower levels to make it meaningful to teams and individuals. How much is one feature in a software solution worth? What’s the value of improving search speed by 30%? How much value did a particular development team deliver on a large project? Companies are trying to create multi-dimensional formulas to crack this code, often engaging external expertise to help. 
• Many companies are measuring external customer satisfaction using NPS. The Net Promotor Score (NPS) is designed to measure satisfaction by measuring the willingness of customers who buy a company’s products or services to recommend them to others. Again, it’s quantifiable, objective, and well-understood. And again, it’s tough to tie it back to work at a lower level to know which projects, teams, and individuals contributed to improved customer satisfaction. 
• They also want to measure satisfaction of their internal “customers.” These are primarily the employees who use the technical solutions teams deliver to do get their work done. It’s important, because employee satisfaction enhances productivity, which ideally leads to improved service and ultimately customer loyalty and satisfaction – and that drives value to the business. Quantifying it is tricky, however. Surveys are common here too, although emotions can cloud objectivity. We see more quantifiable metrics – for example, reduced call time, increased close rates, faster processing time, etc. – used to enhance survey results to help companies understand the value that’s being delivered to internal users.  

We think companies will continue to work on and make progress toward measuring business value and customer/user satisfaction as Agile’s values and principles continue to permeate organizations. 

3. Companies have recognized that velocity doesn’t represent value.

Velocity is the speed with which a team delivers items from its backlog. The most important information it can provide an agile team is the understanding of how much work they complete in an iteration.  Velocity is best used as a guideline for teams to plan the next iteration. This metric, however, is commonly misused, for example: 

• It’s used to compare teams. There is little, if any, value in comparing the velocity of one team to another, yet we see it all the time. Every team is different. One team may be working on user interfaces and the other machine learning – one composed of new grads, the other of senior developers. Many factors come into play when determining velocity, including complexity, experience, and uncertainty. Velocity may be a useful metric for a team to consider, but when it is used outside the context of a single team, it leads to bad outcomes, like encouraging teams to deliver more work instead of more value. 
• It’s used to measure size, not value. A team may measure velocity in terms of number of user stories delivered, looking for an increase over time, but this doesn’t provide a view into the value they delivered. They may count the number of story points represented by the user stories, but again, story points are simply an indicator of size, not value. The goal of an agile team is to deliver value to the customer and the business.  

The fact that velocity has decreased by 25% as the primary indicator of success indicates companies recognize that it has a place in their metrics toolbox, but it’s not the most important thing to measure. We find it to be helpful for new teams to get them used to agile estimation and planning processes. Once they’re high-performing, however, velocity becomes less important to track but still helpful to review from time to time just as an indication of patterns of predictability. 

Common challenges and best practices still come into play. 

Every metric comes with some common challenges. Misused metrics can create bad behaviors, people resist being measured, and the metrics take effort to collect and manage. The standard best practices still apply, including: 

• Clearly define the rationale behind the metric. Why are you measuring this? What will it tell you and what won’t it convey? What’s its importance to the company, to the team, to the customer? 
• Get buy-in to the metric’s importance. The people being measured need to be on board in a positive way. They need to agree with the value a measurement will provide. Transparency is also critical – supported by clear communication and open dialog. 

• Quantify as much as possible but know you can’t quantify everything. It’s challenging to quantify value and satisfaction. Many organizations have derived ways to do it, but they also take other factors into account when interpreting metrics based on them. 

 

How’s your Agile transformation going? Let us know how we can help.