- Barry Whittle
- July 3, 2019
Agile development helps organizations move faster (while retaining quality), make smaller bets on new capability, and more closely align delivery to customer needs. IT leaders must validate the impact of Agile development with metrics that measure Agile productivity.
Agile metrics also track how cloud solutions deliver on their promise of low initial investment to enable experimentation, high developer autonomy to provide agility, and instant scale to accommodate growth and business rhythms.
Measuring Agile productivity
Agile time boxing is a bit different than the standard financial governance process. Rather than thinking about a quarter or month-end close, Agile addresses things at the program increment level (e.g., long term planning), at the sprint level (e.g., incremental delivery and sprinting within that program), and at the release level when shipping that value into the market.
The first step in understanding the metrics that we want to measure is looking at how the business is quantifying the value of Agile initiatives. A recent Apptio-sponsored study identified three critical areas for quantifying the value of Agile: associated time to market, product cost reduction, and the number of new products taken to market.
Output, time, and cost define Agile productivity. Organizations are using Agile ALM and other systems to plan and manage the activity associated with innovation, quality, and maintenance. To effectively understand and manage the value of that activity, organizations need insight into the investment associated with that activity at a product and team level.
This article discusses the first Agile metric to optimize business value: variance to plan.
Agile metric to optimize business value: #1 Variance to plan
Variance to plan is relevant for all IT spend, but for Agile it’s viewed through three specific dimensions: Am I allocating the teams that I said it was going to allocate? Am I doing the work that I plan to do? Am I delivering the investment that the business funded on time?
Analyzing investment burn down against sprint burn down tracks whether a team funded to do a specific scope of work, within a defined time, is delivering on its commitment and responsibly spending money. Organizations complete this analysis in three phases. First, at the program increment planning level. Second, at the individual sprint level to understand the output from two weeks of work. And third, when reviewing sprints at the end of a program increment to see what was accomplished to make future program increment commitments more realistic.
Watch the webinar: Five Agile metrics to optimize business value.
Three questions for your Agile metrics rollout
Jared Howell, Product Manager at Apptio, sat down with theApptio team and gave some answers to the questions most often heard when organizations roll out Agile metrics.
Hi Jared. Thanks for fielding these questions. I’ll read you the question, and you give us your best response.
All good. Go for it.
“We are just starting to standardize our Agile process and looking at different frameworks. How do you start to track these metrics?”
Align your methodology to what the business wants delivered. There are a lot of different frameworks out there. There’s SAFe, there's LeSS, there's Scrum, there’s Kanban. Not one methodology is going to be standard across every strategic initiative. Different methodologies make sense for different products and different methods of delivery.
Know the corporate goal to be accomplished, then standardize for the given teams that are delivering against an initiative. What methodology makes sense for those teams? If five teams support one initiative, align those five teams with each other. It doesn't make sense to align every team to every methodology across every function of business—align those together. This gives you the ability to track reporting against those initiatives while allowing teams to remain creative and not burdened with enforced overhead.
“My org is going through an Agile transformation, and we are getting pushback. How do you overcome objections to Agile transformation?”
The root of all those objections comes from the lack of transparency. When organizations discuss what finance needs vs. a technology requirement, they want to pivot the conversation toward value rather than overhead. You don't want to make teams feel like they must track time, report to finance, or be monitored by their Agile coaches just for the need of governance. Talk about the value of the financial reporting (e.g., capitalizing labor without taking a hit to the P&L). If you do Agile coaching and velocity goes up, improvements in the development cycle get product to market faster. Frame the worth of these metrics for the value they bring before they are regarded as merely a tool to measure (and punish) poor execution.
“Finance is struggling to deliver its necessary reporting with a new Agile methodology. How do you go about aligning finance and technology?”
This is an age-old story—and a bone of contention. Finance needs time tracking to capitalize labor: dev teams don't want to time track because it seems to be needless overhead when they’re doing Agile. This conversation needs to be value-driven, but it can't happen in a silo—one isn’t the boss of the other. Finance saying, “I need these things, so technology must do it” is as unhelpful as dev teams saying, “I'm not going to do it because that's what Agile says we do.”
Bring finance and technology together to talk about a suitable tradeoff. There are accounting strategies that don't utilize time tracking to capitalize labor. Look at some of those or use a story point estimate as a measure of effort. Finance might think that's acceptable to capitalize. I think really that the issue here is again that value-driven conversation of “how do we bring value to both organizations without introducing overhead?” So, I'd say that the short answer is, don't operate in a silo and try to make the tradeoff and meet in the middle. If we have organizations making their own decisions, then the Agile transformation is never going to work between finance and technology.
Jared, you are the product manager for Apptio Agile Insights. How does someone get started with that solution, and how is it different from what I can do in a tool like JIRA?
First, I’d love to talk to you directly about your specific requirements. What are your business drivers, your Agile maturity as well as your data systems, and what you're ultimately looking to measure? Let’s have that dialogue.
As for the JIRA question, we are different from ALM systems because we provide a business performance view of Agile productivity, quality, and labor. And specifically, we offer a combined financial and operational view of metrics that include the cost of Agile teams, work quality, and delivery based around having an Agile activity and investment model.
We have a specific model that looks at those costs, and the ALM systems are not organized in that way because they focus on the activity and work that goes behind Agile.
