Pepperoni, Metrics and Software

Guest post from Don McGreal of Improving Enterprises

The disconnect between the delivery organization and the business is prevalent in the software industry. Somewhere along the line, the real vision behind our projects gets lost. We all know it.

Can better agile metrics help?

Let’s talk pizza.

You work at a growing pizza chain. You are part of their delivery organization. You and your colleagues are responsible for getting pizzas out the door and to the hungry customers. You have managers, drivers, phone operators, vendors, etc.

How do you know if you are being successful? What metrics will you measure your progress against?

I would like you to take a minute to think about this. Write the metrics down. I’ll wait…

agile-metrics-pizza-mcgreal

 

 

 

 

 

Did they look something like these?

 

 

 

 

 

 

Good. I thought so.

A perfectly reasonable set of metrics, right? If you care about improving your department, you should definitely track them.

Now, let’s switch roles. You are now part owner/operator of the pizza chain. I would like you to come up with another list of metrics that matter to you and your partners.

Write them down. Again, I’ll wait…

agile-metrics-mcgreal-2

 

 

 

 

 

Did they look like these?

 

 

 

 

 

 

 

 

 

Great! Nice work.

Now, are your 2 lists much different? I am sure there are a few overlaps (e.g., Customer Satisfaction, Costs, etc.) But I’m willing to bet that they are mostly very different from each other. Why is that? Is that even a bad thing?

Here are 3 things to consider:

1. EFFICIENCY

The delivery organization now has a set of shiny metrics they can work toward. The practices and processes they put in place will focus on moving these numbers. The assumption is that by improving these intermediate metrics, the business will benefit:

“Hey look, we spent all this time implementing these route efficiency algorithms so that we can save 30 seconds on each delivery.” You may reasonably assume that this will be good for the business, but it’s no guarantee and may even become more of a distraction. Popular practices are great, but they aren’t the end goal and we can’t lose sight of the business’ true needs. Otherwise you end up with a Cargo Cult mentality.

3. VISION

The more your people know and understand the true vision and goals of the organization/product, the better decisions they will make. Like it or not, assumptions and decisions will be made independently. You can minimize the negative impact of these decisions by educating your people on the true organizational drivers.

3. INCENTIVE

Your pizza shop has quarterly bonuses to hand out. What would you base them on? The delivery metrics or the owner metrics? Which ones have a greater potential for perversion? Any metric can be gamed, but you’ll notice that the more intermediate circumstantial metrics in the delivery department have bigger opportunity for abuse. This doesn’t just create unintended behaviors; it also reduces the transparency and, therefore, the usefulness of these metrics.

A few years ago, my wife and I went to dinner at a chain restaurant. Before leaving, we were presented with a customer satisfaction survey and told, “If we give all ‘excellent’ scores,” we would receive a free appetizer for our next visit. This was obviously not the way the business intended to use these surveys.

So what does this mean?

The delivery metrics aren’t without worth. They are very helpful, even essential, for guiding our more tactical practices. But we run into problems when they are used as false representations of value and set as achievement goals. If the business doesn’t establish the goals along with more direct value metrics, then the delivery organization will have no choice but to offer up their own.

Software

You have probably already done this, but can you now correlate these metrics with the ones on your software projects?

Which metrics are used by software delivery departments? Which ones by the business?

delivery-agie-business-metrics

*Learn more about Velocity, Code Coverage, Coupling, Cohesion, Code Complexity, and Lead & Cycle Time.

So how does your organization measure success? Are they spending too much time on the left-hand side? Are there opportunities to instead apply metrics that reflect true business outcomes? This concept of using direct rather than circumstantial evidence is known as Evidence Based Management (EBM). It is gaining traction in the technology field and has industry leaders like Ken Schwaber advocating for it.

With EBM, our pizza chain could evaluate the valuable outcomes to determine which parts of the delivery process contribute to them.

For example, if the chain had very large orders for 2 important customers come in at the same time, they may be inclined to wait until both orders are complete before sending out the driver. This would certainly improve delivery metrics (pizzas per trip, miles per delivery, fuel used, orders per driver), but may not be the best move for the business overall.

If instead, everyone involved was focused on EBM, they would be more concerned with the valuable outcomes that have a more direct impact on the organization (customer satisfaction, lead time, repeat customers). This may compel them to create a practice of sending 2 drivers if the orders are big enough.

This disconnect between the delivery organization and the business is prevalent in the software industry. Somewhere along the lines, the real vision behind our projects gets lost.

Challenging our managers, product owners and development teams to identify and promote metrics related to real outcomes is key to bridging that gap. This can promote true agility beyond IT departments and give organizations a real competitive advantage.

—————–

Don McGreal is VP of Learning Solutions at Improving Enterprises and is co-founder of TastyCupcakes.org, a comprehensive collection of games and exercises for accelerating the adoption of agile principles. Improving Enterprises is a leading software development company that offers advanced technology consulting and training across our 5 locations in Texas, Ohio, and Canada.

This entry was posted in Agile Methodologies, Agile Metrics. Bookmark the permalink.

5 Responses to Pepperoni, Metrics and Software

  1. Dante Vilardi says:

    Great post, Don. Your last bolded remark says it all. Gojko Adzic is doing great work in this area. His recent book, Impact Mapping , sets forth a framework for finding these metrics. Impactmapping.org

  2. Eva Bitteker says:

    Great post, Don!

  3. Jason Knight says:

    Excellent stuff. This makes me think that Scrum events like the Sprint Review and Sprint Planning should be saturated with the vision of the product. The vision should be made into posters and hung in our development rooms. If we chance to meet a stakeholder milling about after the review or whenever, we should start the conversation with, “So, how about that vision?”

    I just finished giving part one of a 3 part talk on Dan Pink’s “Drive” (thanks for the reference Don). In our organization, we are getting better at giving teams autonomy and allowing them to pursue master in our work; however, the purpose bit of all of this still seems distant. We definitely need to develop our purpose beyond just making money or being the best we can be.

  4. Matt says:

    Hey,

    Great posts — metrics can be very useful and drive a sense of awareness that leads to continuous improvement and alignment. As you mentioned, the over focus of the things on the left (e.g. velocity and code metrics), especially by the business can be and will result in the concept of “we must re-arrange the deck chairs on the Titanic so we can get more people on the deck” versus seeing the real business actionable metrics that lead to turning the boat away from the path it’s on — and quickly.

    And @Jason, teams I coach and worked with in the past use Movie Poster exercise to express the vision of a release; thus, ensuring constant focus and a shared discussion point for making decisions.

    Good stuff!
    Matt

  5. Pingback: Agile, is it just a delivery mechanism? - Scrum.org Community Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>