Going Down the “Metrics” Rabbit Hole

When we are introducing Scrum to a new environment, we often get into debates, sometimes heated, with people who question the validity, truth and/or value of a particular agile or Scrum principle. My general feeling is that any time spent having a philosophical argument with a client/Product Owner is time not spent adding value.

In my experience, Product Owners and their organizations care about results. Rather than trying to convince someone of a particular point with a carefully thought-out and bulletproof argument (which probably won’t work), figure out what result the Product Owner wants. If the PO says he wants “metrics,” solicit him for the measure he considers most important. Ask the PO what he wants to measure and why. Going down a philosophical rabbit hole will not get the PO or the organization any closer to embracing Scrum. Getting results does that.

As a math geek who spent years gathering, interpreting, revising, and deriving all sorts of fancy “scientifical” metrics for measuring everything you can imagine, I have found that, more often than not, metrics obscure our picture of reality rather than clarify it. There is only one metric that matters: “working product as a measure of progress.” Everything else is, in my opinion, academic. Actually, there is one other metric that matters to business owners: ROI. But then I would put an experienced, high performance Scrum team up against anyone in that category and confidently guarantee greater ROI than a team using traditional methods.

Nonetheless, the metrics question is one of those philosophical rabbit holes that comes up often. Business people are conditioned to expect, value and trust metrics even if they don’t know how they’re derived, what they actually mean or how useful they really are. Often times, the value of the metrics is really only perceived.

Our goal is to determine what it is the business actually wants to measure and figure out how to make such a measurement in an agile context that does not interfere with the team’s ability to meet its sprint goals. What that means in a specific context depends on the situation.

The point is, though, rather than trying to convince the business that a particular metric is “not agile” or useless, we should adapt to the circumstances and try to find a comparable measurement we can make that will satisfy the underlying need. We should NOT do so, however, at the expense of the fundamental principles of agile development and Scrum.

Jimi Fosdick
CST