Software Test & Performance Collaborative

A Quantitatively Managed Process

By Matthew Heusser on November 3, 2009 | 3 comments.

There’s a value system out there that says that in order to prove any argument, you need data – and not just any data, but hard numbers.  (My father in law, David Ellinghausen, a quality engineer at Ford Motor Company for twenty years, once told me “Without data, your just another guy with an opinion.”)

But let me tell you a story …

I work at a Software As A Service (SaaS) company.  We have a website; customers pay us money for a login to the website, then pay for access on a monthly basis, typically on an annual contract.  If we provide good service, people renew, and the revenue graph is a big, up-sloping curve. Why, to make money, we don’t have to do anything but keep our customers.  Yes – it turns out, as the quality of web apps continues to improve, that’s a bit of an understatement, but I hope you see what I mean.

Now, imagine it’s the beginning of the first quarter of a the new year.  We engaged a outside form to perform some research for us – to figure out our customers budgets, their social media strategy, to determine our budget and sales quotas.  Picture these two different scenarios:

Scenario 1: The account rep bursts in the room, his face flushed, his tie loosened, with some papers in his left hand.  He gulps, looks at the floor, slowly looks up, turns to the CEO and says “We just finished running the numbers. 20% of our customers are not planning to renew next year.”

Scenario 2: The account rep walks into the room, smiling.  He shakes the hand of the CEO, gives a high-five to the VP of ops and VP of sales, puts his briefcase on the desk, unbuckles it, and pulls out a glossly, one-page overview, complete with pie chart. “Business is up.” he says “80% of our are customers are planning to renew this year.”

How would you expect the executives to behave in those scenarios?  Very differently, right?

… wait a minute …

Those two account executives just said the exact same thing.

As well all know (my friend Michael Bolton pointed it out again last week at STPCon), even when we have them, we do not make decisions based on numbers.  We make them based on how we feel about the numbers.

Numbers (quantities) don’t manage processes – people do.  Oh yes, there are rare exceptions that are interesting optimization problems.  For example, how much inventory to keep on hand for the Christmas rush – too much and you’ll waste money, too little and the shelves will sit empty.  But ask a computer to recognize that a shopping mall was just put in accross the street, thus foot traffic will improve … it won’t do it.

For that, you need a human as the master, with the number as the servant, as it should be.