Matt Heusser’s Blog
Testing at the Edge of Chaos
Craft, Art, Creative Work, or Engineering?
If you’ve read this blog very long, you’re probably familiar with the struggle between a factory-like production model (or at least the cliche command-and-control 1950’s version of it), and the creative model.
In a factory, using the term ‘quality’ to mean doing the same thing every time might make some sense – if, by doing the same thing, we can drive out defects. Creative Work is different than that. In writing, for example, we value an author who tells compelling stories, but only if the story is unique and different. Why, sure, there are genres of writing that are produced in a templated, reusable style, but we have terms for them: ‘dime store’, ‘hack’, and ‘cheap romance novel’ come to mind. They may be cheaper to write, but they also have lower sales to match.
So the meaning of “quality management systems” in manufacturing – that of defining a process so that if can be stable, repeatable, and predictable, isn’t really all that helpful in a creative process. If that’s true, the question that follows tends to be the very battleground of this blog “So is software work more like manufacturing or more like creative work — or both? ”
Maybe it’s neither.
All this brings me to Edsger Dijkstra. You’re probably heard of Dijkstra; he was a computer pioneer who wrote the famous essay “GOTO considered harmful” and helped usher in the era of structures programming. In the testing space, Dijkstra is sort of ignored because, well, he sure spent a lot of time and effort on ‘formal verification’ and proofs of correctness, the kind of stuff we might mention in passing in software engineering I course, right along with CASE tools and other techniques that didn’t pan out.
But Dijkstra was much more than the one or two sentence summary we tend to give him credit for. He wrote a vast number of essays about software development by hand. The guy was a blogger back when blogging was hard, and I would argue that digging up those essays can save us time and effort today.
… decades pass ..
So Markus Gaertner and I have been discussing the nature of software development, and what we can learn from other fields – and what we should not learn — like maybe the perception of quality as repeatability. Then Markus finds me a link to this article by Dijkstra:
“On the Cruelty of Really Teaching Computer Science”
Here’s a few gems to start:
The usual way in which we play today for tomorrow is in yesterday’s vocabulary. We do so, because we try to get away with the concepts we are familiar with and that have acquired their meanings in our past experience. Of course, the words and the concepts don’t quite fit because our future differs from our past, but then we stretch them a little bit. Linguists are quite familiar with the phenomenon that the meanings of words evolve over time, but also know that this is a slow and gradual process.
It is the most common way of trying to cope with novelty; by means of metaphors and analogies we try to link the new to the old, the novel to the familiar. Under sufficiently slow and gradual change, it works reasonably well; in the case of a sharp discontinuity, however, the method breaks down: though we may glorify it with the name “common sense”, our past experience is no longer relevant, the analogies become too shallow, and the metaphors become more misleading than illuminating. This is the situation that is characteristic for the “radical” novelty.
Coping with radical novelty requires an orthogonal method. One must consider one’s own past, the experiences collected, and the habits formed in it as an unfortunate accident of history, and one has to approach the radical novelty with a blank mind, consciously refusing to try to link it with what is already familiar, because the familiar is hopelessly inadequate. One has, with initially a kind of split personality, to come to grips with a radical novelty as a dissociated topic in it’s own right. … adjusting to radical novelties is not a very popular activity, for it requires hard for. For the same reason, the radical novelties themselves are unwelcome …
It is probably a little more illuminate to go a little bit further back, to the Middle Ages. One of it’s characteristic was that “reasoning by analogy” was rampant; another characteristic was almost total intellectual stagnation, and we now see why the two go together. A reason for mentioning this is to point out that, by developing a keen ear for unwarranted analogies, one can detect a lot of medieval thinking today.
… (ten pages later) …
Nor are we above the equally primitive superstition that we can gain some control over some unknown, malicious demon by calling it a safe, familiar and innocent name, such as “engineering.” But it is totally symbolic, as one of the US computer manufacturers proved a few years ago when it hired, one night, hundreds of new “software engineers” by the simple device of elevating all it’s programmers to that exalted rank. So much for that term.
I don’t agree with everything Dijkstra says – the suggestions for formal proofs of correctness begin at page 12 and just keep going. Yet even formal methods can have a place – a niche – a tool for our toolbox. Google “software testing finite state machine” or “Harry Robinson Model Driven Testing” sometime. I’d certainly suggest that someone who reads the article will be more informed than someone who hasn’t.
But even with just that excerpt, you’ve likely gathered the theme that automatic computing is it’s own radical novelty – that metaphors and analogies can, in many cases, hold us back more than they help. So we can say that development or testing is a form of craft, or it’s like engineering, or like writing – but if we follow those too far, we end up into misleading.
My favorite example is the “Silver Bullet” in Fred Brooks’ famous essay. It’s an interesting essay, and he makes some wonderful, strong points, but software projects are not some monster to slay. If we could radically decrease the cost of software projects, well, that means software projects would have a greater return on investment, so the business world would want more of them. In other words, if we “solved” one software crisis by making development cheap, a hundred new development projects would come to replace the old ones, and we would have created a new “crisis.” It turns out, software development isn’t a ‘crisis’ at all — except perhaps for people trying to get something for free. No, software is an opportunity, or perhaps an enabler.
Can we learn from craft, writing, hardware engineering, building bridges and companies and so on? I certainly think we can; I pull the Stephen King Vs. Romance Novels example out a fair bit myself. But if we want an ideal to look up to, should it be software engineering?
I dunno. Maybe software delivery is like, well, software delivery.
This ties in to something else I heard the other day – Ed Catmull of Pixar labs was speaking at the Stanford Graduate School of business. Ed said that people like to have labels and slogans, like “Quality is the most important thing” in manufacturing, “The story is everything” in movies, or “Our most important role is job creation” in government; maybe “Agile” for technology folks. Once you have the slogan, you can throw it around without actually having to change. (Jerry Weinberg would call that the labeling fallacy.)
The problem with labeling in software is that we can say things like “Software Testing needs to become an engineering discipline!” and stop there, without actually studying what is working in testing, what doesn’t work, and experimenting with techniques that might work better.
So enough of this. My next few blog posts will be about what I actually do while I test. How does that sound to you?
Testers at Work
Several years ago, after a conference in Indianapolis, I was sitting at dinner with Mike Kelly and Karen Johnson when Karen pointed out that she was surprised how little her boss(es) understood what she actually did all day.
Since then, I’ve been listening to that story, and I hear it over and over again. Not just our bosses, but our parents, our grandparents, our spouses. Moreover, testing runs into a dilemma of competence, where people who are ignorant about testing are so ignorant that they do not realize it is a complex skill at all. In the words of John Bruce:
There’s one way I’ve found — though certainly not the only way — that you can tell a job has gone south. It’s when the work suddenly focuses on its most rudimentary component. I’d been writing, coordinating, and publishing all the documentation at the USC computer center, a responsible position. Within a very short period I’d become nothing but a typist. Later, in other jobs, I’d find the work had suddenly changed from being a system engineer to being the kid who pushes a cart around, replacing coffee-sodden keyboards and used-up printer cartridges. That’s a sign that it’s time to go, or if you don’t go of your own accord, someone will make the decision for you. When this happens, the window of opportunity isn’t wide.
When people don’t understand help desk work, it becomes call-forwarding.
When they don’t understand project management, it becomes list-keeping and status-reporting.
When they don’t understand testing, it becomes button-pushing.
I am less than excited about this; in fact, I’d like to start a trend the other way – recognition that testing is complex, challenging work, plus a desire to do it better.
I’ve got an idea.
Three years ago, a journalist named Jessica Livingston interviewed founders of high-tech startups – people like Steve Wozniak, co-found of Apple Computer, and Dan Bricklin, creator of the spreadsheet. She put those interviews down in a book called Founders at work.
The book described what the company founders actually did, not in generic, general terms but in specific; what life was like, what decisions they made and why they made them.
Then last year, the next book in the series came out – Coders at Work.
Of the books in the world, do we have room – and do we have time – for Testers at Work? At book that, instead of trying to provide a generic test strategy for everyone, starts out by interviewing successful testers, finding out what was unique or different for them, providing some practical experience, real examples, and reference information?
If I wrote it, would you be interested in it? Who would you like to see interviewed?
Professional Service Firms and Twitter
No, really, those are two different things.
Yesterday I read this article on solving the IT turnover crisis. The basic idea is to look at how professional service frims like Earnst&Young do their hiring and staffing.
I’ve worked with quite a few people who spent time at that type of firm. Basically, everyone is running around, working very hard, collecting lots of billable hours, trying to make partner. A few people make partner, but many more decide it isn’t worth it after a few years and bail out.
At those firms it is expected that less than half of your peers will be here in five years. For that matter, having a big accounting firm on your resume is considered very valuable; so you can travel and work for a big firm when you are young, as a consultant, then settle down into a corporate job once you have more responsibilities. (Mortgage, Family, and so on.)
Offhand, I can only think of a couple of software houses that work this way – thoughtworks and objectmentor. Thoughtworks in particular has produced people like Jason Huggins (Co-Creator of Sellenium, now at Google), Steve Freeman (Co-Creator of Mock Objects, now independent) Simon Stewart (Author of Web Driver, now at Google), and Chris McMahon (Inventor of testing heuristic “Don’t test for blocking conditions”, now at Socialtext).
Thoughtworks even has an alumni blog. “Look at what the smart people who used to work for us are doing now!”
Overall, to me, the model makes sense – work really hard for us for a few years, build a reputation for yourself, then either go into the world and succeed, or, if you can bring in clients and don’t mind traveling, stick around and do very well.
I’ve long believed that companies that say “Document in case you get hit by a bus” really mean “document in case you get hit by a better job offer.”
It sounds like Thoughtworks is at least one company in our field that manages that way.
I wonder what the world would be like if we saw a lot more of that?
Secondly, about twitter. I just joined twitter, the lightest-weight-est blogging format on the planet. Posts are limited to 200 characters in length and are generally one sentence long. You can see my profile here. Yay!
What is a professional?
For the past few years now, I have heard countless exhortations for software testing to become a “professional” community.
Generally speaking, those exhortations are a sales pitch – come earn by certificate and be a professional.
Sorry, guys, I don’t buy it. The root word for professional is PROFESS – to say out loud. To stop saying http://www.blogger.com/img/gl.link.gif“testing is what I do” and start saying “a tester is who I am.”
You do not need a certificate on your wall to do that. All you need to do is to *care*.
The next step is to get involved.
Now, not everyone is going to go out in public and speak, and not everyone is going to publish – or even blog. But there are other opportunities.
The fact is, for every guy on stage at a major conference, there are probably two behind the scenes. Printing out ID Badges. Dealing with registration. Putting the website together. For non-profits, the work is generally unpaid, but it has considerable value – you get in the conference free and you get to know other people who share common interests.
Right now, somewhere near you, a conference is calling for volunteers. It might be in West Michigan, it might be PNSQC in Portland, it might be Test2008 in New Delhi, India.
If your company doesn’t have budget to go to a conference, I have very simply advice:
Go Anyway.
