Matt Heusser’s Blog
Testing at the Edge of Chaos
On Creation
In this bit of a lull between Christmas and New Years, I’d like to introduce something a little more lighthearted, a little further off the beaten path than my typical blog post.
I would like to talk about creation.
No, not just creativity, not just “creative work” – but Creation – the actual act of making something new.
If you were like me, you had a moment, when you were young; maybe six years old, maybe ten where you sat at the keyboard and typed something like this:
10 Print “Matt is Cool”
20 Goto 10RUN
That time was powerful. It was magical. Without any spec, or any defined process, or
any requirements, we created something new.
If I had to explain that kind of joy myself, I would say that I believe in a creator – the author of the universe. Thus, in a way, when we create, we are sort of ‘tapping into’ the divine – or, perhaps, displaying an attribute of his character. Now that’s just my way of explaining the feeling; there are others. For some reason I expect that even the strident atheist, looking at the 255 specific strings of protein that have to connect in order to form life would say that moment was a kind of magic, and that, in a way, when we create things, we have some shadow of that magic.
… time passes …
Somehow, I end up stuck in a cubicle, given a requirements document (or a test plan) and told that my job is translation – doing stable, predictable, repeatable work.
For some reason, it’s not fun anymore.
Where did the creation go?
No, don’t blame the customer. Don’t blame the boss. As an industry, we trained it out of our own work. We, technologists, are our own worst enemy.
“Creative Work” and “Creative Types”
Let me explain. Consider other fields: Writing, Graphic Design, Advertising Media. In those fields, the deliverable matters – not the process. With creative work, it is expected that the customer can give an extremely vague problem to the freelancer, and the ‘creative people’ will figure it out. The assignment might be “We need our website to look more edgy” or “I need a one-thousand word article on Enron and Integrity”, or perhaps even “We need a series of 30-second ads to make our product appeal to women.”
No, really, they get requirements like that. And there is an entire set of literature about how to be successful in the face of ambiguity – what questions to ask to guide the customer, how to do prototypes and mockups and get feedback.
Nobody talks about ‘process improvement’ for freelancers; instead, editors and agents hire them and pay a flat rate. The incentive for the freelancer is clear: Get the work done faster and you get do more work. Do it better and you’ll get repeat work or you can raise your rates. When they do talk about ‘performance improvement’, it has an entirely different flavor; consider Merlin Mann’s work on time and attention -
Now compare that to the knee-jerk software development reaction. We wanted to make software an ‘engineering discipline’ and get specific, actionable requirements and defined process.
BZZT.
That’s where creativity went in software development. We killed it.
Now, yes, some of us are writing software the drive tanks and automobiles and embedded medical devices, with a slightly lower tolerance for failure than an editorial in the New Yorker. On some level, we will have requirements and we will need process. The question is: How much?
My friend, Markus Gaertner, points out that defined process and creativity are tradeoffs; you can’t be creative if your process tells you exactly what to do at every step now can you? All I am suggesting is that we push the pendulum back the other way, in the direction of choice over defined process..
I see some of that in the recent Software Craftmanship movement. I’ve tried to contribute a bit through this blog and other outlets like Boutique Testing. The general idea is that restoring choice to the workplace can help redeem it.
My Resolution
So here’s a New Year resolution: In the Year to come, I resolve to try to move the needle, just a little bit, to restoring choice and creation to the corporate workplace. I intend to do it in two very specific ways: First, by proposing a book on the life of a good tester that de-mystifies our craft in a meaningful way. In addition, I will work to develop an actually good model for performance improvement for a software team. Yes folks, I resolve to work on an alternative to the CMM that is well-written, straightforward, and actually values human beings as people. While I am skeptical of the whole thing, and may end up recoiling in horror, I do have a few ideas. I resolve to give it my best shot.
All you have to do is one specific thing to move the needle. It may be hard, you may not be rewarded for it, but it’s the right thing to do. Will you join me in this revolution – er, um, resolution?
It’s time for 2010, and a Happy New Year, Indeed.
Let’s make the most of it.
Technorati Claim Code
Hey, Technorati, I’m a real person that actually owns the right to this blog!
The code you are looking for is – E79NX7TBFMFE
(Technorati is a blog listing service.)
UPDATE: I thought I’d be able to put this up, click “verify”, then take it down in a minute or two. Looks like it may be up for a little longer than that. Sorry about that. (I snuck the claim code into an earlier post and the Technorati folks didn’t seem to pick up on it, so I made it it’s own post. Yay?)
Testers at Work
Several years ago, after a conference in Indianapolis, I was sitting at dinner with Mike Kelly and Karen Johnson when Karen pointed out that she was surprised how little her boss(es) understood what she actually did all day.
Since then, I’ve been listening to that story, and I hear it over and over again. Not just our bosses, but our parents, our grandparents, our spouses. Moreover, testing runs into a dilemma of competence, where people who are ignorant about testing are so ignorant that they do not realize it is a complex skill at all. In the words of John Bruce:
There’s one way I’ve found — though certainly not the only way — that you can tell a job has gone south. It’s when the work suddenly focuses on its most rudimentary component. I’d been writing, coordinating, and publishing all the documentation at the USC computer center, a responsible position. Within a very short period I’d become nothing but a typist. Later, in other jobs, I’d find the work had suddenly changed from being a system engineer to being the kid who pushes a cart around, replacing coffee-sodden keyboards and used-up printer cartridges. That’s a sign that it’s time to go, or if you don’t go of your own accord, someone will make the decision for you. When this happens, the window of opportunity isn’t wide.
When people don’t understand help desk work, it becomes call-forwarding.
When they don’t understand project management, it becomes list-keeping and status-reporting.
When they don’t understand testing, it becomes button-pushing.
I am less than excited about this; in fact, I’d like to start a trend the other way – recognition that testing is complex, challenging work, plus a desire to do it better.
I’ve got an idea.
Three years ago, a journalist named Jessica Livingston interviewed founders of high-tech startups – people like Steve Wozniak, co-found of Apple Computer, and Dan Bricklin, creator of the spreadsheet. She put those interviews down in a book called Founders at work.
The book described what the company founders actually did, not in generic, general terms but in specific; what life was like, what decisions they made and why they made them.
Then last year, the next book in the series came out – Coders at Work.
Of the books in the world, do we have room – and do we have time – for Testers at Work? At book that, instead of trying to provide a generic test strategy for everyone, starts out by interviewing successful testers, finding out what was unique or different for them, providing some practical experience, real examples, and reference information?
If I wrote it, would you be interested in it? Who would you like to see interviewed?
The Steel Cage Knife Fight is Live!
Last week I had a deathmatch or, um, “conversation” with Alan Page of Microsoft about code coverage metrics. You can read it on-line here for free. No registration is required to read it, only to leave a comment – and registration is free.
How are you going to *test* that?
A friend of mine is finishing up an article on testing and aesthetics; I thought it was really good, and only had one little point of concern. It wasn’t with the main theme of the article, but more with the theme of my work; I thought it was worth bringing up here. Here’s the short quote:
Test automation checks for this sort of error very well, and today is relatively easy to put in place.
Personally, I’m leaning more and more toward *not* calling it test automation, but instead “automated evaluation” or “automated execution.” Why? Because with our current technology, what we call “test automation” isn’t. In other words, having a computer click-click and inspects is not the same thing as what a thinking human being does when they sit down to test software. (That is at the application, or GUI level. I’m much more okay with calling unit tests, developer-tests, and true TDD “test automation.”)
And no, I do not think that is splitting hairs – there is a whole movement of folks who believe the two are interchangable, and that language has bled into our literature and our conversations. Just about every time I meet a new, sharp team of test infected developers, I have a conversation like this:
Dev: “But … there’s no way to test that!”
Matt: “Sure there is. Check this out – (mouse mouse type type examine) – see?”
Dev: *mystified*
Matt: “I suspect that by ‘test’ you mean computer automated execution and evaluation, right?”
Dev: “Well, yes.”
Matt: “I don’t need that to ship software. It helps, but we don’t need it, and it certain isn’t sufficient on it’s own/ It’s a question of ROI, right?”
—-> I don’t particularly enjoy that conversation. Yet I’m as guilty as the next guy of using the term “test automation” in a sort of naive way, as if it were simply taking the work a human does and making it faster. So I tell you what — I’m going to make a conscious effort to write in a manner to make those conversations happen less often, because the articles those devs are reading will make the distinction more clear.
Will you join me?
What do you want to hear?
I’m putting together a talk for STAREast 2010. This is the abstract:
A Test Odyssey: Building a High Performance, Distributed Team
It seemed simple enough—hire the best available technical staff that would work from home to build some great software. Along the way, the team encountered the usual problems: time zone differences, communication headaches, and a surprising regression test monster. Matt Heusser describes how Socialtext built their high-performance development and test team, got the right people on the bus, built a culture of “assume good intent and then just do it,” created the infrastructure to enable remote work, and employed a lightweight yet accountable process. Of course, the story has the impossible deadlines, conflicting expectations, unclear roles, and everything you’d get in many development projects. Matt shares how the team cut through the noise, including building a test framework integrated into the product, to achieve their product and quality aims. Take away a list of technologies that make remote work possible, cultural ideas to make it effective, and some things to try on Monday—plus, some apparently good ideas that you definitely want to avoid!”
So here’s my question. Say you see that abstract in a conference brochure, and you decide to attend. Can you think of one question – maybe two – maybe three – that you are expecting to see answered in this talk? These are key questions; the ones that, if you did not see them answered, you would leave the talk feeling ripped off?
I’d like to hear from you. Will you help me make a better talk?
Thanks.
Code Coverage Steel Cage Knife Fight!
The November Software Test&Performance Magazine had a little article on Code Coverage Metrics by myself and Chris McMahon. Our conclusion was highly skeptical of code coverage as a tool for testers; Alan Page took that and started a … dialogue with me over twitter.
Some things are just hard to discuss in 140 characters or less. We took our discussion offline, and we’ll be publishing the results as an interview for a weekly STPCollaborative newsletter called “Test&QA Report.”. The interview will appear on Tuesday, December 22nd. We’ve asked Marlena Compton to referee … I mean, um do the introduction. It’s going to be awesome.
And you can be certain to get it in your in-box by creating an account with STPCollaborative and changing your settings (this will get you the download of the November Magazine too, and it’s free).
UPDATE: If registering is too intrusive, you can sign up for just the report by email here.
The people side of Agile
I’ve recently started to write and speak directly about Agile Development. Oh, yes, I’ve always been interested in lightweight methods, in valuing people, in feedback-driven work, tight iterations, and technical excellence.
By “writing about Agile”, I mean capital-A, Branded Agile. Nine years after The Agile Manifesto was penned, we have much more experience and more public examples. This will allow us to get past dogma to talk about what works, and, for that, I think it’s a good time to get involved in the discussion.
So I started a short series for SearchSoftwareQuality. Four articles into the series, I realized that something was … not quite right – so I wrote this reply on The People Side of Agile Software.
I hope you find it valuable. You can rate it on the SSQ, but there’s no room for comments, so please, feel free to leave comments here.
On Virtue
It seems that lately the testingsphere has been all atwitter about standards of behavior and virtue. That is to say, that at the same time, we’ve got people talking about plagarism, inappropriate behavior, kindness and niceness.
But, waitaminute. If I hold ‘nice’ as a universal standard, what do I do when someone behaves in a way I find unacceptable? To call them on it seems to be well … not nice.
And so we’re stuck. Especially if the item is a threat to good work, and claims to be a standard or certification that has universal appeal. Then we’ve really got a problem, because to say nothing is it’s own sort of moral problem.
Obviously, I’m still thinking through how I feel about that.
When I was a cadet in Civil Air Patrol, we memorized, repeated, and were expected to live by the honor code of the US Air Force Academy at Colorado Springs:
“I shall not lie, cheat, nor steal, nor tolerate those among us who do. Furthermore, I resolve to do my duty and live honorably, so help me G*d”
I like those words. I want them to apply to any community of practice I participate in.
It occurs to me that throwing the bums out – being intolerant – is part of that honor code. That might not be perceived as being ‘nice’ as described in Lisa Crispin’s post. (Catherine Powell, who wrote ‘be nice’, above, points out to me that you can give criticism kindly. Can a judge in a courtroom do that? Should he?)
I’m looking for a different word, one that captures the meaning of what Lisa is getting at, but allows us to be ‘intolerant’ when called upon. In that, I am struggling.
Personally, the closest thing I have found so far are the four cardinal virtues. Put in my words:
Justice is treating others the way they deserve to be treated. This is different than fairness, which is treating everyone the same. If I give all the students in my database class a ‘B’; that would be fair. But it wouldn’t be just. Justice also means balancing our own self interest with the rights and needs of others.
Prudence means knowing what actions are right at a given time. That is to say, there are some things that might be just fine to do (borrowing an idea in order to test faster), but not in all situations (presenting it as my original work at a conference).
Temperance is generally associated with alcohol, and the ability to say “no thanks” or “i’ve had enough.” Stopping short with anything (too much food, too much sleep) due to some internal self-will demonstrates temperance.
Fortitude is simply courage. That means, you might be scared to do something, but you do it anyway. It might be standing your ground on the field of battle, or it might just be telling the truth when you are asked a tough question – or volunteer the truth when it is inconvenient to do so. Sometimes, it’s just being the first one to admit that the team won’t make the date.
So, while kindness and niceness are often prudent, I wouldn’t say they are always called for. Sometimes, the demands of justice require us to speak out about a problem – and the ones who do speak are the ones with the fortitude to do it.
Personally, I struggle most with temperance. I find it hard to deny myself a cup of coffee, and I’m fighting a constant battle with my waistline. (I seem to be holding my own, so to speak, which isn’t too terrible.) I find it far to easy to commit to one more magazine article, one more conference presentation, teaching one more college course. I’m over-booked, and that’s not good.
Quite honestly, this is not fair (or Just) to my family and my young children.
And, very painfully, I’ve decided to exercise some temperance. I have not requested renew of my teaching contract for the 2010 academic year. To be specific, I have requested non-renewal. This was a very painful decision for me, but if I want to continue my writing and speaking schedule, and treat my primary employer and family as they deserve, I think it was the right one.
Which leaves me with two parting thoughts. First, I’m interested in your thoughts of virtue – what moral and ethical forces guide your work. (Yes, we can do a post on metrics. Or twelve.) Secnd, I don’t want to be sad about leaving Calvin – I’d like to celebrate the work.
So let’s do that! Next year’s Conference for the the Association for Software testing will be right here on-campus, August 2nd-5th, 2010.
And you heard it first right here! Mark your calendars and start making plans; it’s going to be awesomez.
… nor tolerate those among us who do
The World Agile Qualification Board has, for a second time, plagiarized the work of Janet Gregory and Lisa Crispin by cutting and pasting the outline for their book and using it as a course outline – without permission or attribution.
Earlier this week, the folks at the Project Management Association of Canada took statements of goodwill and good luck, out of context, and claimed they were endorsements for their Agile PM certificate.
At this point, I feel obligated to mention that I belong to a community of practice known as context-driven testing, and when we see this type of behavior, we forcefully remove and dissociate ourselves with these fools and charlatans. Like radiation, the stuff is bad, and you don’t want that rep on you.
I am requesting that the Agile Alliance formally censure these organizations. If you agree, please leave a comment to that effect, retweet, link, etc. Thank you.
UPDATE 12/15/2009: Laurent Bossavit, a board member of the Agile Alliance, just wrote the following tweets:
I confirm WAQB not a member – filled in our form, asked to be invoiced, Agile Alliance never received payment; record deleted now. / They appeared on our corporate member page temporarily, because our system assumes good faith (for 90 days).
In other words, they didn’t pay their Agile Alliance membership dues, either.
