Matt Heusser’s Blog
Testing at the Edge of Chaos
Extreme Programming – In One Page
Now, folks, don’t get me wrong. I am a big fan of Extreme Programming, but I do not think that XP is the “one true way” or the “one right way” to do software development. I do think that it pushed back against the “traditional” school of software development in the right direction, at the right time.
For it’s time, XP was the contrarian consultant, when, where and how it was badly needed.
If you want the elevator speech to explain Extreme Programming, one place to start is the XP In One Page Poster(*).
The poster tries to cram a lot of ideas in a little space. If I had to recommend one single thing that offers the most value – that I would recommend that any commerical or business software team take a long hard look at – it would be the “design philosophy” section at the bottom left.
Seriously. When I saw this today, I printed it out, ran over that section with a yellow highlighter, wrote “READ THIS FIRST!” with an arrow at the top, and put it on my wall o’ attention grabbing stuff. (Mostly cartoons with an occasional big, visible chart …)
Regards,
–heusser
(*) – The poster comes to you thanks to Ron Jeffries, George Dinwidde, and a few other other folks, and dates back to 2002.
Software Fiction
Jon Bruce continues to improve as a writer. He’s got an excellent series going on now called “Startup” – click here, scroll to the bottom of Startup, and read upwards.
In other news …
I am also in the middle of a back-channel discussion with Ben Simo and Shrini Kulkarni about test frameworks.
This feeds off the idea in my earlier blog post that if your framework makes it hard to test, people won’t use it.
Two of the common elements I see in test frameworks are:
(A) Lots of XML
(B) Tough-to-type Syntax
I’ve explained (A) before, but let me talk for a moment about B.
Often, people have a web application. To test it, they may use framework that drives the browser. The tester then writes test ‘code’ in a number of possible languages, often Java (Web Driver), Ruby (Watir), or Visual Basic (Quick Test Pro or WinRunner).
The probem is when these frameworks see everything as an object. Yes, that’s a problem. Because instead of writing:
SendValueToTag(’QTY’,'100′);
PressButton(’Submit’);
my $val = GetValueAtTheTag(’Grand Total’);
ASSERT(’Grand Total should be fifty bucks’,$val, 50.00);
You have to write this:
my $bdy;
$bdy = Object(”Browser”).Tab(1).page(’www.foo.com’).html.body;
$body.form(’form1′).object(’tag’).label(’QTY’).setval(100);
$body.object(’button’).label(’submit’).push;
my $new_bdy;
$new_bdy = object(”Browser”).Tab(1).page(’www.foo.com’).html.body;
my $val;
$val = $new_body.form(’form1′).object(’tag’).label(’Grand Total’).getval();
ASSERT(’Grand Total should be fifty bucks’,$val, 50.00);
Ah!! Ahhh! Ahhh! My eyes! My eyes!
The sad thing is, I am not exaggerating by much.
So at any given shop, one of a few things happen:
(1) Someone tries to use the framework, and goes through so much pain that they give up.
(2) Someone puts an extreme amount of effort into learning the framework and is actually successful. We’ll call him Joe. After that, Joe becomes the in-house expert on the tool. If Joe is assigned to test the software, the software will be tested with that framework. Otherwise, it will probably be tested manually.
(3) You write a custom piece of code that sits on top of the framework that eliminates all the fiddly-bit DOM mappings, so you can just call:
browser.getthetag(’qty’);
browser.setthetag(’foo’, 5);
(4) (Hopefully) You find a better framework.
I have quite a few colleagues who have had success with option three. Ruby/Watir, however, looks a lot like #4 – a non-goofy framework that allow you express complex test cases relatively easily, in a language that looks a lot more like English than other options, so it is self-documenting.
My prediction is that the big, slow, dumb tools will continue to dominate the mediocre “no one got fired by buying IBM” space, and smart people will continue to patch them to make them work.
However, if you want to try something completely different – consider learning Ruby, or least watching the Ruby “Switch” Video.
Strategy Letter VI -
“Writing applications that work in all different browsers is a friggin’ nightmare. There is simply no alternative but to test exhaustively on Firefox, IE6, IE7, Safari, and Opera, and guess what? I don’t have time to test on Opera. Sucks to be Opera. Startup web browsers don’t stand a chance.
What’s going to happen? Well, you can try begging Microsoft and Firefox to be more compatible. Good luck with that. You can follow the p-code/Java model and build a little sandbox on top of the underlying system. But sandboxes are penalty boxes; they’re slow and they suck, which is why Java Applets are dead, dead, dead …”
Taken from Joel Spolsky’s Strategy Letter VI.
It’ll take five minutes of your time and it will be time well spent.
What’s a "Test Framework"?
Shrini Kulkarni has been after me to define my terms; after all, I keep writing about “Test Frameworks” but I’ve never defined the term.
Wikipedia defines a framework as “A basic conceptual structure used to solve a complex issue. It also warns that “This very broad definition has allowed the term to be used as a buzzword.”
When I use the term, I mean any support, infrastructure, tool or “scaffolding” designed to make testing easier, and (often) automated.
For example: Let’s say you have a simple program that converts distance from Miles to Kilometers. The application is a windows application. Every time we make a change, we have a bunch of tests we want to run, yet we can only enter one value at a time, manually. Bummer.
Yet we could think of the software as two systems – the GUI, which “just” passes the data from the keyboard into a low-level function, and “just” prints the answer, and the conversion formula function.
If we could somehow separate these two, and get at the formula function programmatically. Imagine that the formula function is struck in a code library, which can be shared with many different programs.
Then we could write a “tester” program, which takes an input file full of input values and expected results. The “tester” program simply calls the library, compares the actual result to the expected, and prints out a success or failure message.
This is basically how I test a lot of my code, using a program called Test::More. You could call Test::More and it’s friends (Test::Harness, and so on) a “framework.”
We can go dig into the details and test at the developer understanding level, or bubble up and only test things that have meaning to the customer. One popular framework for these higher level, business logic tests is FIT/Fitnesse
Of course, there is more to the application and just the business logic. The GUI could accept the wrong characters (like letters), format the decimals incorrectly, fail to report errors, handle resizing badly, or a half dozen other things. Even with one “framework”, we still have the problem of testing the GUI (not to be forgotten) and testing the two pieces put back together again – “Acceptance” testing, or, perhaps, “System” testing.
This “outer shell” testing can also be slow, painful, and expensive, so there are dozens of free/open or commercial testing frameworks that allow you to ‘drive’ the user interface of windows or a web browser. With the big commercial tools, people often find that they are writing the same repetitive code, over and over again, so they write libraries on top of the tool like Sifl.
The big dog web browser automation systems are Selenium and Watir.
Years ago (back when he was at Microsoft), Harry Robinson once told me that MS typically had two types of testers: Manual Testers, and Developers who like to write frameworks. The problem was that the Developers would write frameworks that no one was interested in using. His assertion (and mine as well) is that people who straddle the middle – who like to test, and like to write software to help them test faster, can be much for effective than people entrenched on either side.
Thus, you don’t set out to write a framework – instead, you write a little code to help make testing easier, then you write it again, then you generalize. Over time, slowly, the framework emerges, like Gold refined through fire.
But that’s just me talking. What do you think? (And, Shrini – did I answer your questions?)
In the mean time …
Here’s a comic that had me laugh out loud, I thought you might enjoy.
The image is clean and appropriate for work(*). It’s a little story about the difference between making a real difference, and, well, hyper-awesomeness marketing.
The cartoon is from BasicInstructions.com, which is a cartoon a day site.
–heusser
(*) – I do not and will not link to things that are inappropriate for work.
Be Careful what you wish for …
I haven’t had much time for blogging lately. A few great, interesting, cool things that are swamping me right now -
- Jim Brosseau’s Book, “Taking Ownership”is completely through the greenlight stage and moving to publication. Yaaay!
- I’m speaking at the Grand Rapid’s Java User’s Group on Tuesday the 18th of September. They literally sent out a call for speaker’s last week, and I responded that I had a couple of lightning talks that I think might string together well for ten or fifteen minutes. So they asked me for an abstract for a full talk … it should be interesting.
- I’m coaching a team of 4 and 5 year old children from Allegan AYSO Soccer. GO ALLEAN TEAM FIVE SILLY FROGS!
- I’m serving as a volunteer and a speaker for GLSEC this year.
More about GLSEC next post, but for the time being, let’s just say I’ve been busy …
