Matt Heusser’s Blog
Testing at the Edge of Chaos
The End of the BlockBuster – III
(Or, how to find your niche)
Did you know that Paul Graham just published an article on this? Or at least, tangential to this?
The list of ideas starts tomorrow, but this I had to link to …
The End of the BlockBuster – II
When Chad Fowler gave his talk, he mentioned that the implied career path for most people in software is to become CIO – perhaps for the ambitious – CEO – of the big company you work at.
He suggested the alternative of becoming CEO of something else. A programming language. A platform. A tool. Become known as a contributor or thought leader in a specific space, and if you live in a reasonably-size city, then at least having a job – even a good one – is pretty much guaranteed. At that point, your options really start to open up – such as consulting, writing, teaching, starting your own company, or just retiring early.
I think this plays well with the end of the blockbuster, because there are now a lot of niche fields within software development – even within software testing. Requirements Engineering.
So how do you find your niche? Chad suggested technologies – either ones so new that the market is small (think ruby in 2003) and the big companies can’t compete, or ones so old that the market is small (cobol, powerbuilder) and the big companies can’t compete. Chad suggested that if you bet on a new technology and it takes off (java in 1996), you could become an “expert”, which is the slingshot to fame and fortune.
Now, that’s a perfectly reasonable strategy, but it’s not mine. It feels … mechanical to me, and I’m not in love with technologies. So I suggest a different route:
If you find that you enjoy a particular subject, that you’ve read all the books on the subject, and, at lunch, invariably, you are the local expert – then you’re already on the path.
Don’t find a technology – find a passion. When you’ve read all the books and find insights between them; when you disagree with some of the authors … when you feel you have something to contribute … when you find yourself in a tiny little community of people who really care about the subject … suddenly you’ve found a niche.
Once you’ve found a niche, the next question is how to gain “recognition” for expertise in your niche.
On it’s face, that’s pretty selfish.
The good news is that in order to be recognized for being good, like Linus Torvalds or that other guy, you have to contribute – you have to give something away.
Sure, you can be recognized for having written a book or five, or having a PhD, or a bunch of other things, but I submit that those kinds of things (”marketing”) are separate and distinct from being good. They may coincide; they often do, but there are a lot of people who want to be famous in the world of software testing, and a few who want to do good work and be recognized.
If you’ve ever read a book that was bone-dry, full of platitude, obvious, and included a bit of impressive hand-waving pseudo-science, it was probably written by someone who wanted to have written a book, instead of wanted to contribute to the community.
So, you’ve found your niche, but you aren’t a developer who’s niche is open source, and you don’t want to be a self-appointed blowhard. How do you contribute?
I’ve got a bunch of ideas, but they can wait until tomorrow. In the mean time, what do you think?
–heusser
(*) – Jerry Weinberg, “Rethinking System Analysis and Design”, and Eli Goldratt, “The Goal”
I’m Unprofessional …
(A Rant)
Really, I am unprofessional. I’ve been told by friends and people I respect that my website is terrible. Unprofessional. Ugly. How dare I have a website without CSS?
None of those people have a website
My blog is hosted at blogspot. Gosh. Awful. I should have my own viritual host (I do) and manage my own subdomain under mod_perl so that I could have blog.xndev.com. After all, that’s the right way to do it.
None of those people have a blog
I’m doing things in the wrong order. First, I need to write a book, then I need to do public speaking. Why, by then, the companies will come to you, Matt! Stop wasting time on your blog; nobody reads it anyway.
None of those people have written a book, or presented at a major conference
What’s going on here?
Two hundred years ago, Voltaire said “The Perfect is the Enemy of the Good”
Twenty Years ago, Richard Gabriel wrote “Worse is Better“
Right about now, Matt Heusser is writing “Good enough is the enemy of done”
Or, to put it another way - I value working software over comprehensive design; I like to deliver working software often, and improve it incrementally.
////—-Rant off
I do intend to improve my website. Eventually, I would like to get Creative Chaos hosted somewhere else. The fundamental difference is one of approach: Instead of a Diamond-Like Jewel – that is never finished – I intend to release systems that are good enough and improve them incrementally.
If you are not convinced, that’s fine. To each his own, but I wanted to get this down. Still, think about this: When the 37Signals guys released the first version of basecamp, the radically successful web-based project management tool, they didn’t have any billing system in place.
They didn’t care, because the only thing you could sign up for was a 30-day trial membership, which was free.
They figured that gave them 30 days to write the billing and payment system.
How … unprofessional.
Wait, aren’t they like, multi-millionaires now?
The End of the BlockBuster – I
Remember back when everyone saw Batman
I mean everyone. Science Fiction Fans watched it; thrill seekers and horror fans watched it (it had a very dark “joker”); kids who played with action figures watched it – though they probably should not have. Mom watched it (romance and drama), Dad watched it, teenagers watched it – Batman had mass commercial appeal.
Batman came out in 1989 – just before the onrush of the Internet and the influx of cable TV stations – back when, for the most part, MTV played music videos.
Back then, people simply had less options. Today, thanks to the Internet, cable TV, and satellite, each and every niche can find exactly what they want, and not have to settle for “good enough.”
That means market segmentation – instead of going for everyone, the disney channel can aim for children aged 4-15, and be very profitable; and NickJr can under-cut them for the 2-6 market.
At the same time, as people demand better and better special effects, blockbusters will cost more and more to produce – but return less and less in results. Sci-Fi fans will watch the sci-fi channel; drama fans have the soap opera channel; thriller fans will watch the thriller channel – and the list goes on.
This means there are less opportunities for a $100 million opening weekend (as GodZilla proved in 1999 and Casino Royale is proving right now), but there are more opportunities for the tiny niche provider.
Then again, less money is just fine for a 20-person cable channel.
While I don’t know the cost of a cable channel, I do know the cost of a blog, and I know that there are plenty of people who are self-employed thanks to blogging, and a few who are self-employed pod casters. (But most of those have mass appeal, such as credit card sites or reviews of digital cameras. Bummer for me, eh?)
What this means for the world of software testing is that the day of the 900-pound gorilla “we provide everything” software testing company is probably going to go away. The big body shops will continue to want to place 20-people on-site for an 18-month contract when we just have one interesting problem to solve, and the big tool providers will continue to make big testing suites when we only want to do one thing and do it well.
For the small company, this is the smell of opportunity.
I haven’t seen many product companies adopt this model — yet. Last year at STAREast I met David Gilbert, a software tester who owns a tiny little company called Sirius-SQA and makes a product that enables exploratory testing called TestExplorer.
He is trying to do one thing and do it well, and I wish him the best.
I have seen some individual consultants adopt this approach – Scott and Mark, the performance testing guys, Johanna Rothman does software management, Ron Jeffries does agile coaching.
Each of these niches is so small that the 900-pound gorilla companies can’t compete – it just is not worth the time and investment to develop expertise in a market for three people when you are trying to place three thousand.
Even people in the Ruby/Rails community are talking about this strategy; last night I heard Chad Fowler recommend it at XPWestMichigan. Chad wrote “My Job Went to India” and, more recently “Rails Recipes.” He’s also a pretty smart guy; as soon as the video is up on the web, I’ll link to it here.
More on this tomorrow …
Testing Computer Software, 3rd Ed
Dr. Cem Kaner has started work on the 3rd edition of his popular sofware testing book by posting a short article here.
If you’d like a short introduction to some deeper issues in software testing, you could read my blog for six months, or, well, check out his post. Seriously, it’s good.
A couple of my favorite quotes:
Testers should not try to design all tests for reuse as regression tests. After they’ve been run a few times, a regression suite’s tests have one thing in common: the program has passed them all. In terms of information value, they might have offered new data and insights long ago, but now they’re just a bunch of tired old tests in a convenient-to-reuse heap. Sometimes (think of build verification testing), it’s useful to have a cheap heap of reusable tests. But we need other tests that help us understand the design, assess the implications of a weakness, or explore an issue by machine that would be much harder to explore by hand. These often provide their value the first time they are run—reusability is irrelevant and should not influence the design or decision to develop these tests.
Forcing people to do make a tests regression-runnable, is, in my experience, often a thinly-vield excuse for the cult of “document everything.” At the same time, if you can decrease the cost of documentation, you can run more testss – and more tests means better software – and the best CYA is to not have the bug ship to the customers. For years, I have kept hearing things like “If you don’t document it, it didn’t happen”; my typical reply is “If you do document it, and stick it in a drawer, you just wasted your time.”
Dr. Kaner also wrote:
The focus of system testing should shift to reflect the strengths of programmers’ tests. <!–span>Many testing books (including TCS 2) treat domain testing (boundary / equivalence analysis) as the primary system testing technique. To the extent that it teaches us to do risk-optimized stratified sampling whenever we deal with a large space of tests, domain testing offers powerful guidance. But the specific technique—checking single variables and combinations at their edge values—is often handled well in unit and low-level integration tests. These are much more efficient than system tests. If the programmers are actually testing this way, then system testers should focus on other risks and other techniques. When other people are doing an honest and serious job of testing in their way, a system test group so jealous of its independence that it refuses to consider what has been done by others is bound to waste time repeating simple tests and thereby miss opportunities to try more complex tests focused on harder-to-assess risks.
We allmost got into this a few weeks ago in Indiana, but I didn’t take the bait. I probably should have; we could have learned something from each other. I would put it slightly differently:
If your developers are doing automated tests, and you find that a test technique (such as bounds testing) isn’t finding any bugs, it’s because the devs are covering it. So you should probably shift your focus away from a technique that isn’t yielding results to focus on things that provide a better return. Maybe not entirely, but a shift is called for.
Come to think of it, if you are using any test technique and not getting results, it’s probably because the software seems to work in that way, so try something else.
Fifteen years ago, as a cadet in the Civil Air Patrol, I sat through a class where they explained this principle. In a missing aircraft search when you have a rader hit, the probability of discovery decreases the further you get from the rader hit. The Mission Coordinator can calculate the probability of discovery (POD), and when you’ve searched the close areas enough that the POD is up, you start to believe that the plane isn’t close, and so you move the search parties further out.
The application to testing is an exercise for the reader. 
In the mean time, check out Cem’s post … more later.
Presentations – III
For some time I’ve been trying to put together a post about the Blue Man Group – the seven-city theatre production that is, well … unique. I recently saw Blue Man in Chicago, and it is an amazing combination of theatre, art, audience interaction, and improvisational comedy that creates a shared experience.
And yes, I know that’s a lame explanation. I still haven’t told you what the Blue Man Group does, or, the obvious reason to post – how that should or could affect the way we do presentations on software development.
Here’s the first obvious part – Blue Man combines watching with doing – or at least, they create the illusion of it. It makes the entire experience more memorable, and, as I see it, a big part of the problem with the “drive by training” we do in software today is that it is entirely disconnected from what we do on Monday. Disconnected from who we are. That might be a good way to make money, but it’s a lousy way to help an industry improve.
So, here’s the second thing – Blue Man Group is going to be on SCRUBS on Thursday, November 30th. So before I bother to try to explain what they do, check out the TV show. If the TV show doesn’t reflect the group, we’ll there are a few blue man links available on video.
Also, I think the post on the requirements problem is pretty good, so if you haven’t read it, please, scroll down …
The Requirements Problem
Software Requirements are hard.
Ok, let’s do some critical thinking on that. Why are requirements hard?
In my honest opinion, the skill set to do requirements is a combination of writing skills, an understanding of the problem domain, and an understanding of technology. To paraphrase Jerry Weinberg, it’s not that you have to analyze requirements (break into component pieces) – it’s that you have to synthesize them – get them to play nice together.
I would like to talk about that – and do it in an interesting way.
So here is the first Execelon Development PodCast (10MB) and also a handout to help follow along.
Since most of the success literature approaches requirements from a customer viewpoint, the podcast talks about requirements from a developer’s viewpoint.
The fact that different people use the same document for different purposes is an entirely different problem; perhaps that’s a follow-up.
Agile "Architecture"
I’m not a huge fan of the concept of software architecture. All too often, I find architecture is an excuse to have highly compensated people who don’t produce working software. Heck, I even published an article about it.
Still, All too often is not the same as “always.”
Scott Ambler has an interesting article on architecture in this month’s Agile Journal. His take seems to be that if your team is big enough then you must do some architecture. In that case, the challenge is to keep the architecture light and grounded in reality.
Good stuff.
Metrics Madness – II
UPDATE: Mark Waite is quick to point out this article by Cem Kaner on Metrics Dysfunction, which predates Joel by years. The style of the two articles is very different; Joel uses a little bit of logic, a little bit of generalization, some common sense and emotion to make his point, where Dr. Kaner wrote a legal brief.
Of course, Dr. Kaner is a lawyer.
The great thing about the article (which I have printed off to explore in depth) is that it is comprehensive. Perhaps when the issues comes up again, I can find a way to politely ask “Have you read the Kaner paper on the subject?” Of course, he has published others.
This brings up an interesting question: So far, I’ve been listing interesting resources here as I find them. It would be neat to have some sort of categorization scheme to make them available quickly; something like Brett Pettichord’s Software Testing Hotlist.
Metrics Madness
For years I have contemplated writing an article on the problems with Metrics. It seems that Joel Spolsky just wrote it for me.
Now, again, don’t get me wrong(*) – Metrics can be very helpful, but they need to be explained in context. For example, let’s say that the data points to less and less defects found per week, or even per day. Why, that’s a “converging trend line on defects open” – the product is ready to ship!
Or, it could just be the month of December, when everybody took vacation.
Or maybe a tester quit, and the lead and supervisor are spending a lot of time interviewing and no time testing – without context, the uninformed reader begins to make up explanations for the behavior of the data. That can be very, very bad.
What bugs me the most about metrics, and, well, I’ll be brutally honest here – is the purpose they seem to serve.
As they are presented in the textbooks (and I have read a lot of them), Metrics make things easier. After all, once we define “Good” and put performance metrics in the way, then all the decision maker has to do is breathe easy (when the numbers keep going higher) or make a stink (when they don’t.)
Now, read that paragraph again. I submit that in some cases, it’s not really about making the job easier – it’s about creating a situation where people don’t have to think. After all, they can just manage to the numbers.
When I think of the context of software development, everytime I can think of a situation where someone was clinging to an idea because the alternative was scary and involved thinking for yourself, it has gone bad. CMM, UML, RUP, CASE, record/playback testing, Agile … take your pick – when the motivation was to solve all our problems and avoid thought …
Well, people got what they deserved.
Doesn’t anybody read Fred Brooks anymore?
–heusser
(*) – (the 3 sentances = 1000 words principle kicks in)
