Software Test & Performance Collaborative

How would you test this? – Take II

By Matthew Heusser on October 1, 2009 | 9 comments.

We’ve had a few responses to the earlier test challenge – lots of good strategy tips, but besides Justin Rohrman’s blog post, I haven’t seen any test plans.

So yes, am going to reveal my test strategy – but first – I’d like to up the ante. I’ll be giving away a $50 gift card – Prepaid MasterCard, PayPal, or Visa – take your pick – to the best test plan developed. And yes, besides my own judgement, I do have a little bit of objectivity; I’ll be reviewing the test plans against the actual bugs filed to see which would be most effective.

No, I’m not a huge fan of exhaustively detailed “test plans”, with one “test case” per page. At Socialtext, a “test case” is more like a row in a spreadsheet. We use the “plan” like a checklist; I’ll test the actual feature intelligently, as a human being, then come back later to review the plan to see if I missed anything important.

So for this exercise, Justin’s format would work – if that’s the way you’d prefer to document.

Now, about how to develop those test cases. I was a little surprised that the standard answer for strategy was “ask the customer what to test, then test it.” It’s a rational strategy, and, if you aren’t a subject matter expert, perhaps it’s necessary – yet it doesn’t put the tester in a role to add a lot of value, does it?

At Socialtext, the whole technical staff are heavy users; we are our own best customer. Product owners do provide examples, which are reviewed by developers, which help with communication and help test the happy path. Here are a few of the actual tests for the Activities Widget:

So what follows below is what the product management team came up with. My question is what you, as the tester, would come up with – the plan, the strategy, and your defense of the plan. (I’ll give extra credit for plans that can be expressed concisely).

And I’ll give $50 US to the best answer, $25 to the second best. I’ll put up my answer next Tuesday, but before that – do you have any insight?

HOW TO ENTER: I would prefer discussion in the comments, but I realize some answers could get very long. So – you can put you answer in a comment, you can create a blog post and link to it in a comment, or email me at matt.heusser@gmail.com.

DELIVERABLES: There are two deliverables; the strategy is how you would manage risk – how much time you would put into testing this iteration, and then every one after that – what you might have run automatically, how you would set appropriate expectations, and so on. The test plan is a series of things you would explore, and the order you’d explore them, to determine if the product is fit for use.

Here’s what the product owners came up with:

activities widget shows grammatical representation of filter
… follows the form: “Showing all of the from within ”
… defaults to: events: “events” (all), feed: “all activity”, network: “all my networks”
… user can choose different components for the filter; feed is updated on each change of choice
… if no events match the configured set of selectors, a “no events match this filter” message is shown
… RSS chicklet is shown; URL for feed is based upon configured selectors
user mouses over selector; list of options is shown
… user clicks one of the options; the option is selected
activities widget contains an expandable/collapsible Signals row (as mocked up)
footer contains pagination links: “newer”, “newest” and “older”
… user clicks on “newer”; more recent page of events is shown
… if there are no more “newer” events, “newer” link is disabled
… user clicks on “newest”; most recent page of events is shown
… if there are no more “newer” events, “newest” link is disabled
… user clicks on “older”; older page of events is shown
… if there are not more “older” events, “older” link is disabled
activity list follows the format already defined in the signals & conversation widgets
… signals are displayed like in the “signals” widget, not like in any of the other event widgets
… all event items the “X ago” line in a single line (like SD Activity tab, unlike Recent Conversations)

… events selector …
events selector contains: “events” (all), “signals”, “contributions”, “edits”, “comments”, “tags”, “people events”
… each choice shows only those events within the parameters of the two other selectors (feed, network)
user chooses “events”; feed is updated to show all recorded events (except views)
user chooses “signals”; feed is updated to show only signals
user chooses “contributions”; feed is updated to show only non-signal contributions
user chooses “edits”; feed is updated to show only edits
user chooses “comments”; feed is updated to show only comments
user chooses “tags”; feed is updated to show only tags on pages
user chooses “people events”; feed is updated to show only person tags and person follow/unfollows
invalid selector combinations are grayed out and unselectable
…people tag/follow events with the “my conversations” feed selector is invalid

… feed selector …
feed selector contains: “everyone”, “people I follow”, “my conversations”
… each choice shows only those events within the parameters of the two other selectors (events, network)
user chooses “all activity”; feed is updated to show all activity
user chooses “people I follow”; feed is updated to show activity from the people the user follows
user chooses “my conversations”; feed is updated to show activity from (pages the user watches) + (pages the user has edited, tagged or commented on)
… cannot be combined with any events selector that excludes page events (e.g. the “signals” selector)
incompatible selections are greyed out

… network selector …
network selector is hidden when the user belongs to only one network
network selector contains: “all my networks”, … each choice shows only those events within the parameters of the two other selectors (events, feed)
user chooses “all my networks”; feed is updated to show activity within all of the networks the user belongs to
user chooses a specific network; feed is updated to show activity only within the chosen network

… signals interaction …
when signals component is closed, expand icon is shown
when signals component is open, collapse icon is shown
user clicks expand icon; signals component is opened
…the height of the widget increases to accommodate
user clicks collapse icon; signals component is closed
…the height of the widget decreases to accommodate
if signals component is closed and user clicks on individual signals event action (reply, reply privately), then signals component is opened and displays @user in body
signals network selector is updated based on the feed selector choice
feed selector choice is not updated based on signals network selector
haha, good one. Don’t forget the Signals page tests as well.

… widget gallery …
new users have activity widget instead of the Recent Conversations & Signals widgets
…it’s placed in the middle column by default
widget gallery and manage widget gallery contains this widget
widget gallery and manage widget gallery does not contain Signals nor Recent Conversations widgets

… automatic updating …
items in list refresh every 30 seconds
… # of items remains constant with new items coming in (pushing extra items to the “next page”)