Posted: November 24, 2012 in Testing Lessons

Back in the misty mists of time, I came across something in the BBST Foundations course that struck a chord with me.  It was the following statement – “Karl Popper argued that experiments designed to confirm an expected result are of far less scientific value than experiments designed to disprove the hypothesis that predicts the expectation” (Slide 67, 2010 version of the BBST Foundations slide pack).

“Yes, of course!” I cried (errr, just in my head… because I’m not crazy and prone to shouting “Yes, of course!” aloud for revelations immediately applicable only to myself).  Let us aim to disprove the claims about the software (wherever those claims may come from) and “break” it more effectively (though it could be argued testers don’t break software, but that’s a topic for another day…).  And although I had unknowingly (or perhaps semi-knowingly) skirted this philosophy with my own test efforts, having it explicitly stated was a heady experience.

I presented this revelation to my manager.  “Testing can be done better!  Confirmatory testing doesn’t have to be the only way!”  Well, perhaps a little more cogently than that… Anyway, my manager’s reaction was decidedly luke-warm.  “We need you to make sure the software works – pretty much what you have been doing to this point”, came the reply.  “Sacrilege!” I cried (again, just in my head…).  But when thinking about my high and mighty ideas of how testing should be done, there were a number of points regarding the circumstances that needed to be considered –

  • It’s an application that was not being released for general sale, only internal use
  • As a result, business users use the system in a predictable, “happy-path” way
  • Bugs requiring complex fixes with only minor impact could be documented and passed on to the users to work-around

In this context, confirmatory testing is probably (and to this point, has apparently proven to be) the best approach to testing the application.  The business (and IT) was looking for an answer to the question “Does the application work well and meet the requirements?”.  Now, that statement and the underlying definitions of “work well” and “meet the requirements” could be somewhat loaded.  But to ruin any potential fun, they meant in this case “does the software allow the user to perform the function the business wants with minimum hassle?”.  No (or very few) bells and whistles, no customising for maximal usability, just the functionality in reasonably good order (as ultimately accepted by the business).

Context-driven testing is something I was quick to latch onto as a label and not so quick to roll it around in my head, getting a feel for what it really means. I wanted a better way of testing than “just test cases” (which in themselves have a useful time and place), but wanted that better way to be a one-size-fits-all uber-method.  How easily one can end up right back where one started… all it takes is a little thoughtlessness.

So, having wandered around for several months with insufficient thought as to what “context-driven” meant to me, I finally began to ponder (in true “The Thinker” pose) less about “Context-Driven Testing: The Concept”™ and more on “Context-Driven Testing: How It Applies To Your Work”.  And while the latter phrase might be less catchy, it has proven far more useful in determining how I approach each and every testing project, large and small.  Less templates (convenient as they are), more thinking.  Less “rigid process”, more “the risk for this three-field report is sufficiently low that maybe I don’t have to spend a week planning and designing an amazing suite of tests that is sure to find EVERY POSSIBLE bug”.

And although it was a mildly embarrassing shake-up of some of my dearly-beloved preconceptions, it ultimately proved to be a very valuable learning experience.  I look forward to tracking down the others as they surface and putting them under the searing light of examination.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s