Test is Dead They Say, Long Live Test


I first came to know Alberto Savoia's work back in 2008 when I was at Microsoft and collaborating with Scott Barber on his book “Performance Testing Guidance for Web Applications.” Alberto had just published a paper which demystified the definition of “concurrent users” in load testing terminology, a paper I use as a reference whenever I am talking about concurrency.
   
More recently, Alberto has gained notoriety with his  "test is dead" keynote speech at GTAC 2011, a presentation which led to some incorrect understandings of his remarks. Let's face it, today's business language and Zen presentation style can be filled with lots of loaded language and codified speech that can leave your head spinning.
   
For Alberto’s sake, let’s try and break it down.
    
- Alberto gave this presentation as the motivational keynote speaker at a Google software testing automation conference, so we can assume he's talking about software testing, and also that his remarks are intentionally thought provoking and self-promoting.  In both his presentation and his blog, this opinion is conveyed:
"Alberto Savoia suggests that software testing is dead."
- Alberto is loosely borrowing from Nietzsche here, so his perspective (as he eventually explains) is not to be taken literally. Instead this is a philosophical statement which he encourages testers to take seriously. That brings our understanding of the statement above to:
"Alberto Savoia suggests (philosophically) that software testing is dead."
- When we jump forward to the first question after his presentation, where Alberto is respectfully asked whether his philosophical statement applies equally to testing "enterprise grade software" (mission-critical) and also "artificial hearts" (life-critical). Alberto's response is that his “New Testmentality” doesn't necessarily apply to traditional testing practices at companies “like Boeing or Pfizer (life-critical) testing...especially like testing an artificial heart.” So now an understanding of the statement might be:
"Alberto Savoia suggests (philosophically) that traditional software testing is dead, except for life-critical or mission-critical software testing practices."
- Now, consider the context for the location and setting of this presentation: a Google conference on software testing where Alberto responds to his first question from his experience working at start-ups. "If you want to work at start-up" he says, "you need to consider this [new] approach." Now he has identified  the intended target audience for his statements about testing:
"Alberto Savoia suggests (philosophically) that start-ups should consider that traditional software testing is dead, except for life-critical or mission-critical software testing practices."
- Alberto uses the word "traditional" in his explanations which we should consider code language. If you are an entrepreneur in Silicon Valley, you spend all your time being hell-bent on innovating or inventing "the NEXT BIG THING." During the VC pitches that I've witnessed, many presenters will get dinged on their use of the word "traditional" because it is seen as a weak competitive reference to the "dominantly successful products or practices" in the marketplace. So, that changes the context for Alberto's statement:
"Alberto Savoia suggests (philosophically) that innovative start-ups should consider that the dominantly successful products and practices for software testing are dead, except for life-critical or mission-critical software testing practices."
- Also, let's not forget that Alberto is pitching his own practice or approach, called pretotyping. So the further-demystified statement should now read:
"Alberto Savoia suggests (philosophically) that innovative start-ups should consider that the dominantly successful products and practices for software testing are dead, except for life-critical or mission-critical software testing practices and they should adopt the pretotyping approach as an alternative."
- Let's dig a little deeper into pretotyping, because it's the essence of what Alberto is suggesting here.  Pretotyping is the "it" that Alberto is building. The definition for pretotyping from his website is "verb: Testing the initial appeal and actual usage of a potential new product by simulating its core experience with the smallest possible investment of time and money." The definition includes testing for "initial appeal and actual usage," which are the requirements for evaluation of a new product's market viability. There is no mention of testing software code: boundary testing, calculation testing, scalability testing, exploratory testing or even quality requirements. The focus here is on launching a start-up and testing the market viability by "building the right it" before doing rigorous software testing. So let's replace all references to "software testing" in his statement with the appropriate phrase:
"Alberto Savoia suggests (philosophically) that innovative start-ups should consider that the dominantly successful products and practices for testing a new product's market viability are dead and that they should adopt the pretotyping approach as an alternative."
- In the end, if we consider who Alberto Savoia is as an author, a start-up entrepreneur, and an employee at Sun and Google (when they were still considered start-ups), and that he admits to receiving "significant revenue, recognition and numerous awards" throughout his career, we can finally decode the entire statement to it's most native and direct form:
"Through the promotion of his own book, a founder of multiple Silicon Valley start-ups who has received significant revenue, recognition and numerous awards suggests (philosophically) that innovative start-ups should consider that the dominantly successful products and practices for testing a new product's market viability are dead and that they should adopt the pretotyping approach as an alternative."
Now why didn't Alberto just say that?

That is great advice and very encouraging to any software entrepreneur who is trying to decide which people and practices to employ just to get off the ground with their idea. Pretotyping is perfect for the 2-person startup (a technical genius + a business-minded guru) where you are trying to get angel funding to sow seeds of an idea, and it might work for non-critical web applications where failure is well-tolerated.

Unfortunately, this advice isn't very useful or valuable to the majority of IT departments in the real-world outside the start-up bubble of Silicon Valley. The corporations who provide the backbone for the global economy aren't start-ups. They know exactly what they care about and what their customers care about so they have little need for market research when it comes to IT. They test their software and they will continue to test their software to ensure their success.  And here is where I can end with my own statement, one that needs no demystification: Long Live Test!

5 comments:

Post a Comment