It is easy to feel sorry for Testers. There they are, the poor sods, right down at the far end of the software sewer, the last filtration point before the awful foul-smelling rubbish gets pumped out all over the users. Whatever grandiose coke-fuelled opportunity the Proposition Team hallucinated, whatever looney tunes nonsense the designer dreamt up, whatever awful mess of cut’n’paste jerry-built hackery-pokery the developers bodged together, they have to get it running and figure out if it actually does what it’s meant to. Everyone else further up the line had the chance to influence the system for the better. The Product Manager could have pruned a few of the loopier notions and better defined the remaining requirements. The Architect could have refrained from wheeling out his latest Super-Service-Oriented-Mega-Loosely-Coupled-Technofreak-Utopia design blueprint. Again. The engineers could have done a bit of unit testing, maybe, or optimised a couple of the queries, perhaps, before punting it into the code repository and praying that it compiled. And now there it is, dumped in a steaming pile all over the QA environment, refusing to allow the hapless spod lined up to test it even to log in.
But your pity would be misplaced. In the shambolic IT project, no-one is innocent. Testers are generally better organised and less confused than developers, and cases of extreme bloodymindedness are rarer, but you are just as likely to encounter stupidity and slacking as you are anywhere else. In fact, some of the richest and thickest veins of stupidity can be found within the QA department. You will at some point meet a tester so useless that he doesn’t appear to have encountered a computer programme ever before in his life, let alone been set to test one. You might as well ask a tribesman from the Amazon Jungle to put the accounting module through its paces.
But above and beyond simple idiocy, the characteristic test team neurosis is a brand of aggressively dim helplessness. They are chronically unable to do anything, consistently up a gum tree, but it’s totally out of their hands, it’s not within their purview(*) and Somebody Else needs to sort it out for them. They couldn’t write the test cases because the spec wasn’t signed off. Once it had been signed off, they still couldn’t write any test cases because they didn’t have a fully validated data model. They couldn’t run a comparison against the old version as we couldn’t provide them with a technical user manual for it. When we did dig one up, they experienced serious comprehension issues and demanded consultancy from the original team in Finland that wrote it, plus a Finnish translator. No matter that what was under test was an upgrade to a mobile phone platform and that its key features were understood by everyone in the developed world from pre-teen chav to granny, not to mention three quarters of the developing world too and even a good solid wedge of those tribesmen from the Amazon Jungle. No. At each point before any single bloody box in the Test Checklist could be ticked, Somebody Else had to lead a reluctant tester by the nose through an explanation of the bleeding obvious that would patronise a bright nine-year old. Somebody Else is normally an engineer who is able to figure out for herself how to make a phone call, add a contact and even send a text without the need for extensive documentation and/or Knowledge Transfer Workshops.
Unless you can bus in a load of clever junior school kids or smartphone-savvy indigenous Brazilians, you’re stuck with the Testers. As ever, you might be lucky and find one that can figure out how to use a computer unaided. But mostly you won’t. Testing therefore functions as a 50% high-rate tax on development: in order for one dullard to be allowed to write code, you have to pay for another to pretend to test it. Occasionally some actual testing gets done, but usually the wrong sort. I worked with one lad who could only test destructively. His passion was devising complex tortures for innocent computer systems. Leave those fields blank, shove in some Chinese characters there, set those amounts to zero, press the back button twice, hit Submit and Bingo – stack trace a-go-go ! At these moments his face would light up like a six-year old finding a forgotten Creme Egg three weeks after Easter. Which is all very well. It’s good to find bugs, and the code should cope with nutters taking circuitous routes through it. But the point of testing is to establish that the flaming software works, not that it can be poleaxed if you really put your mind to it. Any piece of kit can be kiboshed if you try hard enough. Take your old video recorder, for example. It’s been surplus to requirements for the last decade so you can afford to experiment on it. You’ve got The Princess Bride and the Hitchhiker’s Guide (telly series – not the film) on DVD, and all the clips you taped from Top of the Pops in the 80s are now on YouTube. When you’ve fished it out of the loft, fill it with washing up liquid, chop it up with an axe, put all the bits in a plastic bag, and bung ’em down the lavatory. I guarantee that after that treatment it will no longer play your well-thumbed VHS of “Porky’s Revenge”.
Vyvyan and Mike illustrate the inadequacy of purely destructive testing (circa 6m 45s in).
So we have established that you are doomed to invest vast amounts of effort, time and money in testing for a perishingly meagre yield. Unbidden, the question comes to mind. Why Bother ? Why not stuff the QA Principles, say knackers to the Testing Charter and shred the Zero Defect Guarantee. Just hoof the ruddy cobblers into live and see what happens. Rather than pratting about with clueless testers and their code coverage metrics and requirements traceability matrices, have the actual customers use it for real and mop up the mess afterwards. Could be a bit frantic for a few days but as long as we can fix forward quick enough, it can’t be much worse. Can it ?
I once worked at a place where the No-Test Philosophy was followed with rigorous Neo-Marxist zeal. The CTO had a well developed, tightly argued ideology of software development which went much further than the old put-your-brown-trousers-on-cross-your-fingers-touch-wood-install-oh-dear. His first theorem of testing was that users were so unpredictably destructive there was no point trying to second-guess their behaviour. Whatever use cases you planned for, the swine would bust the kit in some utterly unexpected new way. Alongside this observation he placed the central tenet of his worldview, which was that if one had designed and coded correctly, there could be no bugs. It was a logical impossibility. Ergo, taking these two principles together, testing was 100% unnecessary. QED. QA was for wimps and Write-Compile-Deliver-Invoice was the ballsy macho methodology we followed. We could not have been more cavalier if we had been wearing three-cornered hats, wielding rapiers and straddling white thoroughbreds whilst stroking a King Charles Spaniel. Luckily for the CTO, as well as being feckless and gung-ho, the techies he had hired were to a man sharp, resourceful and hard-working and so somehow he got away with it for decades.
If you have a crack team of ninja programmers with an insatiable appetite for disaster and the euphoria that follows hauling a system out of the abyss, Can’t-Test-Won’t-Test will work, and it is fabulously efficient. But the first fortnight after a live drop is always unmitigated carnage. The stress, lack of sleep, mess, terror and psychological torment can only be likened to an attack of chronic diarrhoea and projectile vomiting while looking after a newborn baby during an artillery barrage. I exaggerate a little, but only for effect. If you haven’t got superheroes on the staff, or their devotion falters, or you simply have a run of bad luck, you will drop down into that abyss and never drag yourselves out.
And there endeth the neatly circular argument. Starting from the position that system testing is without merit and worthless, I conclude that it is vital for the preservation of order, sanity and tranquil living. I retain a few bitter, lingering grudges nonetheless. Specifically:
(a) the code just ought to work better in the first place.
(b) even if it is cack it shouldn’t be so chuffing arduous to establish what exactly is right and wrong with it.
(c) why are there so many bits of paper and why is so much time spent filling them in ?
(d) why can no-one ever work out anything for themselves ?
But there is no viable Plan B. You must lash together a tester and an engineer, force the unhappy pair to lug the code up out of the cess pit and, if they can’t completely disinfect it, they can at least scrape off the biggest and smelliest lumps of crud. The alternative, unfortunately, is chaos.
Testing in this regard is just like all those things you used to moan about when you were young. Parents. Policemen. Multinational Corporations. Teachers. The Government. Chores. Bedtime. God, it’s so unfair. If they’d all just back off and stop laying down their bad vibes, things could be groovy / cool / spiffing / awesome / rad / sick / bangin’ (delete according to generation). This teenage sentimentality was taken to its illogical conclusion in the late 60s, leading to a troubling spate of Giant Outdoor Flower Child Rock Festivals. The Hippy Nation was convinced it could look after itself. All it needed was Love. They didn’t need no education, cops, big business, politicians or squares bringing them down. Sadly, you can have as much Love as you like but without education, cops, big business etc you also have to forego potable water, sewage treatment, public safety and food. If you keep out of the way of the Hell’s Angels and stay whacked off your crust on pills and dope it’s doable for a long weekend, but that’ll be quite enough. After a few days, you’re going to want a bath and a clean sit-down toilet.
Just like you do a deal with The Man when you get your first job, you are going to have to get along with Formal System Testing too. Like its equally unloved cousins Governance, Quality Control and Process, it seems (in moderation) to be essential in preventing a standard issue grotty IT shambles becoming a continent-wasting meltdown which could trigger the collapse of civilisation and a wholesale descent into anarchy. Box-ticking and tedious they may be, but abandon any one of these burdens and drug-crazed hippies will within hours be smearing themselves with their own filth, fighting and fornicating in the streets outside your house.
(*) Purview. Like Paradigm, I don’t know what Purview means and I don’t want to. But it’s always Bad News when a Purview lumbers into view. Nobody ever turns round and says yep, that fits right bang slap in the middle of my purview that does, I’ll sort it. Oh no. Whatever is in question, every bugger else’s purview is certain to be too small to accommodate it. A purview must be the absolute zero Kelvin of volume – two fifth’s of a Gnat’s Chuff, perhaps.