Estimation, Part 2: How Bloody Much, or If You Think It’s So Easy, Why Don’t You Have A Bash At Writing The Bleeding Code Yourself ?

The very worst technology cock-ups kill people. These dreadful mistakes are thankfully rare, but they do happen(*). Half a notch down, and a Premier League technology disaster can cause injury, large-scale physical damage (to objects rather than humans), or buggeration, chaos and tumultuous upset on a massive scale. Putting aside the awful cases where people are hurt, and so long as you are not directly affected, these are the very best types of IT Shambles. Space probes explode, telephone networks implode, banks and ATMs grind to a halt, and airport departure lounges fill up with weary and irate travellers as booking or baggage handling systems go haywire. Usually the blame lies with some grandiose, pompous and terribly self-regarding bunch of twerps: a government department, a tax-avoiding, worker-exploiting multi-national or, best of all, a bumper-bonus-for-the-big-bosses bank. The aftermath is all over the telly and the papers, and those of us on the outside can relish a nice self-righteous gloat with oodles of delicious Schadenfreude.

Travellers stranded at Brisbane airport in 2010 when Virgin’s check-in and boarding systems collapsed. An excellent catastrophe as (a) nobody died (b) it happened a long way away to Australians and (c) smuggo Beardy Branson gets the rap for it.

Travellers stranded at Brisbane airport in 2010 when Virgin’s check-in and boarding systems collapsed. An excellent catastrophe as (a) nobody died (b) it happened a long way away to Australians and (c) smuggo Beardy Branson gets the rap for it.

But the IT Shambles that attains widespread notoriety by causing palpable newsworthy distress to Joe Public is still relatively exceptional. The bog standard debacle stays behind the closed doors of the corporation that spawned it and the one solitary thing it blows a big hole in is the balance sheet. Typically the rickety boneshaker being hacked together works eventually, or mostly works, or is abandoned and put down before anyone is daft enough to try and use it. It’s impact is thus limited to

  • Costing far more than anyone expected
  • Taking vastly longer than anyone expected.

The failure therefore boils down to planning, or in other words, a sharp difference between the theoretical cost and duration as hypothesised before the whole juggernaut set off, and the actual cost and duration after it had landed. If said differences were modest or negative nobody would give a toss, but frequently they are colossal and positive. One could argue that this points to a shortfall in execution: the Requirements Logs, Gantt Charts, Work Breakdown Structures, High Level Designs, Baseline Budgets and what not were all unquestionably first class, but the useless clowns we got in to build it couldn’t stop falling down on their arses and tripping over their enormous comedy shoes. This is exactly like claiming categorically that the weather is wrong because it is tipping it down when the lady on Look East said yesterday it was going to be bright sunshine. Whatever will be, will be, to quote Doris Day. If, when attempting to foretell the future you ignored key initial considerations such as the Atlantic Low coming up from the Bay of Biscay on the one hand, or the bunch of incompetent nutters you’d hired for the gig on the other, your attempt at reading the tea leaves was seriously knackered from the off.

Que Sera Sera. Whatever Will Be, Will Be. The Future’s Not Ours To See. Que Sera, Sera. Wise words ever-pertinent to the topic of Software Project Estimation, from the erstwhile Ms. Van Kappelhoff.

So, to make planning disaster merely quite likely rather than unconditionally predestined, what must one do ? The diligent reader will recall the IT Shambles advice for those about to give any species of estimate related to the production of any amount of software. It is the same advice one would give to a friend who was about to take up Morris dancing, or to transfer 500 quid to a Nigerian bank account in order to enable the release of frozen funds totalling many thousands of pounds, or to sniff glue. Don’t. Stand firm, don’t give in, Just Say No. Any other course of action is weak, foolish, grossly naïve and will only ever bring misery raining down upon your head in the very near future. Instead of making a firm commitment to sponsors, bosses and customers that you will positively definitely finish everything outright on date X having spent no more than £Y, why not instead hand them the free end of a noose tied round your neck, and invite them to give it a good hard yank any time they want, simply for the joy of hearing you choke.

Sniffing Glue. Not a good idea. Daft lads, The Ramones. Not surprisingly all dead now, apart from Tommy, and snuffed it in all three cases well before earning their bus pass.

But sometimes the level of threat, cajolement and chicanery is such that all of your prevarication cards have been trumped, all your wily dodges countered, and all your misdirection and sleight of hand seen through. You have finally been outwitted by senior stakeholders even more devious, ruthless and underhand than you are and have no way out other than to fold and give the bastards an estimate. Or rather a quote. Everybody says estimate, but discards its old-fashioned meaning in plain English of “rough calculation or guess” in favour of the new corporate business definition of “100% accurate prediction of what will happen.” This fervent belief that projects can be scientifically forecast in advance is one of the many absolute truths of the modern workplace which we are required to accept as Iron Laws of Nature in public while simultaneously being fully aware are utter steaming nonsense in private. Similar anti-axioms which must continually be swallowed include:

  • the huge efficiencies obtainable via outsourcing and off-shoring
  • the necessity of matrix management
  • the vitality of a robust performance management framework as imposed by a dynamic Human Resources function
  • and so on and so on and so on

Nonetheless, you’re stuck with the appalling responsibility now. You have to take the farrago as it has been described in acres of UML diagrams, thousands of rows of Excel, or a banker’s box full of bits of paper covered in magic marker scribble, and make a plausible stab at how much effort will be required to lash the damned crate together, tighten up all its bolts and get the engine ticking over.

Your business analyst awaits delivery of the penultimate container load of requirements, yesterday.

Your business analyst awaits delivery of the penultimate container load of requirements, yesterday.

But one must not lose hope. Before being manoeuvred into a position where you had to come up with an infallible clairvoyant prognostication of what is to come, you would surely have fired off a whole battery of cunning and devious delaying tactics. These have all failed, but as a side-effect of their use you will now have:

  • Some requirements, which are ostensibly complete, fairly detailed, and more or less comprehensible.
  • A full list of the technology standards, rules, regulations, by-laws, decrees, edicts, directives and commandments that the architects and any other floating techno-autocrats demand you adhere to.
  • Half an idea of the boxes and wires on which you will be graciously allowed to install your marvellous creation.
  • The names of the guys and gals that will be constructing it with you.

So you know what you’ve got to do, most of the constraints and handicaps you’ll be burdened with, and who’s on the squad. The next few steps are obvious, if not easy: assemble your crack team, tell them what you know, plough through the massive spreadsheet containing the 7,846 MUST-HAVE use cases, and put figures on each one. There are lots of ways to do this. Some like Planning Poker, are moderately entertaining. Others like Function Point Analysis, not so much. None are perfect, and all are slanted to some degree towards over-confidence: that story shouldn’t take more than a week, should it ? Same way you reckon you’ll only have a couple of pints and then head home, and are pretty convinced that it won’t rain.

Assuming the Optimism Bias is not too marked, what you will end up with once you have totted up the 7,846 entries in column W is a partly (but not very) educated conjecture at how many idealised person days it will take until It Works On My Machine, or I.W.O.M.M. as we shall henceforth be abbreviating this to. To transform an ethereal vaporous notion into something which is written and runs, albeit solely on the desktop of a bloke who knows every subroutine inside out, is a minor triumph and should be celebrated with cake and maybe even a small sweet sherry. But it is a long way from the end, and there is much else to be done before you can safely allow unsupervised and primitive end users to get their mitts on your kit. The question of how much more is where opinion sharply divides.

The fact that it I.W.O.M.M. is a milestone worth recognising, but modestly. Don’t be getting caterers in or anything fancy, not yet anyway.

The fact that it I.W.O.M.M. is a milestone worth recognising, but modestly. Don’t be getting caterers in or anything fancy, not yet anyway.

There’s no single straight answer to this, but there are a lot of theories. Many people still strongly maintain that if I.W.O.M.M, then it is effectively done. Bung it live, reboot the server, job’s a good ‘un. I would strongly maintain that anyone who thinks that is mental and / or whacked off his mash on hash, pills and booze. You will find this brand of wildly cocky self-assurance coming from both blue-suited expensively-shod Programme Management Consultants as well as sandal-wearing nylon-trousered oddly hirsute Unix freaks. The suits use it cynically as yet another lever to squeeze time out of plans, labour out of proles, and blood out of stones. The beardy weirdys on the other hand are in deadly earnest, as they genuinely believe they always get everything right first time. They think they really are Jesus, rather than just resembling his pastier, fatter brother.

The sainted Fred Brooks, who is the opposite of a nutter, contends that getting the first draft up on its hind legs is about 1/9th of the whole shebang. That’s a ninth. 1 part in 9. 11.111%. From that point to tying the whole package up with a bow will require eight times as much manpower as you’ve already expended so far. His reasoning is that to get from a program running in one place to a programming product which has been generalised, tested, documented, holds up in a variety of environments with all manner of data, and can be maintained by others, you need to expend triple the effort. Similarly, to move from a program to a programming system component that uses and obeys defined interfaces and which cleanly integrates with other components, once again you need to triple the effort. In all likelihood you want to travel along both axes and start with a program but end with the sturdier, handier and more effective utensil that Brooks terms a programming systems product component. That then is going to take you nine times as long overall.

Most people are sane enough to realise (or at least can be persuaded to realise) that development-complete is a long way from actually-complete or even mostly-complete. The question is specifically how far away ? There is no straight answer to that. It varies, depending upon the neighbourhood you are operating in. When cobbling together a semi-disposable utility to be used by yourself and couple of other blokes, there might genuinely be little more to do once you’ve got your first draft going. But if you’re writing an engine management unit, a mass-market video game or a shrink-wrapped office application, then Uncle Fred’s 9x rule will be bang on if not a little rose-tinted. Most of us, most of the time, are somewhere between those two poles, but as we are typically being paid by a third party to write software for civilians, usually a lot nearer to the turnkey, off-the-shelf, heavier-weight end of the spectrum.

A powerful and virulent strain of myth persists that if you take the I.W.O.M.M number and double it, that should be ample time to do all that is required including buying the packet of celebratory custard creams. Folks that take that line generally feel that because they have added the same amount again in terms of budget / time / effort / people / man months / “resource”(**) to the I.W.O.M.M. figure, they have thereby been pretty darn conservative and careful. They will maintain that a 100% uplift should be more than enough to cover all ancillary work, any reasonably conceivable calamity and sundry other unexpected eventualities up to and including miscellaneous umbrella repairs. If you put it to them that the estimate should be twice or three or four times as much again, they are liable to feel that you are featherbedding to a ludicrous degree, or possibly even swindling them for personal gain. Much wailing, gnashing of teeth and grinding of spreadsheets will ensue before they will even grudgingly accept that you have not padded an estimate purely so as you can put in a gentle three day week with plenty of nice lie-ins and long tea breaks over the next six months.

Well-padded estimates in order to facilitate tea-breaks, lie-ins and pissing-off-earlys on Fridays are fundamental as a basis for decent civilised living. Take South Korea. A hive of industrial efficiency and hard graft where everyone gets up at the crack of dawn, stays late at the office, sings the company song with ardently faked gusto and never ever sneaks out for 20 minutes to read the Racing Post behind the bike sheds with a cup of tea and a fag. They put us decadent Europeans to shame. But as a consequence, the poor overstressed underleisured sods have the highest level of suicide and the lowest birth rate amongst the major industrial nations. Seriously, they could learn a lot about slacking off from us.

Well-padded estimates in order to facilitate tea-breaks, lie-ins and pissing-off-earlys on Fridays are fundamental as a basis for decent civilised living. Take South Korea. A hive of industrial efficiency and hard graft where everyone gets up at the crack of dawn, stays late at the office, sings the company song with ardently faked gusto and never ever sneaks out for 20 minutes to read the Racing Post behind the bike sheds with a cup of tea and a fag. They put us decadent Europeans to shame. But as a consequence, the poor overstressed underleisured sods have the highest level of suicide and the lowest birth rate amongst the major industrial nations. Seriously, they could learn a lot about slacking off from us.

End Note: this was intended to be the second and last diatribe on the topic of Software Project Estimation. But, despite having droned on for thousands of words on the topic already over two long screeds, all I have really succeeded in establishing is that:

(a)    It’s a mug’s game

(b)   You’ll get it wrong

(c)    It’s going to cost tons

(d)   It’s going to take ages

(e)    People are idiots and / or scum (see also: well, any previous post)

I have not got anywhere near quantifying how much it could cost or how many ages it might take. It is apparent now that reaching that goal is going to take me a few more thousand words of droning recrimination. In a move which is both neatly illustrative and ironically self-referential, I have profoundly under-estimated the amount to be said about project estimation.

– – –  E n d . . . M o r e  F o l l o w s . . . M u c h  M o r e . . .  – – –

(*) Computers bugs have killed people. The Therac-25 was a software-controlled medical instrument used to deliver measured bursts of radiation to cancer patients. A race condition resulted in at least six occasions where patients received massive overdoses, and three died from radiation sickness as a result. Less categorically, an RAF Chinook crashed in Scotland in 1994 and though not certain the most credible explanation is faulty engine control software. There are another couple of handfuls of fully attested fatal incidents, and then there is this one when a Soviet missile defence satellite misinterpreted the sun reflecting from clouds as a pre-emptive first strike by the Yanks. Fortunately, the Russian duty officer’s mercifully sound gut instinct told him it was a mistake and a false alarm, and he reported it up the line as such. Were it not for that, we could have had retaliation, escalation and the whole Mutually Assured Destruction finale straight out of Dr Strangelove in 1983. Milk Chocolate Hobnobs, the Internet, Father Ted and many other good things would never have been invented, but then again neither would Piers Morgan or Vanilla Ice.

(**) Anyone who uses the word “resource” to mean “person” outside of “ironic quotation marks” should be shot. But then, anyone using “ironic quotation marks” should be shot too.

Advertisements
This entry was posted in Uncategorized and tagged , , , , , , , , . Bookmark the permalink.

6 Responses to Estimation, Part 2: How Bloody Much, or If You Think It’s So Easy, Why Don’t You Have A Bash At Writing The Bleeding Code Yourself ?

  1. maxvernon says:

    It is with supreme glee that I read your latest diatribe. My only wish might be to hear from you more frequently!

  2. itshambles says:

    You are very kind. The bile seems these days to be coming out in bigger chunks but rather sporadically. I have (in all seriousness) been trying to channel my professional irritability into regular bite-size chunks, but it seems to be going the other way. Next post will probably be 20,000 words of stream of consciousness disgust, sans either punctuation or capital letters. Still, will aim to get that done this month, before kindly medics with bags full of psychotropic drugs cart me off.

  3. itshambles says:

    He’s right: agile won’t save you from rubbish developers. But if you have good developers it might stop you consuming their precious time in endless rounds of planning insanity.

    My particular night-terror at the moment is Wagile, whereby we all agree, Cargo Cult style, that we are being Modern and Progressive and Doing Scrum Properly. We have stand up meetings and stories and points and sprints, and also freely variable priorities and backlog, but somehow get the rest of the old rubbish from the Dark Ages too – fixed deadlines, fixed budgets, fixed scope but no requirements. Upshot being it’s all gotta be done by Easter, but we don’t know what it is and even if we settle for today on what it might be, we’re gonna add chunks to it tomorrow and turn it all upside down the day after. Joy.

  4. Pingback: Estimation, Part 3: The Art and Science of Predicting The Future With 100% Accuracy Using Only Guesswork and Microsoft Excel. | IT Shambles

  5. Pingback: Estimation, Part 4: Are We Nearly There Yet ? | IT Shambles

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s