Estimation, Part 4: Are We Nearly There Yet ?

We have now got past the first three stops on the Bus Journey of Estimation:

  1. First, we valiantly struggled to avoid making a forecast at all, and failed at that endeavour. But having carried out a lengthy, dogged rearguard action with skill, vigour and finesse, we have bought valuable time, squeezed concessions and information out of our enemies(*), and put ourselves into a far better position to make the guesses which will shortly be demanded of us. Sometimes, even when you’re doomed, it’s better to fight and lose than to give up without throwing a punch.
  2. Second, we tallied up how many mythical man days it ought to take until It Works On My Machine, and we took bloody good note that It Working On My Machine is a long way away from the Fat Lady singing.
  3. Third, we produced a much larger but even more mythical number of man days that probably will cover the whole thing right through to going live and mopping up bugs afterwards. We are by no means certain that it is the right number, but having been careful to allow for difficulties both expected and otherwise, it does have a fighting chance of being sufficient, for once.

A Fat Lady, Singing. Wagner, apparently. Not my cup of tea as it goes, but illustrative.

At this point you have a putative figure for days which you can multiply by a speculative average rate per person in order to come up with a theoretical amount of notional money which you can advance to your senior stakeholders as a supposed budget for the undertaking as a whole. Well done. This is an achievement not to be sniffed at. But immediately on producing this imaginary number, two disturbing things will happen. Firstly, it will be heavily published, circulated and discussed in business cases, cashflow projections, cost-benefit analyses and the like. Through this process, it will change state; though it may have begun as a heavily caveated and pretty fluid range of financial possibilities, it will rapidly be crystallized for you into a single integer – a solid, immutable, unquestionable, rigid block of hard cash. Simultaneously your weeks of conjecture and educated guesswork will be transmogrified retrospectively into mathematically rigorous and provable formulae, and this leads us to the second disturbing event. Now that your wholly scientific prediction has been officially chewed up, digested, regurgitated and thereby allegedly validated, risk-mitigated and baselined, some dirty rotten swine is going to ask you when they can have the damn thing. Or rather more probably, they will explain to you in forceful terms the date by which it will be ready, based upon their deep understanding of software development, technical delivery, man management, time and maths.

Imaginary, irrational and indeed wholly fictitious numbers feature heavily in the delusional world of Software Project Estimation.

Imaginary, irrational and indeed wholly fictitious numbers feature heavily in the delusional world of Software Project Estimation.

The highly sophisticated algorithm they will employ to produce and justify that date will run broadly like this:

  • Take proposal as presented by techie grunts and regard sceptically over top of spectacles whilst sucking air through teeth.
  • Despite presentation as a range of options each with different conditions attached, ignore conditions and pick smallest.
  • Discard any and all contingency.
  • Divide days in suitably improved estimate by five to derive ideal developer-weeks
  • Divide ideal developer-weeks by ridiculously large assumption on team size to produce duration
  • Increment today by duration to determine end date. Obviously, coding will start right this very minute.
  • Round up to end of week to cover any overruns.
  • Job done. All that remains is to book the caterers, the backroom at the pub and the DJ for the post-launch shindig.

Black Lace there, with “Agadoo”. Marvellous. An essential record at any celebratory-type knees-up occasion, particularly if said festivities were to be occurring in West Yorkshire during the mid 1980s.

Making use of this marvellous logic, which can be easily replicated on either a discarded bus ticket or in a dozen cells in a spreadsheet, one can kick off with a sensible guesstimate of effort and end with a completion date of stratospheric lunacy. Let us drill down through a specific example of this brand of bullying stupidity using the It-Works-On-My-Machine-equals-100-days-so-500-days-all-told example from our last lesson. This eminently reasonable punt will quickly be forged by your business chums into an unachievable but ironclad deadline as follows:

  1. Let’s be frank, it’s not 500 days. The calculations baldly admit there is a 15-20% uplift for risk. Right away we can snip off that fatty 70 day contingency which is plainly there solely for padding or some other species of technical timewasting.
What could possibly go wrong, with the engagement of such capable and seasoned professionals ?

What could possibly go wrong, with the engagement of such capable and seasoned professionals ?

  1. Even the 430 days we are left with in the proper estimate is transparently a development bargaining position, and thus not meant to be taken too seriously. Ergo, nip, tuck and shave corners off at various points. Testing can’t take that long. If our developers spend that long bug fixing they must be half-wits. What is this code review poppycock? Does it generate a value-added organisational uplift ? Why must the Business pay for this nonsense ? Our procedures are slick and streamlined, there is no way so much resource is allocated purely for dealing with governance. Fiddlesticks, pish, tish and fooey. With ruthless, hard-headed pruning, the core is revealed: 350 days actual work.
  1. 350 divided by 5 makes 70 full-time mythical person-weeks. That mythical person in this instance being a modern-day Bob Cratchit, a terrified Victorian-style drone who never dares to take holiday or call in sick, and sweats away at maximum efficiency at all times with absolutely no distractions whatsoever. We should perhaps be grateful your customer doesn’t want to repeal the Factory Acts and so base his reckonings on a good old-fashioned early 19th Century working week of 14-Hour days including Saturday and Sunday.
Leisure time for the lower orders ? Fie Sir, a Bolshevik notion – it would only be spent in drink, licentiousness and criminality !

Leisure time for the lower orders ? Fie Sir, a Bolshevik notion – it would only be spent in drink, licentiousness and criminality !

  1. We can without doubt put a team of 10 onto the task, all toiling at full whack for the whole duration.
  1. Seven weeks then. It’ll be done before the end of next month.

There may be a tiny morsel of mockery and misrepresentation in my burlesque on the thought processes of your classical senior business stakeholder thug here, but really very little. I’ve been in the meetings, I’ve read the emails and I’ve wept over the Gantt charts. In fact, it can often be a lot worse. I have ignored the commonest response of all which is flat-out sneering rejection in place of the first two steps: “Surely you cannot expect to retain respect within the wider organization with an estimate of almost two man years for what is no more than a straightforward change. It is completely unacceptable. I am convinced that a two smart guys can easily do what needs to be done well before the end of October. You need to go back round whatever loops you boys go round and come up with a proposition that has the sort of credibility that allows me to present it to the Programme Board, and you can start that exercise by dividing everything by two.” So seven weeks is by no means the worst case scenario, but nonetheless the sheer volume of reality denial, false assumption and sheer boneheaded stupidity rammed into that five step recipe is breathtaking. Let us try the same exercise again but with a degree of honesty. One could get all fancy with Gantt charts and dependencies and resource levelling and the like, but that merely serves to create the illusion of science, confidence and certainty from a nebulous cloud of suppositions, rules of thumb and half-truths. Quickest, best, and less self-deluding to stick to the back-of-an-envelope approach but to do so with grim, clear-eyed objectivity.

  1. If we think 500 days now right at the off, then the rational thing to do is to plan for at least 600. Can anyone recall a single blasted undertaking within living memory that crept in even within spitting distance of its budget, never mind within it ?
  1. Only macho cretins make plans without contingency. Things will always, but always, go wrong. Terrible appalling events will occur which no-one would have even dreamed of in their worst nightmares. We will need time to deal with those horrors. Alright, so we’ve bunged in several different chunks of just-in-case time already but no harm in adding on another 50 days for good (or rather, bad) luck.
Alas, the omission of crash barriers on Italian mountain roads had never been formally logged as a risk. With proper governance, all of this needless drama would have so easily been avoided.

Alas, the omission of crash barriers on Italian mountain roads had never been formally logged as a risk. With proper governance, all of this needless drama would have so easily been avoided.

 

 

  1. Your common or garden white collar IT spod during a standard sort of year somewhere in the developed world
  • takes around 25 days annual leave. Scandinavians, Germans and other enlightened Northern Europeans get about 30 (hooray). Americans get 12 (boo) and the poor old Japanese get 18 but take almost none (double boo with knobs on)
  • has a couple of weeks of public holidays. Religiously observant Mediterraneans (the Cypriots, Maltese and Spanish) get around three weeks while we oppressed Britishers get eight wretched measly days. And it always flaming rains, especially in May.
  • is ill for about a week
  • spends a week in training
  • and loses another week or two to parental leave, team-building events, HR survey feedback sessions, heavy snow, inspirational management initiatives, the Christmas piss-up or any of a dozen other extraneous non-work or not-really-proper-work-type interruptions.

All combined, that comes to 10 or 11 weeks, or roughly 20% of the available time. So there are in practice just four days for your actual doings in a regular week.

Time away from the dismal daily grind: Clacton-on-Sea at the height of a glorious English summer. A bit of drizzle, but we still had a good time and the whelks were outstanding.

Time away from the dismal daily grind: Clacton-on-Sea at the height of a glorious English summer. A bit of drizzle, but we still had a good time and the whelks were outstanding.

 

 

  1. But that’s not all. We then need to factor in how much time gets spent on unfortunate diversions. Typically this will manifest as folk being roped in to sort out live problems and / or drafted into activities even more terminally shagged than the one they’re officially engaged upon, in order to dig them out of their latest hole. How large this effect is varies from place to place but I’d say it uses up at least another half day per person per week. And this burden falls disproportionately on the razor sharp problem solving geeks you need to get your project finished – they’ll be the first ones who get kidnapped to sort out some other mess. You’ll be allowed to hang on to the thickos, the idlers and the terminally bewildered. As ever, best to assume the worst and say it’s going to amount to another full day a week.

So, in practical terms, you’re looking at a Three Day Week. These Icons of the ’70s are your Poster Boys:

Racism, power cuts, industrial collapse, decay. Weren't the 70s great ?

Racism, power cuts, industrial collapse, decay. Weren’t the 70s great ?

650 days divided by 3 makes a slightly-less-mythical 217 person-weeks.

  1. How many people can you usefully put on a job ? Depends on the job. Nine blokes might dig a ditch nine times quicker than one, but to flog a clichéd old nag once more over the hurdles, nine pregnant women can’t make a baby in a month. Software is somewhere between hole-digging and baby-making in the fungibility stakes – more people make it quicker up to a point but it’s decidedly non-linear. Say in your case four engineers and a couple of testers can be kept busy most (but not all) of the time, with a few other part-time and cameo appearances from managers, analysts, architects, DBAs and so on. When you tot up who does what and with which and to whom and precisely when, say it’s equivalent to a team of between six and seven overall.

217 idealised person-weeks divided by six and a half hypothetical people is 33 notional weeks. Note that despite starting from precisely the same point, the application of reason in place of insanity has the thing taking nearly five times as long. And we haven’t even finished yet.

  1. Up to this point we have been applying caution, good sense and realism to planning, in counterpoint to the more usual cocktail of machismo, stupidity and optimism. Conceivably the assignment in question could be done in seven or eight months if we were simply allowed to get on with it. But though we may have planned carefully and prudently for once, that doesn’t mean our workplace will suddenly start to operate in a pragmatic and judicious fashion. Nope. It’ll still be the same bumbling officious farce it always has been. So now we must apply caution, good sense and realism to figuring out just how daft it will be in this particular instance, and in how many different ways and to what extent that daftness will slow us down. Bureaucratic drag normally manifests itself in the shape of approvals; every step you take, some fool has to tick a box before you’re allowed to lift one foot in front of the other. To move from stage A to stage B, to pass through gate C, to promote to environment D, or to be allowed to draw on resource pool E or cost centre F, some tea party of nitwits have to convene and agree that all your papers are in order first.
Approved, at least by one idiot. Only another 97 to go.

Approved, at least by one idiot. Only another 97 to go.

Idiocies and red tape will vary from locale to locale but something along these lines would be about par:

  • Financial approval. Outline and final, obviously.
  • Design approval. Again, outline and final – this is the big league blue-chip Enterprise Space, not some tin-pot garage operation.
  • General bumph approval: some git will want to see that your Initiation Document, Terms of Reference, Mandate, Brief, High Level Design and Architectural Principles are all in order. Papers Please !
  • Stage Transitions. The same git will turn up with his flashlight, mirror and rubber gloves to inspect the inner cavities of your project every time it crosses the boundary from one phase to another. Naturally there are many borders to be crossed as you move from Conception to Initiation to Build to Test to Accept to Deploy to Adopt to Support. Prepare to spread them, cough and answer questions each time. Christ, why can’t these people just apply for a proper bottom-inspecting position in Customs and have done with it.
  • Change Control. Every sodding time you have to deploy one sodding miserable one-line bug fix.

This will take months if you obey the last letter of every rule, regulation and guideline. Even with ducking, diving, bribery, and a dollop of Jedi mind control, it will take an age. As wearily noted elsewhere, under max-strength corporate governance getting a Hello World program from inception to wholesale ratification as a strategic system of record can take over a year.

These aren't the droids you're looking for, and in any case they both come with outline _and_ final design approval from the Strategic Enterprise Architecture Forum as well as business case sign-off to boot. You can go about your business. Move along.

These aren’t the droids you’re looking for, and in any case they both come with outline _and_ final design approval from the Strategic Enterprise Architecture Forum as well as business case sign-off to boot. You can go about your business. Move along.

Let’s not be too defeatist, though. With a bit of wangling and applied dodginess you’ll be able to keep going most of time and either nip round or under the various bureaucratic obstacles. You will lose the most time at the beginning, as you labour to get funds released, people assigned, designs authorised, and a dozen different forms countersigned in triplicate by 57 different apparatchiks. Let us say three weeks, with another three weeks for miscellaneous pratting about later on in the game. These later hold-ups will chiefly be due to waiting hours and days for clueless Change Management blockheads to deign to open your three-line email, read it, and click a button. The massive twats.

And so we end up with 39 weeks. A deal more than the 7 weeks your Business Gangmaster would have liked to thrash you into swallowing, but far more likely to be genuinely enough time to do the necessary. Planets sometimes align, biorhythms can synchronise and your office may be sitting on a conjunction of ley lines; in such karmically favourable circumstances, it could come together a bit swifter. Or you might squeak in the door with minutes to spare, or the cock ups and calamities may line up so catastrophically that you overshoot even this conservative conjecture. But with this brand of wary, circumspect planning, those gut-knotting outcomes will become rare and exceptional. You may even come to relish them, as an exciting but thankfully occasional diversion from your normal run of serene, calm, ordered and even early delivery of software.

Early delivery. Should not be limited exclusively to milkmen and moonlighting priests.

Early delivery. Should not be limited exclusively to milkmen and moonlighting priests.

 

 

 

 

 

 

 

 

 

 

– – – – E n d – – – –

(*) By “Enemies” I am naturally referring to

(a) Customers, and;

(b) Senior Management

These will always be the twin poles of your workplace’s local Axis of Evil. But clearly the term can and should be extended out to include assorted affiliated Minor Belligerent Powers: the Programme Office, Change Management, Human Resources, Systems & Infrastructure, End Users, Architects, Marketing, Sales, Product Managers … well, pretty much everyone except the cleaners and the folk in the canteen.

The custard may be lumpy, the chips a little soggy and the customer focus a shade under par, but they rarely impede progress in any significant way.

The custard may be lumpy, the chips a little soggy and the customer focus a shade under par, but they rarely impede progress in any significant way.

Advertisements
This entry was posted in Uncategorized and tagged , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s