Obviously, Dear Reader, you, along with everyone I’ve ever worked with or interviewed have never really screwed up a bit of software development. Oh no. You’re a professional. You don’t make mistakes. Like the Pope, or Mrs. Thatcher, or Stalin. Well done you. But I am aware that there have been quite a few massive cock-ups in the IT industry over the years that I was nowhere near. The industry as a whole has form. The industry also has a long history of wondering why it cocks up so spectacularly so often. The first and very best stab at self-critical navel-gazing was published in 1975 and was based on the author’s experiences at IBM in the early 60s. I refer to Fred Brooks’s genuinely seminal “The Mythical Man Month”. This is a wonderful book and lays out very neatly most of the reasons why large scale computing projects so quickly go to hell in a handcart. It is a shame to reduce 300 pages of crisp, erudite prose to bullet points, but here goes.
- For starters, writing software at all is hard. It’s a branch of pure mathematics, for chrissakes. To do it well, you have not only to understand algorithms, logic, data structures and abstract syntax but also to be fluent with and enjoy all that stuff. And get it exactly right, almost all the time. Who matched that profile in your school ? Just you and four geeky chess club mates, that’s who.
- Big projects tend to wander off into the wilderness, far from their original goals and design principles. As they do, they become fragmented, over-elaborate, incomprehensible and .. well … just … nasty.
- There are always far far more bugs than anyone would reasonably expect, and a surprisingly large number of them are agonisingly intractable.
- There is always tons more testing to do than you expect. And documentation.
- All estimates of manpower and time required are ludicrously optimistic.
- The bigger the project, and the bigger the project team, the more likelihood there is of carnage.
- You can’t just throw extra random bodies at it and get it done faster. It isn’t potato picking.
- Requirements change as you go along. And they’re always wrong. Nobody actually knows what the hell they want. Not even when they’ve finished.
- We all think we know how to code, test, manage or whatever. But we don’t. Most of us are amateurs and the rest completely clueless.
- Self-delusion is endemic and infectious. It’s all going to be fine. Definitely. We’re not behind. If we are, we’ll catch up. There won’t be any bugs. It should just work. Zero defect is my middle-name.
- Self-delusion is normally backed up by threat. The Big Chief is absolutely positively certain it’s all going fine, and is not in the mood to hear otherwise. Oh no. So quit your bitching, unless you want to get a bloody good leathering and your P45.
- Everyone’s terrified, no-one wants to be the bearer of bad tidings, no-one wants to take the blame.
- Ergo, arising from all the above, planning software development is impossible. Just give up and go home.
I paraphrase somewhat but you get the gist. Read it yourself in full – it’s still in print and available here.
The net effect of Mr. Brooks’s widely-read book in preventing IT disasters has been, to a first approximation, nil. There have been numberless debacles since 1975, many of which I had nothing at all to do with (*). ZDNet propose a Top 10 but I have two personal favourites not listed there. The first, close to home, was the introduction of Oracle Financials at the University of Cambridge. So much time and money was spent that the University felt compelled to devote an entire issue of its official magazine to analysing what went wrong. The other is NHS Connecting for Health. This programme was appallingly wasteful on a cosmic scale, burning around 12 billion quid over six years. It made the national press, and Private Eye in particular reported on it extensively. Connecting for Health delivered nothing useful, and has just been canned.
These two projects handily illustrate two distinct breeds of shambles. Connecting for Health was doomed, insane, grandiose and without merit from the start. It wasn’t filling any need; the medics who would be its primary users could see no merit in it right from its earliest stages. That these starting advantages were combined with aggressively brainless macho management at the top level, jawdroppingly large amounts of dosh for those running and supplying the programme, and enterprise levels of incompetence across the board raised it up from your common-or-garden large government white elephant project to a world-class exercise in advanced pointlessness and flushing wads of cash down the khazi.
Contrastingly, the university finance system project was not in conception particularly daft. There wasn’t even much code to be written; in the main, the work was to install and set up a pretty standard suite of financial software used widely by stacks of big organisations. So, one would think, little chance of making a balls of it. But with the right approach, even what ought to be a cakewalk can become a deathmarch.
So. Self-styled IT Professionals have been making a mess of things for at least fifty years and in many and varied ways. I find it comforting to reflect that as I fail, I am upholding that august tradition.
(*) I should make it clear at this juncture that I was still at junior school in 1975. I therefore only start to become accountable for software project catastrophes from the early 90s.