As a freelance developer, I’ve always thought that formal requirements specifications are just too much overhead. Sure, big projects in big corporations will have to ensure the budget and time-frame very accurately using rigorous methods like IBM’s RUP or the ISO approved Function Point Analysis, etc. But for a one-man show, I mean, I could have got the whole project done in the time I had to prepare the specifications, right?

Wrong

Programmers have always notoriously over-estimated how fast they can write code. Of course, we enjoy writing code and time passes by just so quickly, but it doesn’t mean writing code is as really as efficient as writing English. Many small teams shy away from formal specifications because they think they could have written the damn code by the time they’ve got the specification down.

I’ve always believed that, as much as I believed I’m a programmer with adequate skill, until a recent revelation hit me:

Writing the program is, in itself, a DAMN SPECIFICATION PROCESS

That’s right. Instead of confirming with the customer what the requirements are about in clear concise English, we somehow feel better describing the problem to a computer using programming languages. We have advanced through several decades and have lived past the dark eras of Assembly programming to higher level programming languages like Ruby, Python and idioms like functional programming. Why did we choose to move away from low level languages like Assembly and C? Because higher level languages are inherently more expressive — we can say much more with fewer words in a higher level language. In other words, high level languages allow us to program faster.

And what’s the highest level language of all?

English!!

That’s right. Surprise.

But computers can’t read English, doesn’t that mean duplication? We now need to define it in two sets of languages.

Yeah, and that upsets my DRY spirit a lot.

In an ideal world, we could understand the requirements 100% accurately, code it once in the programming language of our choice (Ruby anyone?), deliver the product and bring home the bacon. Everybody is happy

But how often does that ideal scenario happen?

The ideal never happens. Let’s face it, we need to rewrite the damn code anyway.

That’s right. Unless you are an uber skillz haxx0r incarnate, I believe you don’t expect your program to work totally correctly the first time you run it, right? For any programs that more than 10 lines long, I’m it takes at least a couple of retries at least to get it compile without errors. Even if you get it to work correctly, how many times have you written a program and say to yourself “Yeah, this is the perfect solution to the problem”?

That is, even after you get it to work technically correctly, you’ll realize there are some ways to improve it, some quirks that really shouldn’t be there. It’s called “I know it when I see it.”

See? So much for DRY, we need to rewrite it over and over. It’s our damn fate.

If you have to throw away “code,” would you rather throw away a couple of English descriptions plus some wireframes, or a full set of engineered objects?

Many Agile proponents advocate writing code as soon as possible. Ironically, it’s pretty stupid once you think about the pseudo-math behind it:

Let the total number of iterations needed be R
Let the amount of time needed for each English iteration be E
Let the amount of time needed for each Coding iteration be C

Obviously,
    R ≥ 1
    C ≥ 1

For total amount of time needed T:
    T = (R - 1)C + RN
        where N is either E or C

For example, in an ideal case, we do no specification,
just code it once and everything goes fine:
    T = C

Scenario 2: the first version gets rejected by client since we
misunderstood the requirements (or they think we
misunderstood, whatever). It took 3 times of coding:
    T = 3C

Scenario 3: we made a detailed specification, client realized
they mispresented the requirements (notice how the client seem
to understand their own faults when you just tell them in English
instead of code). It still took 3 iterations:
    T = 2E + C

Obviously, E < C. Which means we've saved time!

You see, the only case where writing the detailed specification would “waste” time is when you get everything right the first time. Honestly, how often does that happen?

So what should I do?

Should I go out and read on all rigorous processes out there? Probably, I went down that route and it was quite fun to learn each of them. But ultimately, I must also acknowledge the huge overheads those processes incur. And enforcing the client to go through rigid processes don’t always work without a full team of experienced lawyers and contract writers.

After a lot of trial and errors, the approach I’m adopting now is to have as detailed specification as possible before writing any code. It doesn’t necessarily have to be 100% accurate UML diagrams with spell-checked use cases. I find that just a set of PowerPoint slides, annotated wireframes along with some rough use case scenarios seem to work fine. I try to avoid having charts because people tend to misinterpret quite often. It’s the right mix of flexibility and details that maximizes my productivity.

I’ve been reading about software project management, agile practices and stuffs for several months now, almost becoming the metaphorical PM that’s read 20 books on agile. Came across this post that’s hands down the most true, funniest description about software management I’ve seen.

A man is flying in a hot air balloon and realizes he is lost. He reduces height and spots a man down below. He lowers the balloon further and shouts: “Excuse me, can you tell me where I am?” The man below says: “Yes you’re in a hot air balloon, hovering 30 feet above this field.” “You must be a software developer,” says the balloonist. “I am,” replies the man. “How did you know?” “Well,” says the balloonist, “everything you have told me is technically correct, but it’s of no use to anyone.” The man below says, “You must work in business as a manager.” “I do,” replies the balloonist, “but how did you know?” “Well,” says the man, “you don’t know where you are or where you are going, but you expect me to be able to help. You’re in the same position you were before we met but now it’s my fault.”

There has been a lot of fuss about agile methods — XP, Scrum in this decade. My experience is primarily with fixed-price so I can’t speak for myself the effectiveness of agile methods. On the other hand, what I do feel is how sucks when the client is firing change requests: often trying to get it free.

I recently worked with a client who just refuses to write any detailed specs. When I pressed him, he put up agile to his defense.

Recently read the book Extreme Programming Refactored, which sums up my thoughts pretty well:

In the real world it’s called making your mind up, having a clue. Yet XP wants to stand all this on its head and encourage ever-changing directions and goals.

Yeah, it’s called making your mind up, adults.