Flexibility vs Efficiency

This post by Dmitri kicked off a mini-debate about agility and rigidity. The debate is about whether a developer should be interrupted for a day to perfrom some side job. Check out the posts linked above for the whole story.

This is a common enough issue that it requires further examination.

At the root of the issue is what I call the flexibility-efficiency dilemma. This tradeoff is, in fact, at the root of many issues of agile vs traditional processes. The essence is that if you want more flexibility, you trade in less efficiency and vice versa.

Example

You have ten packages that you want to send by post. Each package involves buying the cartons, collecting the items, packing the items, writing out the address, filling up forms, and finally going to the post office and delivering them. There are two ways to approach this problem.

Approach 1

1. Buy ten cartons
2. Collect all the items
3. Pack the ten packages
4. Write out ten addresses
5. Fill up the ten forms
6. Go to the post office and drop off all ten packages

Approach 2

1. Buy one carton
2. Collect items for one carton
3. Pack the carton
4. Write out the address
5. Fill up the form
6. Drop off carton at post office
7. Buy another carton, and repeat the whole process ten times

Which approach is better? Most people would say the first one. Hmm. First, we need to define what we mean by better. Let's rephrase that question. Which one is more efficient? I think we can all agree on the first one. Its a no brainer. Just the ten trips to the post office makes the second approach way more inefficient. Let us say that it took ten days to send the ten packages via Approach 1 and fifteen days via Approach 2.

Now, which one is more flexible? Ah. This time its the second one. Why? Because it is much better suited to handle unexpected surprises.

I'm going to add in a surprise now. Surprise: After five days, you are informed that you need to go overseas for an office trip right now. Packages sent using Approach 1: Zero. Packages sent using Approach 2: Three.

Lets try another surprise. Surprise: It seems that customs is not letting such large packages go through. You need to repack into smaller cartons. Approach 1 impact: All your ten packages come back in two days. You repack everything for another ten days. It takes twenty days to get everything through. Approach 2 impact: After two days, your first package returns. You repack and resend it. The unpacked items are packed into smaller cartons the first time around. It takes seventeen days to get everything through.

Apart from the above approaches, are intermediate approaches. You could make five trips to the post office, sending two packages each time. The net result of efficiency and flexibility will be somewhere between the two extremes.

Software processes

Surprisingly (if you think about it, not such a surprise), the two approaches have a direct correlation with software processes. Just a couple of quick definitions. In the next paragraph, when I say 'agile', I mean an Agile (captial A) process. When I say 'agility', I mean ability to handle unexpected requests. So high agility mean high capacity to take on unexpected process, not adherence to an agile process.

At one end is traditional waterfall, exemplified by an up-front plan and minimum adaptation to changing conditions. It can have high efficiency—provided the requirements never change and the software is understood perfectly.

At the other end is the ad-hoc process (bet you thought it would be agile at this end!). With ad-hoc process, everytime someone asks, you can just drop what you are doing and handle it. There is no process constraining you. However, efficiency is extremely low. You are multitasking so often that you never get anything done.

Somewhere in between, but more towards high agility, are agile processes. This is something like the five trip,two package per trip, example above. The idea is that you fix an iteration length. During the iteration, programmers are left alone to work. At the start of every iteration is a 'change point.' This is a point where stakeholders can come in and change the course of an iteration. By choosing shorter iterations, you create more 'change points' for more flexibility. Or you can choose a longer iteration for more efficiency. I'll write more about choosing iteration length in a future post.

FINALLY: The answer

With that under our belt, we can finally decide what to do about the problem. The situation is that one of your developers needs to be taken off for a day to implement a small feature for a sale. Should you do it? Scrum says no disturbance in the middle of the sprint. Joel says that doesn't sound agile to him.

What to do?

The simple answer: Ask the managers to decide if the sale is important enough to delay the other project and take a developer off if it is.

The complex answer: Your average manager is not Joel Spolsky. Most managers are under the mistaken impression that there will be no impact on the original project (hey, its only one developer for a day right?). Not all requests are really all that important. Be clear about the cost to the original project, because this is often hidden to them. Then go ahead and accept the request, but be _very_ careful that the process does not degenerate into a ad-hoc-handle-every-change-every-day process.

This post is a part of the selected archive.