A case study in poor management

If you dont follow cricket, skip to the lessons to learn section below

India got thrashed by South Africa in the cricket match a couple of days ago. Well, no surprise there. The Indians have always struggled in South Africa. What was a surprise was that the BCCI (the Indian cricket board) vice-president, Shashank Manohar came out and said that if it were up to him, none of the players would be paid a single penny. Quoting:

Manohar said the board was not to be blamed for the team's poor performance. "We are administrators and we have done our utmost to provide the players with the best of facilities and support staff. It's the job of the players to deliver on the field, not ours. We at best can take corrective action and measures."

This is pathetic for a number of resons. If Mr.Manohar feels that the players are not good enough, the BCCI should take responsibility, because they are the ones who selected the players. If Mr.Manohar feels that their technique on fast pitches is suspect, the BCCI has to again take responsibility, because they prepare slow pitches for domestic games. When will the BCCI learn that you cannot 'get acclimatised' to fast pitches with one warm-up game, and that it takes years of playing on these pitches to learn to handle them? Mr.Manohar is trying to wash his hands off the matter and put all the blame on the players, when in fact, the majority of the responsibility falls on the BCCI.

Lessons to learn

This is a perfect case study of what not to do. Here are some lessons to be learnt.

How many times have you heard a manager complain that the people under him are not good enough? Pretty often I'm sure. Managers, remember that you hired the people. If they are not good enough, you should take responsibility for it. Now, it is perfectly okay to make hiring mistakes. Everyone does it. But never say that "the person is not good enough", say instead "I made a hiring mistake", and then move forward from there.

Sometimes you don't hire the people under you. Maybe you moved to a new team and have to work with the existing team members. Can you then say that the people are not good enough? No. As a manager, one of your responsibilities is to make sure that the right people are doing the right work. When you take over a team, the first step is to make sure you've got the right people, and take steps to correct the situation if you dont.

Here is a situation: You have a great developer whom you promote to a Project Manager. Management requires a different set of skills to development, and the same person who was so good with development now struggles with management. Do you find this situation familiar? Different situations require different skills. These skills dont magically appear on people as the situations come up. The manager should have an idea of the skills that will be required, and have a personal development plan to help the team members acquire the skills that they are missing. Most managers do not do any personal development and then blame the team member when a situation arrives that the person is not equipped to handle.

Another golden rule is that you always back your team. Within yourselves, you can sort out any issues that need sorting out, but to the external world, you are together. You celebrate together, and you face the music together. When the going gets tough, you face the music on behalf of the team. You never, ever, abandon the team. Yet, when facing the heat, we often see managers who turn round and push the blame on the team. If you stick with the team, you will earn trust and respect, otherwise the team members will only remember you as the weasel who backstabbed them.

Management is unique because you are responsible for the results even though the actual work is done by someone else. A great manager helps others achieve results. The key word here is 'help'. Not 'forces', not 'commands', but 'helps'. How few managers there are who have that attitude. Jack Welch said it best — Before you become a leader success is all about growing yourself. Once you become a leader success is all about growing others.

This post is a part of the selected archive.

Selected archives available

Its almost three years since I started this blog. Thats a long time. I was going through my archives when I realised that its impossible for anyone to find the good posts amidst all the other posts. So I sat down and collected some of my favourite posts into a selected archive page. Check it out.

Flexibility vs Efficiency

This post by Dmitri kicked off a mini-debate about agility and rigidity. The debate is about whether a developer should be interrupted for a day to perfrom some side job. Check out the posts linked above for the whole story.

This is a common enough issue that it requires further examination.

At the root of the issue is what I call the flexibility-efficiency dilemma. This tradeoff is, in fact, at the root of many issues of agile vs traditional processes. The essence is that if you want more flexibility, you trade in less efficiency and vice versa.

Example

You have ten packages that you want to send by post. Each package involves buying the cartons, collecting the items, packing the items, writing out the address, filling up forms, and finally going to the post office and delivering them. There are two ways to approach this problem.

Approach 1

1. Buy ten cartons
2. Collect all the items
3. Pack the ten packages
4. Write out ten addresses
5. Fill up the ten forms
6. Go to the post office and drop off all ten packages

Approach 2

1. Buy one carton
2. Collect items for one carton
3. Pack the carton
4. Write out the address
5. Fill up the form
6. Drop off carton at post office
7. Buy another carton, and repeat the whole process ten times

Which approach is better? Most people would say the first one. Hmm. First, we need to define what we mean by better. Let's rephrase that question. Which one is more efficient? I think we can all agree on the first one. Its a no brainer. Just the ten trips to the post office makes the second approach way more inefficient. Let us say that it took ten days to send the ten packages via Approach 1 and fifteen days via Approach 2.

Now, which one is more flexible? Ah. This time its the second one. Why? Because it is much better suited to handle unexpected surprises.

I'm going to add in a surprise now. Surprise: After five days, you are informed that you need to go overseas for an office trip right now. Packages sent using Approach 1: Zero. Packages sent using Approach 2: Three.

Lets try another surprise. Surprise: It seems that customs is not letting such large packages go through. You need to repack into smaller cartons. Approach 1 impact: All your ten packages come back in two days. You repack everything for another ten days. It takes twenty days to get everything through. Approach 2 impact: After two days, your first package returns. You repack and resend it. The unpacked items are packed into smaller cartons the first time around. It takes seventeen days to get everything through.

Apart from the above approaches, are intermediate approaches. You could make five trips to the post office, sending two packages each time. The net result of efficiency and flexibility will be somewhere between the two extremes.

Software processes

Surprisingly (if you think about it, not such a surprise), the two approaches have a direct correlation with software processes. Just a couple of quick definitions. In the next paragraph, when I say 'agile', I mean an Agile (captial A) process. When I say 'agility', I mean ability to handle unexpected requests. So high agility mean high capacity to take on unexpected process, not adherence to an agile process.

At one end is traditional waterfall, exemplified by an up-front plan and minimum adaptation to changing conditions. It can have high efficiency—provided the requirements never change and the software is understood perfectly.

At the other end is the ad-hoc process (bet you thought it would be agile at this end!). With ad-hoc process, everytime someone asks, you can just drop what you are doing and handle it. There is no process constraining you. However, efficiency is extremely low. You are multitasking so often that you never get anything done.

Somewhere in between, but more towards high agility, are agile processes. This is something like the five trip,two package per trip, example above. The idea is that you fix an iteration length. During the iteration, programmers are left alone to work. At the start of every iteration is a 'change point.' This is a point where stakeholders can come in and change the course of an iteration. By choosing shorter iterations, you create more 'change points' for more flexibility. Or you can choose a longer iteration for more efficiency. I'll write more about choosing iteration length in a future post.

FINALLY: The answer

With that under our belt, we can finally decide what to do about the problem. The situation is that one of your developers needs to be taken off for a day to implement a small feature for a sale. Should you do it? Scrum says no disturbance in the middle of the sprint. Joel says that doesn't sound agile to him.

What to do?

The simple answer: Ask the managers to decide if the sale is important enough to delay the other project and take a developer off if it is.

The complex answer: Your average manager is not Joel Spolsky. Most managers are under the mistaken impression that there will be no impact on the original project (hey, its only one developer for a day right?). Not all requests are really all that important. Be clear about the cost to the original project, because this is often hidden to them. Then go ahead and accept the request, but be _very_ careful that the process does not degenerate into a ad-hoc-handle-every-change-every-day process.

This post is a part of the selected archive.

Leaving today

I'll be leaving Singapore today at 2035 SGT, arriving in Chennai at 2200 IST. Only an hour to go before I leave for the airport.

Returning to Chennai

Last Friday was my last day of work here in Singapore. I will be returning to India sometime this week. I've been in Singapore for four and a half awesome years. So why am I returning? I just get this feeling in me that a lot of cool stuff is going to happen in India. So far, India has only been seen as a cheap destination for outsourcing, but that is starting to change. There are a lot of startups, interesting conferences and user groups that are being formed. I could feel the change in the environment when I was in Bangalore a year ago. Very different from when I was there four years ago.

Yesterday, I found that the second Barcamp Bangalore is going to be held on December 2-3. This is exactly what I mean. In the last year there have been Barcamps in virtually all the major cities. Add in stuff like the Agile India conferences, Proto.in, TieCon India.. the list goes on. Can you imagine that the Bangalore Python yahoo group has 230 members? A few years ago, I would have stuggled to find ten people interested in a language that is outside of what is used at work. Very cool. Its for stuff like this that I'm coming back.

Spree River

Spree River
Originally uploaded by Siddhi.

Berlin: Friederichstraße S-Bahn (Metro) Station over the Spree river

Do we need an Apgar score for software?

Daniel Read has a very thought provoking post on whether software needs an Apgar score.

First, I encourage you to read both the original New Yorker article and Daniel's post.

What is the Apgar score? If you don't know, you didn't read the article, so go back and read it. Okay, okay :) The Apgar score is a method used to determine the health of a baby at childbirth. The baby is scored between 0–2 on five factors: heart rate, breathing, muscle tone, skin irritability and skin colour, giving a total score between 0–10. The score is measured once at 1 minute after birth and again at 5 minutes after birth. A score of 7 or more is a healthy baby. Less than seven does not necessarily indicate an unhealthy baby, but doctors will look at the baby to be sure. Before the score was devised, determining the baby's state was a complex mass of subjective measures which often meant that the baby was not given treatment when required because some factor was overlooked.

The key idea behind the Apgar score is that it is simple. It is simple to calculate. It is simple in that it condenses a number of subjective factors into one number. It is simple in that an instant decision can be made as to whether the baby is healthy. The final Apgar score does not take into account the myriad complexities, but it is intentionally kept simple so that it can be used as a quick, shorthand method of assessing the baby's health. That's what it is—a quick, shorthand assesment.

Daniel asks a question—Do we need an Apgar score for software development? Something that can serve as a quick, shorthand assesment of the project's health. Current methods of determining a project's health are a complex mass of subjectivity. Maintainability? Security? Relaiability? Code coverage? Usability? Requirement satisfaction? Customer satisfaction? There is no effective means of quickly measuring the project quality in a way that would give a rough idea as to the quality of the software. An Apgar score could help a lot here. It need not be exact. It just needs to be somewhere in the ballpark so that a decision can be made as to whether the project is healthy or we need to investigate more.

I think this is an interesting idea. It may not be practical as yet for software, but is nevertheless thought provoking. Check out some of the comments on Daniel's post as well.

This post is a part of the selected archive.

Twenty five posts on agile

I was re-organising my bookmarks yesterday and I decided to collect some of them into a post. Here then, in no particular order, are twenty five posts on agile:

1. Ten-Minute Build
2. The Next Big Thing
3. Knowing when you're done
4. The Coad Letter: Strategy Edition, Issue 40, Working Together -- Lessons Learned From Fred
5. An XP Team Room
6. Using Continuous Integration? Better do the "Check In Dance"
7. The Satir Change Model
8. Being an effective Onsite Customer or Product Owner
9. Agile team room photos
10. Scrum Tools Roundup
11. Agile Adoption across the Industry
12. Testing: Coverage Reports Considered Dangerous
13. It's Not Just Standing Up: Patterns for Daily Stand-up Meetings
14. Risk To Pattern Table
15. Buildix: The Agile development platform on a disk.
16. Dysfunctional Agile, Agile-in-the-Large
17. Project risk reduction patterns
18. Sending People Home
19. Agile - Losing Sight of the Goal?
20. Agile, Top Down
21. No Best Practices
22. Baby steps for your project in the new year
23. A Pragmatic Project: Live In Concert
24. Badge of Merit
25. Unjust Deserts? [PDF]

This post is a part of the selected archive.

Proto.in, Chennai, January 20-21, 2007

What is Proto.in?

Proto.in is positioning to be something like the DEMO conference. From the Proto.in blog:

One question that came up over and over again, was the question of What this event is supposed to achieve and what is the goal that we are aiming for? It’s simple. We are not just an outsourcing destination. We have innovation happening and we just want to shout that out to the world, so that the investments are varied and will also take that into account.

The event is scheduled for the dates of January 20th and 21st. We are narrowing down on the venues and the final list of companies will be 30 of them, with ten minute slots to present and make their pitch. The three questions that basically need answering are:

1. Who are you?
2. What do you do?
3. Why do you want the money? or What can you do with more money?

This is a pretty exciting time to be in India. Lots of cool stuff happening.

Agile in the Communications of the ACM

I got my October 2006 issue of the Communications of the ACM in the post today. The featured topic of this issue is "Flexible and Distributed software processes", and a lot of articles feature agile software development, especially in the context of distributed software development. If you have ACM digital library access, you can read the articles online at the above link.

The leading article, Flexible and Distributed Software Processes: Old Petunias in New Bowls?" (ACM Digital Library membership required) has some interesting commentaries by well known experts. Although about global software development and distributed teams, much of it is also applicable to co-located teams. Here are some snippets

David Parnas: I have been hearing the term "software crisis" for more than 40 years. Clearly, it is not a crisis; it is a chronic problem. Each time someone uses the term "crisis," it is a preface to the announcement of a new miracle cure for the problem. Examinations of the cure usually reveal new words for ideas the have been tried before without much effect. This decade's crisis is global software development (GSD); this decade's miracle cure (for all the ills of the industry) seems to be a collection of methods known as "agile."

It is fashionable to speak of "grand challenges." The real grand challenge is not to find ways to to avoid producing documentation, but to find ways to produce useful documents—documents that take time but save more time. We will find that real agility comes from good design that is well documented in precise, lean documentation.

Barry Boehm: There are no one-size-fits-all solutions. The best way I have been able to find is to use risk as a way to determine where to go agile and where to go document-driven. Thus, for example, if you are developing a graphic user interface (GUI) for an unprecedented decision support system and want to document its requirements, the most frequent answer you will get from users is, "I can't tell you in advance, but I'll know it when I see it (IKIWISI)."

In such a case, it is a high risk to try to document the GUI in advance, and with a GUI builder tool, it is a low risk not to document it.

On the other hand, when you are outsourcing a relatively stable piece of business logic to a contractor 10 time zones away, it is a high risk not to invest in a significant amount of thorough documentation of the interfaces and protocols connecting the outsourced sofware to the rest of your software.

Giancarlo Succi: No one thinks that analysis and design are useless. But consider a system to dispatch tracing messages and other information to a group of trucks. This domain is likely to be alien to most software developers. In such a case, would it be better to first spend a lot of time doing the upfront design, and eventually writing the code, or to work incrementally, involving the end customer, interleaving some analysis, design, and even coding, so that the developers grow their knowledge of the system domain?

Matthew Simons: While the idea of story cards appeals to those with no desire for reading or writing technical documentation, the reality of the situation is that such scanty artifacts are rarely sufficient for development of anything beyond simple systems for highly accessible individuals or small groups of users. This scenario rarely applies to GSD. As the picture becomes more complex, we have found the story card-only approach quickly becomes inadequate.

While writing effective documentation is a good place to start, we have found it takes a lot more than that to deliver complex systems with distributed teams. Without the discipline that comes from the remaining agile practices (such as Test-Driven Development, Continuous Integration, Pair Programming, among others) good documents are nothing more than a step on the long and perilous path toward successful delivery.

Like I said, the quotes above are only snippets. The actual commentaries include a lot more context and information than I have provided here. Read the whole article if you can, it is well worth it.

This post is a part of the selected archive.

Agile is not a set of techniques

A few weeks ago, Steve Yegge had a post on Good Agile, Bad Agile which caused a huge flutter in the blogosphere. The reactions on the Extreme Programming list went from "this is the usual 'I don't understand agile, so I'll bash it' crap" to "listen to your enemies. They'll tell you things about yourself that your friends never will." with a whole bunch in between. The post even turned up on Slashdot and Joel. Wow. Steve then posted a followup titled Egomania itself.

I personally think that both the articles have some good parts and some bad parts. He correctly points out that agilists can sometimes be loud and dogmatic. This is true and I've written about it before: Agile is not XP.

Having said that, I really do think that agile principles from the Agile Manifesto have a lot of value. Most of the problems associated with agile come from the 5 pitfalls list. Just to summarize, here is the list in short:

1. Unfamiliarity
2. Top-down thinking
3. Culture change
4. Incomplete implementation
5. Silver bullet syndrome

When something starts to get popular, there are any number of managers who want to implement it to get immediate results. "This book says that if we write requirements on index cards instead of an excel sheet, we will be more successful." Obviously that is nonsense, but for a number of people, that is the take away message from agile. "Do X and you will be more successful," where X can be test first, pair programming, daily standups or what have you. You then have the manager forcing the team to have standups everyday, its 45 minutes long (Standups are supposed to be short, not more than 15 mins), and naturally everyone just hates it. The result? The engineers hate it, and the manager is disillusioned.

Don't obsess over techniques. If pair programming doesn't work for you, that's perfectly fine. Don't do it. Do what works. Maybe you are having a lot of success with daily standups. Continue to do it. A confession: I don't do pair programming either.

Techniques do not exist in isolation. There are a number of factors that contribute to the success of a technique: the people, the environment, the culture, the management. A technique might work great for Google, but be terrible in your environment. A technique that is terrible for Google might work great for NASA.

So if you are not going to focus on techniques, what do you focus on? I prefer to focus on principles. Being agile is not about writing requirements on index cards or doing test first development. To me, being agile is about valuing people, working collaboratively, improving continuously and delivering consistently.

Now, that is a lot easier said than done. A comment on Steve's post goes like this (the company mentioned is Google):

I worked on a team (at the same company as you) where we were essentially "forced" to use Scrum. I don't think a single engineer on our team found it useful, and I personally found it annoying, but we didn't really have a recourse for evaluation: I gave feedback that I didn't like it, but I'm not sure that everyone else was comfortable doing the same, and there weren't any of these wonderful double blind weekly evals. I also think its results were mis-used. The data that was derived from it (how much our team could accomplish in a week) was tainted and it shouldn't have been turned around and used to make decisions.

If the engineers didn't find it useful, the practices should have been changed. Continuous improvement of the process by the team is a cornerstone of agile. This is a perfect example of following the techniques and ignoring the principles.

Agile is not a set of techniques. It is the principles that matter more than the techniques. Focus on the principles, and do the techniques that work in your environment.

This post is a part of the selected archive.

Stairs

Stairs
Originally uploaded by Siddhi.

Near my office, late at night

Techcrunch on PayPerPost

A year and a half ago, I wrote a post, pointing to this cartoon on corporate blogging. I had said

While its meant to be a bit funny, I can see the day when something like this actually happens.

A couple of days ago, Techcrunch had a post about the controversial PayPerPost, where bloggers are paid to post product reviews and the company can mandate that they be positive. This kind of thing is nothing new, as it happens all the time in mainstream media (what, you though all those magazine articles were unbiased? See this), but it has generated a lot of controversy for 'tainting the unbiased nature of the blogosphere', whatever that means. Check out the comments on the Techcrunch page.

Bug Bash

Bug Bash is a techie comic strip that I've been following for a long time. It's absolutely hilarious, so if you haven't heard of it yet, take a look here.

Export django database to an xml file

Here is a simple python script to export your django database to an XML file. I haven't tested it out very thoroughly. It seems to work for the fields that I have in my model - CharField, TextField, DateField, IntegerField, PositiveIntegerField and ForeignKey. If you find any bugs, add a comment to this post.

# setup the environment
import os, sys
sys.path.append(os.pardir)
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"

class XMLWriter:
    """Helper class to write out an xml file"""
    def __init__(self, pretty=True):
        """Set pretty to True if you want an indented XML file"""
        self.output = ""
        self.stack = []
        self.pretty = pretty
        
    def open(self, tag):
        """Add an open tag"""
        self.stack.append(tag)
        if self.pretty:
            self.output += "  "*(len(self.stack) - 1);
        self.output += "<" + tag + ">"
        if self.pretty:
            self.output += "\n"
        
    def close(self):
        """Close the innermost tag"""
        if self.pretty:
            self.output += "\n" + "  "*(len(self.stack) - 1);
        tag = self.stack.pop()
        self.output += "</" + tag + ">"
        if self.pretty:
            self.output += "\n"
        
    def closeAll(self):
        """Close all open tags"""
        while len(self.stack) > 0:
            self.close()
        
    def content(self, text):
        """Add some content"""
        if self.pretty:
            self.output += "  "*len(self.stack);
        self.output += str(text)

    def save(self, filename):
        """Save the data to a file"""
        self.closeAll()
        fp = open(filename, "w")
        fp.write(self.output)
        fp.close()

import django.db.models

writer = XMLWriter(pretty=False)
writer.open("djangoexport")
models = django.db.models.get_models()
for model in models:
    # model._meta.object_name holds the name of the model
    writer.open(model._meta.object_name + "s")
    for item in model.objects.all():
        writer.open(model._meta.object_name)
        for field in item._meta.fields:
            writer.open(field.name)
            value = getattr(item, field.name)
            if value != None:
                if isinstance(value, django.db.models.base.Model):
                    # This field is a foreign key, so save the primary key
                    # of the referring object
                    pk_name = value._meta.pk.name
                    pk_value = getattr(value, pk_name)
                    writer.content(pk_value)
                else:
                    writer.content(value)
            writer.close()
        writer.close()
    writer.close()
writer.close()
writer.save("export.xml")

Note to self: Find a better way to colorize the source code

This post is a part of the selected archive.