Twenty five posts on agile

I was re-organising my bookmarks yesterday and I decided to collect some of them into a post. Here then, in no particular order, are twenty five posts on agile:

1. Ten-Minute Build
2. The Next Big Thing
3. Knowing when you're done
4. The Coad Letter: Strategy Edition, Issue 40, Working Together -- Lessons Learned From Fred
5. An XP Team Room
6. Using Continuous Integration? Better do the "Check In Dance"
7. The Satir Change Model
8. Being an effective Onsite Customer or Product Owner
9. Agile team room photos
10. Scrum Tools Roundup
11. Agile Adoption across the Industry
12. Testing: Coverage Reports Considered Dangerous
13. It's Not Just Standing Up: Patterns for Daily Stand-up Meetings
14. Risk To Pattern Table
15. Buildix: The Agile development platform on a disk.
16. Dysfunctional Agile, Agile-in-the-Large
17. Project risk reduction patterns
18. Sending People Home
19. Agile - Losing Sight of the Goal?
20. Agile, Top Down
21. No Best Practices
22. Baby steps for your project in the new year
23. A Pragmatic Project: Live In Concert
24. Badge of Merit
25. Unjust Deserts? [PDF]

This post is a part of the selected archive.

Proto.in, Chennai, January 20-21, 2007

What is Proto.in?

Proto.in is positioning to be something like the DEMO conference. From the Proto.in blog:

One question that came up over and over again, was the question of What this event is supposed to achieve and what is the goal that we are aiming for? It’s simple. We are not just an outsourcing destination. We have innovation happening and we just want to shout that out to the world, so that the investments are varied and will also take that into account.

The event is scheduled for the dates of January 20th and 21st. We are narrowing down on the venues and the final list of companies will be 30 of them, with ten minute slots to present and make their pitch. The three questions that basically need answering are:

1. Who are you?
2. What do you do?
3. Why do you want the money? or What can you do with more money?

This is a pretty exciting time to be in India. Lots of cool stuff happening.

Agile in the Communications of the ACM

I got my October 2006 issue of the Communications of the ACM in the post today. The featured topic of this issue is "Flexible and Distributed software processes", and a lot of articles feature agile software development, especially in the context of distributed software development. If you have ACM digital library access, you can read the articles online at the above link.

The leading article, Flexible and Distributed Software Processes: Old Petunias in New Bowls?" (ACM Digital Library membership required) has some interesting commentaries by well known experts. Although about global software development and distributed teams, much of it is also applicable to co-located teams. Here are some snippets

David Parnas: I have been hearing the term "software crisis" for more than 40 years. Clearly, it is not a crisis; it is a chronic problem. Each time someone uses the term "crisis," it is a preface to the announcement of a new miracle cure for the problem. Examinations of the cure usually reveal new words for ideas the have been tried before without much effect. This decade's crisis is global software development (GSD); this decade's miracle cure (for all the ills of the industry) seems to be a collection of methods known as "agile."

It is fashionable to speak of "grand challenges." The real grand challenge is not to find ways to to avoid producing documentation, but to find ways to produce useful documents—documents that take time but save more time. We will find that real agility comes from good design that is well documented in precise, lean documentation.

Barry Boehm: There are no one-size-fits-all solutions. The best way I have been able to find is to use risk as a way to determine where to go agile and where to go document-driven. Thus, for example, if you are developing a graphic user interface (GUI) for an unprecedented decision support system and want to document its requirements, the most frequent answer you will get from users is, "I can't tell you in advance, but I'll know it when I see it (IKIWISI)."

In such a case, it is a high risk to try to document the GUI in advance, and with a GUI builder tool, it is a low risk not to document it.

On the other hand, when you are outsourcing a relatively stable piece of business logic to a contractor 10 time zones away, it is a high risk not to invest in a significant amount of thorough documentation of the interfaces and protocols connecting the outsourced sofware to the rest of your software.

Giancarlo Succi: No one thinks that analysis and design are useless. But consider a system to dispatch tracing messages and other information to a group of trucks. This domain is likely to be alien to most software developers. In such a case, would it be better to first spend a lot of time doing the upfront design, and eventually writing the code, or to work incrementally, involving the end customer, interleaving some analysis, design, and even coding, so that the developers grow their knowledge of the system domain?

Matthew Simons: While the idea of story cards appeals to those with no desire for reading or writing technical documentation, the reality of the situation is that such scanty artifacts are rarely sufficient for development of anything beyond simple systems for highly accessible individuals or small groups of users. This scenario rarely applies to GSD. As the picture becomes more complex, we have found the story card-only approach quickly becomes inadequate.

While writing effective documentation is a good place to start, we have found it takes a lot more than that to deliver complex systems with distributed teams. Without the discipline that comes from the remaining agile practices (such as Test-Driven Development, Continuous Integration, Pair Programming, among others) good documents are nothing more than a step on the long and perilous path toward successful delivery.

Like I said, the quotes above are only snippets. The actual commentaries include a lot more context and information than I have provided here. Read the whole article if you can, it is well worth it.

This post is a part of the selected archive.

Agile is not a set of techniques

A few weeks ago, Steve Yegge had a post on Good Agile, Bad Agile which caused a huge flutter in the blogosphere. The reactions on the Extreme Programming list went from "this is the usual 'I don't understand agile, so I'll bash it' crap" to "listen to your enemies. They'll tell you things about yourself that your friends never will." with a whole bunch in between. The post even turned up on Slashdot and Joel. Wow. Steve then posted a followup titled Egomania itself.

I personally think that both the articles have some good parts and some bad parts. He correctly points out that agilists can sometimes be loud and dogmatic. This is true and I've written about it before: Agile is not XP.

Having said that, I really do think that agile principles from the Agile Manifesto have a lot of value. Most of the problems associated with agile come from the 5 pitfalls list. Just to summarize, here is the list in short:

1. Unfamiliarity
2. Top-down thinking
3. Culture change
4. Incomplete implementation
5. Silver bullet syndrome

When something starts to get popular, there are any number of managers who want to implement it to get immediate results. "This book says that if we write requirements on index cards instead of an excel sheet, we will be more successful." Obviously that is nonsense, but for a number of people, that is the take away message from agile. "Do X and you will be more successful," where X can be test first, pair programming, daily standups or what have you. You then have the manager forcing the team to have standups everyday, its 45 minutes long (Standups are supposed to be short, not more than 15 mins), and naturally everyone just hates it. The result? The engineers hate it, and the manager is disillusioned.

Don't obsess over techniques. If pair programming doesn't work for you, that's perfectly fine. Don't do it. Do what works. Maybe you are having a lot of success with daily standups. Continue to do it. A confession: I don't do pair programming either.

Techniques do not exist in isolation. There are a number of factors that contribute to the success of a technique: the people, the environment, the culture, the management. A technique might work great for Google, but be terrible in your environment. A technique that is terrible for Google might work great for NASA.

So if you are not going to focus on techniques, what do you focus on? I prefer to focus on principles. Being agile is not about writing requirements on index cards or doing test first development. To me, being agile is about valuing people, working collaboratively, improving continuously and delivering consistently.

Now, that is a lot easier said than done. A comment on Steve's post goes like this (the company mentioned is Google):

I worked on a team (at the same company as you) where we were essentially "forced" to use Scrum. I don't think a single engineer on our team found it useful, and I personally found it annoying, but we didn't really have a recourse for evaluation: I gave feedback that I didn't like it, but I'm not sure that everyone else was comfortable doing the same, and there weren't any of these wonderful double blind weekly evals. I also think its results were mis-used. The data that was derived from it (how much our team could accomplish in a week) was tainted and it shouldn't have been turned around and used to make decisions.

If the engineers didn't find it useful, the practices should have been changed. Continuous improvement of the process by the team is a cornerstone of agile. This is a perfect example of following the techniques and ignoring the principles.

Agile is not a set of techniques. It is the principles that matter more than the techniques. Focus on the principles, and do the techniques that work in your environment.

This post is a part of the selected archive.

Stairs

Stairs
Originally uploaded by Siddhi.

Near my office, late at night

Techcrunch on PayPerPost

A year and a half ago, I wrote a post, pointing to this cartoon on corporate blogging. I had said

While its meant to be a bit funny, I can see the day when something like this actually happens.

A couple of days ago, Techcrunch had a post about the controversial PayPerPost, where bloggers are paid to post product reviews and the company can mandate that they be positive. This kind of thing is nothing new, as it happens all the time in mainstream media (what, you though all those magazine articles were unbiased? See this), but it has generated a lot of controversy for 'tainting the unbiased nature of the blogosphere', whatever that means. Check out the comments on the Techcrunch page.

Bug Bash

Bug Bash is a techie comic strip that I've been following for a long time. It's absolutely hilarious, so if you haven't heard of it yet, take a look here.

Export django database to an xml file

Here is a simple python script to export your django database to an XML file. I haven't tested it out very thoroughly. It seems to work for the fields that I have in my model - CharField, TextField, DateField, IntegerField, PositiveIntegerField and ForeignKey. If you find any bugs, add a comment to this post.

# setup the environment
import os, sys
sys.path.append(os.pardir)
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"

class XMLWriter:
    """Helper class to write out an xml file"""
    def __init__(self, pretty=True):
        """Set pretty to True if you want an indented XML file"""
        self.output = ""
        self.stack = []
        self.pretty = pretty
        
    def open(self, tag):
        """Add an open tag"""
        self.stack.append(tag)
        if self.pretty:
            self.output += "  "*(len(self.stack) - 1);
        self.output += "<" + tag + ">"
        if self.pretty:
            self.output += "\n"
        
    def close(self):
        """Close the innermost tag"""
        if self.pretty:
            self.output += "\n" + "  "*(len(self.stack) - 1);
        tag = self.stack.pop()
        self.output += "</" + tag + ">"
        if self.pretty:
            self.output += "\n"
        
    def closeAll(self):
        """Close all open tags"""
        while len(self.stack) > 0:
            self.close()
        
    def content(self, text):
        """Add some content"""
        if self.pretty:
            self.output += "  "*len(self.stack);
        self.output += str(text)

    def save(self, filename):
        """Save the data to a file"""
        self.closeAll()
        fp = open(filename, "w")
        fp.write(self.output)
        fp.close()

import django.db.models

writer = XMLWriter(pretty=False)
writer.open("djangoexport")
models = django.db.models.get_models()
for model in models:
    # model._meta.object_name holds the name of the model
    writer.open(model._meta.object_name + "s")
    for item in model.objects.all():
        writer.open(model._meta.object_name)
        for field in item._meta.fields:
            writer.open(field.name)
            value = getattr(item, field.name)
            if value != None:
                if isinstance(value, django.db.models.base.Model):
                    # This field is a foreign key, so save the primary key
                    # of the referring object
                    pk_name = value._meta.pk.name
                    pk_value = getattr(value, pk_name)
                    writer.content(pk_value)
                else:
                    writer.content(value)
            writer.close()
        writer.close()
    writer.close()
writer.close()
writer.save("export.xml")

Note to self: Find a better way to colorize the source code

This post is a part of the selected archive.

Handling conflict

Conflict. It's that word again. We have all faced it and most of us would rather not discuss it, but sooner or later you will be faced with such a situation and it's helpful to be able to understand it. In this article, I will discuss what I understand by conflict, and how we can handle it.

Conflict - good or bad?

Let's face it. People are different - that's what makes us people rather than machines. People with different viewpoints will naturally see things differently, inevitably leading to conflict.

A complete lack of conflict represents a dangerous situation. One possibility is that everyone thinks and behaves the same way. This represents a lack of diversity in the group that can easily lead to not exploring alternate viewpoints and 'groupthink'. Another possibility is even worse - that team members are afraid to speak up, due to some underlying fear. Most commonly, those who present alternate viewpoints are told that they are not 'team players', and the person in question stops raising objections from then on.

To a certain extent then, conflict is essential. It represents the heterogeneity of the group and leads to the exploration of alternate viewpoints.

It is all too easy however, to get caught up in a situation where you are forever arguing with someone with no end in sight. This happens when good conflict degenerates into dysfunctional conflict. After a point, the arguments are no longer on the merits of each case, but tied into the egos of the participants. A decision at that point takes political connotations and is perceived as an attack against the 'loser' of the conflict, which in turn leads into rapidly deteriorating relationships between all those involved.

Thus we see that conflict can be both good and bad. If you have no conflicts at all, then that is a sign that you need more diversity. In the rest of this article I will only talk about dysfunctional conflict.

Alignment

The first step in conflict resolution is alignment. The most obvious form of conflict is when two individuals/teams/groups are working in opposite directions. There are two possible reasons for this. One reason is that the groups are deliberately sabotaging each other due to political reasons. If this is case, the conflict is just a symptom or some other root cause, and you need to go fix that root cause. You have bigger problems.

The other reason is where both teams think that they are doing the right thing, but because there is no common vision, they are going in opposite directions. Both are convinced that they are doing the right thing and wondering what is going on with the other group. The result is a conflict where both sides are convinced about their righteousness. Resolving these conflicts is simply a matter of aligning both groups to a common vision, which of course is easier said than done and is left as an exercise to the reader :)

Handling conflict

Even when you have two people working to the same goals, it is likely that there will be conflicts in the various ways to get to that target. How can these conflicts be handled?

Acknowledge the conflict: The first step is to acknowledge the conflict. Acknowledge both sides in the conflict. Listen to both sides and make sure that they both get heard. The worst possible course of action is to pretend that a conflict does not exist or ignore it and hope that it goes away. First, it will not go away. Second, people want to be heard. It makes then feel empowered and feel that they are valued. Ignoring the individuals in the conflict sends a message that you don't care for their issues. Simply acknowledging the conflict says that you care about what they are feeling.

Separate the problem from the person: The next step is to separate the problem from the person. In any conflict, both sides have something invested in their position, which ties in their viewpoint with their ego. As long as that exists, it is impossible to solve the problem without making it seem personal. By separating the problem from the person, you can attack the problem without attacking the person.

Attack the problem together: Having got the problem in front of you, you can all discuss the problem together. By solving the problem together, you ensure that both parties will find the final solution acceptable. This is in contrast to mandating a solution, where one party might not like the solution but might accept it in order to close the issue. Remember, resolving a conflict is not about closing the issue, but about a solution that everyone feels good about. If one or both of the groups involved is not satisfied with the solution, you can be sure that the conflict is still simmering under the surface and will come out with more force the next time.

Example: How not to handle conflict

Amy: We need some way to bring new team members up to speed
Bob: We need some documentation
Chris: NO! We don't need more documentation

Amy: Chris, Bob has a lot of experience at this, maybe you should listen to him
Later...
Amy: Chris, you are always arguing with the team. You need to be a team player, not a disrupter
Chris: #$@!#$

The above is a case of attacking the person. Chris is likely to be extremely unhappy at the 'resolution'. Relationships are going to go downhill.

Example: A better way

Amy: We need some way to bring new team members up to speed
Bob: We need some documentation
Chris: NO! We don't need more documentation

Amy: Why do you say that?
Chris: Nobody reads documentation
Amy: Okay, that could be a problem. Bob, what do you think?
Bob: Yes, but the current situation is even worse
Amy: What other alternatives can we think of?
Chris: How about mentoring?
Bob: I'm not sure. We don't have the time to mentor each new person
Chris: Mentoring is the only way to teach the details that get lost in documents
Amy: Maybe we can have a small light document that covers the core areas, and we have mentoring to transfer knowledge of the details?
Chris: Yeah, we could do that
Bob: Good idea

In this case, Amy acknowledges both the team members positions, then attacks the problem, not the person. This allows all three parties to work together as a team to solve the problem.

Handling deep conflict

In some cases, the conflict has already been going on for a while. Both sides have already developed deep mistrust for each other and have their positions firmly entrenched. Solving such a conflict is much harder. The same principles apply as above, but because of the mistrust between parties, they do not want to talk to each other. More effort is expended towards preserving their position than towards resolving the conflict. In such cases, the you will probably need to work with the parties individually, as they will probably not see eye to eye together. The key goal is to change the mindset from defending a position to working towards a resolution. Once that happens, the above principles can be used again.

Most deep conflicts are rooted in certain past events. By understanding the history, it is possible to understand the viewpoints of the two sides and bring them to a resolution.

Example of deep conflict

Developer: The management is out to squeeze every ounce of work out of us. They make us work overtime for no particular reason. I refuse to work this weekend.

Manager: The developers are so unreliable. They keep giving excuses about why something cannot be done. We need to monitor them closely and keep pushing them or nothing would get done.

Another example

Widget team: The application team gets all the support. All the good programmers are there. We never get any support from them. It's better not to work with them if possible.

Application team: The widget teams keep asking us for support. How are we to support them and still get our work done? We need to minimize interaction with those teams.

Conclusion

The next time you face a conflict, keep in mind these three guidelines

• Acknowledge the conflict
• Separate the problem from the person
• Attack the problem together

This post is a part of the selected archive.

Testing is...

James Bach provides a great definition:

Testing is the infinite process
of comparing the invisible
to the ambiguous
so as to avoid the unthinkable
happening to the anonymous

Keep a lookout for these languages

some articles are doing the rounds about the languages to learn, so I decided to add my two cents to the discussion. These are the languages that I have on my radar -

• Python: I've been using Python for my side projects for the last few years. Needless to say that I like it a lot.
• JavaScript: Essential if you are going to do anything related to browser client-side coding. With the popularity of AJAX, JavaScript is quickly becoming a language that you need to know.
• Ruby: Many of the ex-Java coders are moving to Ruby in a big way. The language has gotten another big push with the popularity of Rails. This is on my to-learn list.
• A functional language: Scheme and Haskell are popular choices. Even if you don't actually use it regularly, learning a functional language is great for completely changing the way you think about programming. One of those experiences was when I got hold of a Scheme interpreter and followed along SICP and HTDP. Both Scheme and Haskell are on my to-learn-more list.
• Erlang: This is on my to-learn list. I think concurrent programming is going to increase in importance and I've heard a lot about Erlang.
• Perl: I find it useful as glue in *nix environments, especially if I need to use it's awesome regex capabilities, but don't use it all that often. The syntax is a pain to remember.
• C/C++/Java/C#: Chances are that your organization uses one of these languages, so you need to know it. That's about the only reason I can think of. Java has some useful libraries and frameworks for it, so that's a plus. Not sure about C#.
• Maybe someday: Smalltalk !!

Factory factory factory

Hilarious spoof on over-engineered frameworks. I was literally rolling on the floor laughing, and the best part is that it is all just so true!

What skills do I need to be a good scrummaster?

Based on my experiences, my list would go like this -

Have empathy. This is the most important. Learn continuously. This is the second most important. Apart from these, you need

• Understanding of the organization and its culture
• Understanding of the team and its dynamics
• Understanding of the people on the team
• Facilitation skills
• Conflict resolution skills
• Ability to work with people
• Ability to coordinate with teams
• Ability to work with management
• Ability to solve problems and get things done

What does you list look like?

Steve Irwin Tribute

Why software processes are like exercise

Software processes are like exercise. Everyone knows that its the right thing to do, but most people don't do it.

Why don't more people exercise? Because its too hard and it involves too much change. You see, exercising involves things like 'determination' and 'discipline' and 'goals', and who has time for that? An entire industry has evolved around the fact that people would like to be fit but do not like to exercise. At one extreme is the quick fix solution - wear some gadget and get fit without doing anything. At the other extreme are the hardcore fitness people who spend hours in the gym and scoff at any fitness scheme that tries to water down exercise. In the middle are a whole lot of activities that try to make exercise more fun and meaningful. The idea is not to be the fittest that you can possibly be, but to be as fit as you can be, without feeling like giving up.

Software processes are exactly the same. Processes are the right thing to do, but its far too easy to not see results and give up in frustration. At one end are the silver bullet providers - do X and your project will deliver itself. At the other end are those who live and breathe process, the process gurus, the people who have paparazzi following them all around.. well ok, maybe no paparazzi.

The agile early adopters are high up on the agility scale. The mainstream following behind are low on this scale. The big challenge for the agile community is to transfer the benefits of agility to the mainstream.

One way to do that is to take a bunch of processes that are radically different from what they are used to doing and ask them to follow them all. Some will try it out, get overwhelmed by the change, and give up in frustration. Others will resist the change. Still others will quit and go with what the silver bullet providers have to offer. A few will cross over. Those who didnt make, well, the fault lies with them - they are weak and have no discipline, and no wonder their projects failed.

Another way is to provide a transition path from zero agility to full agility. Pushing companies to go all the way, all at once, just makes them give up completely. This is bad. Companies should adopt agility at the point that they are comfortable. The idea is to maximise benefit, without feeling like giving up. That way, companies know that they can take the next step, and get better results, if they want to, or else they can stay where they are comfortable if they so desire. Just like exercise.

What would this transition path look like? How would you define 50% agility? How would you reconcile this against different agile processes that advocate different things? What should a company be doing if they want to be semi-agile? I don't have answers to these questions yet. What do you think? Is this possible? I would love any comments and feedback.

This post is a part of the selected archive.