Proto.in registrations open

A few weeks ago, I joined the Proto.in organising team. When I knew that this event is going to happen in Chennai, I had to be a part of it.

For all those interested in attending Proto.in as attendees:

Registrations for Proto.in are open!

Yes, Proto.in is open for the general public, so if you are neither a VC nor a selected company, you can still attend. Just go to the above page, click Technologist, fill up the form and you're done. Attending Proto.in is not free — this is a serious event for serious people — but at Rs.500 it's a steal. If you are a student, there is a special rate of just Rs.250.

Think about it. For Rs.500, you get

1. To meet the entrepreneurs
2. To meet venture capitalists
3. To meet like minded people like yourself
4. ... all in one place!
5. And of course, see thirty companies present demos of their products

Our aim is to nurture the entrepreneurial and innovation ecosystem in India, and doing that means providing a forum where different people can interact. And if you are interested in technology, come and see what the upcoming trends are.

Here is the registration link again.

SVN Revision 200

Today I checked in revision 200 into the source control tool — SVN in this case. I'm particularly happy because it means that I'm checking in frequently at the right level of granularity.

Now, frequent checkins don't actually mean anything much, but as a rough rule of thumb it indicates that code is being changed in small, independent increments. This in turn is important because it is a measure of the flexibility of the design. When the design is inflexible, changes involve touching lots of code and may take a few days to complete and checkin. A flexibile design makes it possible to perform changes one at a time.

Just for fun, here are some statistics

Revision milestones

Revision	Date
1	September 17, 2006 (Start as a hobby project to learn Django and AJAX)
50	October 18, 2006
70	November 16, 2006 (Return to Chennai, start full time work)
100	November 21, 2006
150	December 15, 2006
200	December 27, 2006

Revisions per month

Month	Number of checkins
September	27
October	40
November	69
December	62

Catalyst

Update: For more on Catalyst, see the Catalyst home page.

This post is an overview of Catalyst, the project management tool that I'm currently writing. The reason I'm putting all this online is to a) get feedback and b) to clarify my own thinking. I wrote a small story where I explained my vision for Catalyst and another about my experiences developing Catalyst. This post will follow on from those.

What is Catalyst?

Catalyst is a tool to manage software development for small teams, distributed teams and agile teams.

Catalyst is designed to make project details visible to all those involved in the project so that everyone knows what is happening.

Aren't there hundreds of project management tools already?

YES! The funny thing is that they are not suited very well to small, agile, distributed software development. You would think that at least one would be suitable, but I couldn't find that one.

About six to eight months ago, when I was working in my previous company, I was struggling to coordinate everyone involved in the project. That's when I started investigating to see if there was any tool that was useful and I found that most of them were not.

Catalyst features

My requirements were simple.

1. Easy to install
2. Easy to use:
   • Could I create 20 tasks, estimate them and assign them to team members easily?
   • Could I see the tasks in progress and adjust the estimate easily?
3. Did it support distributed teams? If two teams were involved in the planning would changes made by one propagate to the others?
4. Did it support all the stakeholders in the project — Dev team, QA, project manager, management, customers and anyone else involved?

When I tried various tools, what I found was

• Many tools were hosted solutions only
• Some had server and database requirements that I didn't have (IIS + SQL Server, for example)
• Many were extremely complex. Some companies even provided training to use the tool. Thats too complex for me.
• Lots were unusable. The 20 task test was often extremely painful to do.
• Many were designed with large teams in mind
• There was no way to synchronise teams doing planning in different locations
• The tool was often meant for only the dev team to use, so managers or customers still didn't have visibility

That's when I decided to write my own tool. So how is Catalyst different?

1. It is dead easy to install. Catalyst does not need any web server, it has one built in. No need for a database engine, thats built in too! Just run one executable and the server starts and you are ready to use it!
2. Easy to backup or migrate. Have you ever tried moving tool data from one machine to another? In Catalyst, all the data is stored in one file. Backup that file to backup your data. To migrate, install Catalyst on a machine and copy the data file over.
3. Catalyst does only project management. Many tools do everything — project management, user management, bug tracking, requirements tracking, the works. Catalyst does only project management. Not only does it keep everything simple, but there are already good bug tracking tools around, so why reinvent bug tracking? Use the best tool for the job. Besides its not a good idea to ask people to migrate all their data from the existing bug database into your tool.
4. Web API for easy integration. Catalyst has a web services API through which you can access and change data. So if you want to write a script that displays the project status in your custom dashboard, you can do that.
5. Participant/Observer model. A login can be either a participant or an observer. Participants can view and edit project details. Observers can only view them. This allows you to give the team a participant login, while giving customers and managers an observer login. Now everyone can view and follow the project. The tool is no longer just for the dev team but supports all stakeholders involved.
6. Auto-synchronisation. If many team members are on a page and one changes some data, it gets automatically refreshed for everyone else. This is to support distributed teams doing planning.
7. Easy to use. In many tools, operations are arranged by function, so adding a task means going to "Requirements" and adding a task, then going to "Team" and assigning to a user, the clicking "Planning" and estimating it, and so on. In Catalyst, operations are arranged by usage patterns. Thus, the "Planning" screen allows you to add a task, assign, estimate and schedule it, all in one screen. The "Dashboard" screen shows you current iteration status, and allows you to update estimates on one screen (because those operations are usually done together just before or during a daily standup).

So that's a summary of what Catalyst is (at least what I'm hoping it will be). Any questions or feedback? Leave a comment here.

Using python decorators to implement guards

One cool technique that I learnt while going through the Django code was using python decorators to implement guards.

What are guards?

Take a look at this bit of pseudo-code

if condition1:
  if condition2:
    {do_something}
  else:
    return error2
else:
  return error1

This is a common pattern where you do something provided condition1 and condition2 are false. The problem with this code is that it is difficult to seperate out the core logic of the function contained in {do_something} and the error handling code in the rest of the function. Another disadvantage is that the condition is at the top of the function, while the failure action is at the bottom. This makes it difficult to correlate the condition with the failure action.

The solution is to refactor the code to use guards.

if not condition1:
  return error1

if not contidion2:
  return error2

{do_something}

Guards are the conditions at the top of the function. They act like security guards — If the condition passes you go through, otherwise you leave the function. It is now a lot easier to see the conditions and the failure actions, and you can easily identify the code logic block by just skipping past the guards.

Python decorators

Python has a decorator feature that allows you to modify the function that it is applied to. Here is an example:

def decorate(fn):
  def _decorate():
    print "before calling"
    fn()
    print "after calling"
  return _decorate

@decorate
def myfunction():
  print "in function"

What we have is a function myfunction that prints the string "in function". To this, we apply the decorator 'decorate' (denoted by the @ symbol). decorate is itself a function that takes one function as a parameter and returns another function. In this case, it takes fn as a parameter and returns _decorate. Everytime myfunction is called, it will actually call the returned function, in this case _decorate. _decorate prints "before calling", then it calls the original function, then prints "after calling". Here is the output

>>> myfunction()
before calling
in function
after calling

Implementing guards using decorators

We can now see how guards can be implemented using decorators. Let me take a real example that I've encountered — my admin page. My tool has an admin page. In order to access this page, you must be logged in, and you must be an admin. If you are not logged in, you need to be redirected to the login page. If you are not an admin, an error message should be displayed. This is how the code would normally have looked

# check for login
try:
  user = request.session["user"]
except KeyError:
  # redirect user to login page

# check for admin
if not user.isAdmin():
  # display error page

# show admin page
...

Here is a version using decorators

# login decorator checks whether the user is logged in
def login_required(fn):
  def _check(request, *args, **kwargs):
    try:
      user = request.session["user"]
    except KeyError:
      # redirect to login page
    # user is logged in, call the function
    return fn(args, kwargs)
  return _check

# admin decorator checks whether the user is an admin
def admin_required(fn):
  def _check(request, *args, **kwargs):
    user = request.session["user"]
    if not user.isAdmin():
      # return the error page
    # user is admin, call the function
    return fn(args, kwargs)
  return _check

@login_required
@admin_required
def admin(request):
  # show admin page
  ...

This is how it works. We first have the admin function. All it does is implement the admin code. We decorate it with the admin_required and login_required decorators. When the admin function is called, it first enters the login_required decorator function which checks for login. If the user is not logged in, it redirects to the login page, else it calls the function. The function passed to login_required is the admin_required decorated function. So if login passes, it calls the admin_required decorator, checks for admin. If the user is not an admin, it displays an error message, else calls the function, which in this case is the original admin function.

See how neatly the guards are separated from the core logic. The admin function contains only the core logic, while the list of guards is neatly arranged as decorators. This method also has another advantage — it is easy to apply the same guards to other functions. Take a look at this

# no decorators
def login(request):
  ...

@login_required
@admin_required
def admin(request):
  ...

@login_required
def dashboard(request):
  ...

@login_required
@project_permission_required
def view_project(request, project):
  ...

Now each function only implements the core logic, while all the guard logic is taken care of by the decorators. It is easy to see the core logic and easy to see the guard conditions applied for the function. If I want some other function to have a guard, I can just add a decorator to it without touching the core logic. Best of all, the code is self descriptive and very easy to read.

This post is a part of the selected archive.

Rapid Web App Development — My experiences while developing Catalyst

My talk at the barcamp was titled "Rapid Web App Development — My experiences while developing Catalyst". Catayst is the name of my project management tool. The presentation was created using a modified version of Eric Meyer's S5 presentation tool. The template allows you to create the presentation content in XHTML and style it using CSS. It's really awesome, and I feel a lot more comfortable with it than Powerpoint or OpenOffice. However, there is no place to upload it. Another problem of putting the slides online is that it lacks the voice over and discussion that actually took place. Since the slides by themselves are pretty meaningless, I've added some commentary below.

---

The goal: Write a web application with one person, one month of work to get to version one. That's what I'm trying to do with Catalyst. I've spent about 80 hours working part time on Catalyst. At that time, it was only a hobby project for me to learn Django, and AJAX. I've worked about 100 hours on it full time since I returned to India. I'm hoping to get to beta before the end of the year.

Catalyst is a project management tool for small, agile and distributed teams. If you have ever been frustrated with the lack of visibility into what everyone else is doing in the project, especially with multiple offices, then Catalyst is for you. But this presentation is not about Catalyst. It is about what I have learnt while attempting to develop an application in a month.

There are six points that I want to discuss, split into two areas. The first area is the philosophy, and the first point is that constraints are good.

The more constraints one imposes, the more one frees one's self. And the arbitrariness of the constraint serves only to obtain precision of execution — Igor Stravinsky

Most of us think of constraints as bad, something that limits our freedom, but constraints can be good. Take the example of the one month limit, an arbitrary constraint. Why develop version 1 in a month? Why not six months or a year? I could have chosen a year and done a lot more features, but the one month constraint forces me to select the most important features and implement them first. In this case, the constraint helps me to focus on what is essential. Constraints can also drive innovation and spur creativity.

Second is the 'less is more' philosophy. There are two angles to this. The first is that the fewer requirements you have, the less you have to design, code and test. In that sense, it obviously help you finish faster and get the product out. The other way of looking at it is simplicity. If you look at Catalyst, it does not have an integrated bug tracking system like many project management systems. Why? By eliminating bug tracking from the list of features, I retain the focus on project management. Not only does it simplify Catalyst, but the users can continue to use their favourite bug tracking tool. In my opinion, this combination is actually worth more than having integrated bug tracking within Catalyst.

The second area I want to focus on is execution. This is the actual development part. How do you speed it up? My favourite is the combination of expressive languages and powerful frameworks. Catalyst is written in python and django. Here is an example: In the project dashboard screen, we need to get all the tasks that are a part of the interation that is currently in progress. The python code to do this is just six lines, with no SQL to be written!

The fifth point is to do smart testing.

There are two ways to write error-free programs; only the third works. — Alan Perlis

You'll never write error free code and since there is only one person, you have to rely on automated tests. Javascript code is unit tested using jsUnit. I am looking to start using the Django extensions to python's unittest module for testing the server side code. Another tool I want to learn is Selenium for automated system tests. Of course manual tests have their place too. I am using catalyst to manage my development work, so that kind of dogfooding helps in areas like usability and finding bugs via basic exploratory testing.

Finally, use libraries! The less code you have to write, the easier it is. Catalyst uses Dojo and Mochikit on the client side. Dojo has a bunch of widgets that really ease development, while Mochikit has a really cool DOM creation API. Catalyst uses both. PIL is used on the server side to generate charts. Apart from this, a lot of repeated code has been refactored into libraries. This is general good practise of course, but its often not done. See how the constraints force you to develop better?

To summarise, here are the six points again

1. Constraints are good
2. Less is more
3. Expressive languages
4. Powerful frameworks
5. Smart testing
6. Libraries

Performance reviews reviewed

Harvard Business School's Working Knowledge newsletter has an article by James Heskett titled What's to Be Done About Performance Reviews? that is worth reading. James first tries to clarify why we have performance reviews in the first place:

Perhaps a more important issue is the objective of the review itself. Is it to weed out poor performers? To recognize the so-called A players? To provide the basis for compensation decisions? To provide clues to future opportunity within the organization? To map out an individual plan for personal development? All of these? Too often this is unclear. Is it any wonder then that managers, many of whom receive little or no training in how to do it, conduct the task of reviewing performance so poorly?

Most of the performance reviews that I have been involved in have not gone as well as planned. Why?

Their goals were not well defined: Why exactly were we conducting the review? The reviews were used to determine salary, promotions, past employee performance and future employee development. With virtually everything riding on one review, it is unsurprising that most of us hated it, much like a final exam at school.

They are too infrequent: A recurrent theme in the comments is that performance feedback should be a continuous process between the manager and the reports, not a once a year event. The feedback should be a conversation, not a test. Employees also need feedback at the time of the event, not many months after.

They are too backward looking: The reviews that I have been involved in have tended to be too backward looking. They spend far too much time analysing what happened in the past and not enough time about what should be happening in the future.

Some interesting comments. Jon Clemens says:

In my experience, the nature of reviews at most companies is overly burdened with administrative headaches involved with passing documents back and forth, knowing when things are due, and getting access to timely, relevant information. This in large part contributes to the standard practice of pain that is associated with many company's reviews.

Thad Juszczak says:

Too often I have seen managers tell employees "good job!" and the employees have interpreted that as "outstanding performance!" Managers should not be waiting for the once or twice a year period when feedback and performance results are formally recognized. If either the managers or employees are "surprised" during the formal review process, then someone was not doing their job or listening.

Anonymous says:

The most effective model I have had to implement was Deming-philosophy-based and assumed that everyone did a satisfactory job and did not tie performance to reward. However, we did add to that skill-based pay or project/performance goals (3-5) that would support the organization or department strategic initiatives or self-development. Also, good performers were able to get promoted. This is when we achieved the most growth in our department and organization; employees were put in control of whether or not they achieved their goal and therefore their reward. Performance issues were addressed in real time, not once a year at review time.

And finally, B.Greenspan says:

I have known many colleagues in other organizations where the review systems are essentially "fear based." How can a person get a helpful review when the manager is more fearful of his or her review?

I'll close up this post now. The original article has 93 comments plus a closing summary so do check it out.

The role of documentation in agile projects

A commonly misunderstood aspect of agile project management is the role of documentation in agile projects. The agile manifesto says:

Working software over comprehensive documentation

This is misunderstood to mean that agile projects do not have documentation. Discussions about agile often revolve around this point, and statements of the form "Agile says no documentation, so lets stop documenting and we'll be agile" are heard once in a while. At the other end is the "Agile says no documentation, so it's just ad-hoc coding" camp.

Agile processes are not about eliminating documents. Agile processes are about delivering working software. The line in the manifesto is a reference to waterfall processes.

A traditional waterfall project has phases: Requirements, Analysis, Design, Coding, Integration, Testing. For the sake of example, lets say that each phase takes a month to complete, so the whole project time is six months. Three months into the project, we have delivered the requirement spec and design documents, but no software. By the fourth stage we have some software but it's not integrated and hence unusable at this point. Only at the end of the sixth stage — the end of project — do we get any working software. In the meantime we have delivered lots of documents, but no software. Agile projects should deliver software right from the start instead of delivering only documents for five out of six months. That is what is meant by working software over comprehensive documentation. Deliver the software, not the documentation.

Having said that, agile teams also produce do less documentation internally. The rest of this article will explore why that is so.

There are two main purposes for documentation in a project:

1. Documents are a form of communication between the various parties involved in the project
2. Documents are a point of reference on the various aspects of the software system

A detailed spec is supposed to communicate exactly what the software is supposed to do, so that consulting the spec is as good as consulting the customer. A design spec is a form of communication between the designers and developers. A user manual is for communicating between the developers and the users. Documents can also be used for reference. If a new member joins the team, she can just read up the documents to figure out what is happening. The maintenance team can read the design doc to figure out the code.

Traditional processes are reliant on documents because it is the only form of communication. Once the spec is signed off, communication with the customers slows down to a trickle. Secondly, groups are isolated from each other. Developers never get to speak to the customers directly, so documents are the only way for them to learn what the customer is thinking.

Agile processes turn things around. Continuous communication with the customer is a key principle of agile projects. Since there is constant communication, the need for documentation-as-communication reduces. There is still a need for some lightweight documentation, but not anything too detailed.

Secondly, with frequent releases, communication can be around the software. What this means is that instead of looking at the requirements for reference, you simply make a release and get feedback from the customer.

Finally, agile teams are cross-functional. Groups that were previously isolated, such as analysts, testers and developers, sit and work together. Again, this increases the amount of communication between groups, thus reducing the need for documentation.

Not all projects have great levels of communication. These projects require more documentation. Other projects can make do with less documentation because communication is better.

We see that reducing documentation is not the cause, but the effect. Teams that reduce documentation in an effort to be agile are getting the cause and effect mixed up. The key is to adjust the communication level and match the documentation to your needs, not the other way around.

This post is a part of the selected archive.

Software Project Management — A short story

Since returning to Chennai, I've been working on a project management tool for small, distributed and agile teams. To illustrate my vision for the product, here is a small story. Do you find any part of the story familiar? Are there any characters that you can relate to? Maybe some of this stuff happens in your project. Solving this kind of problem is what I'm setting out to do.

---

The characters

Geoff (Male, 42)

Geoff is a manager at the LoansNow bank. He wants Quality Software Inc to develop an IT system to automate some processes. This is the third time that Geoff is trying to get this system built. The first two times were complete disasters and left a bad taste in Geoff's mouth.

Quote: “If I dont ask for lots of reports, I'll never know what the hell is happening to my project”

Susan (Female, 38)

Susan is a senior manager at QoS Inc. She spends most of her time soothing the nerves of her customers. ‘Yes, you will have your software on time and on budget,’ she tells them, but she doesn't believe it herself.

Quote: “The developers have gone dark again! What do I tell the customer now?”

Alex (Male, 31)

Alex is the project manager for the LoansNow project. Alex finds himself in a tough position, stuck between an irrational customer and the unpredictable developers.

Quote: “Hey! I just remembered something else that we need to do”

Barbara (Female, 29)

Barbara is a senior developer. She has years of experience developing solutions for a number of clients, right from when she was 15. She thinks that software development will be so much better if there were no customers.

Quote: “Not another change in the requirements!”

Kevin (Male, 30)

Kevin is a senior developer leading the second team in Chicago.

Quote: “Why does the other team always mess up?”

---

“Susan, I still haven't got my status report.” Geoff's voice thundered through the phone.

“Alex is working on it” I replied. “You'll have them by this evening.”

After the call, I stopped by Alex's cubicle. Alex is the project manager for this project. We call him the Scrummaster.

“How is the report going?” I asked.

“I'm almost done. Why does he want the reports every week? Susan, it's insane! I spend two days of the week preparing this damn report, and I'm not sure that he even understands what is on it.” Clearly, Alex isn't happy.

Both of us know that the time spent preparing the report can be better utilized in actually managing the project, but what are we to do? We are the third company that Geoff has given this project to. The previous two attempts were complete failures. Not only were they over the schedule, but the delivered system was unusable. Geoff had no idea what was going on till the system was delivered. Geoff is under pressure to make sure that the project goes through smoothly this time, and one thing he has learnt the hard way is that he needs visibility into the project.

I take a look at the report. It contains a bulleted list of tasks completed this week. Visibility. It's that ugly word again. I sympathize with Alex. After all, I spend a couple of days every week writing up a report to email to my boss back at headquarters. It's a pain, but they need to know what is happening.

“How is the project going?” I ask.

“Alex!” A voice rang out from the distance. It was Barbara, our technical lead. Barbara looked angry. She quickly walked down to Alex's cubicle.

“Alex, you won't believe it. It's the Chicago team again. Kevin didn't do the multiple edit feature. He thought that we were going to do it. I thought we made it clear in yesterday's call that it was reassigned to Kevin. Alex, this is crazy. We have a release every two weeks, and these mistakes take up valuable time.”

Alex looked at me. I knew what that look meant. Just a couple of releases ago, a vital feature was not implemented because of a misunderstanding between the teams.

“Let me think about this again,” I said and went back to my cubicle. This was a recurring problem and no solution had been found yet. I decided that writing things down might help. I got myself a coffee and headed to the conference room. Fortunately, it was empty. I took a marker and started to write:

• Communication problems
  • HQ needs to know how the projects are going (once a week)
  • Geoff needs to know what is happening (once a week)
  • Alex needs to keep track of the project (daily)
  • Barbara and Kevin need to synchronize their teams to the current project status (daily)
  • Any changes made by Barbara, Kevin or Alex needs to be propogated to everyone else on both teams (immediately)

Below that I list out our current practices and problems with them:

• Current practices
  • Weekly status report to Geoff
    • Takes two days a week out of Alex's schedule
  • Weekly status report to HQ
    
    • Takes a couple of days a week out of my schedule
  • Daily call between the team here and in Chicago
    
    • Great for status, but details tend to get missed out or misunderstood
  • Excel sheet maintained by Alex
    
    • Alex seems to be the only one who sees this, the rest of the team hardly open it
    • The situation is even worse in Chicago, they hardly ever open this file
    • Everyone thinks they know what they should be working on, but they don't synchronise with the spreadsheet
  • Chart maintained by Barbara's team
    • It's only visible to the team and anyone else who can physically walk there, which rules out Geoff, HQ and Kevin's team

I think for a while. Then I add another two lines below that

• Excel sheet is not visible enough
• Chart is not visible enough

Dang! It's that visibility word again. What a mess. I finish my coffee and look at the whiteboard again. "This is stupid," I think to myself. "Just look at the effort spent in synchronizing everyone and it doesn't even work very well".

I take the digital camera by the side of the whiteboard and take a picture of the board. I email it to my home account. I also take the opportunity to check my mail. There is a mail from Kevin. He is not happy either. He blames our team for the misunderstanding. He suggests that Alex mail out the final task sheet daily after the call. That sounds like overkill. I shut down my computer. It's been a tiring day.

Back home, I take a shower and cook my dinner. Once dinner is over, I take some time to think about the problem again. There is only one project, so why is it so hard to keep everyone in sync? Then it hits me. The DRY principle. The DRY principle is short for Don't Repeat Yourself. It makes sense that if there is only one project, there should be only one authoritative source of information concerning that project. Since everyone needs information about the same project, we can just point to this source and they can get information out of it. Why are we creating lots of different sources and reports and struggling to keep everything consistent?

I take out my notebook and make some notes:

• Keep project information DRY
  • The information should be accessible online. This is for those who cannot be physically present.
  • The information should cover the needs of the various parties. In this project, that would be Geoff, HQ, Alex, Barbara, Kevin, the two teams and myself.
  • The information should be easy to change. That will make sure everyone uses it.

I am on a roll now. I add few lines below that:

• As a side bonus, neither Alex nor I would have to prepare reports, leaving us free to concentrate on our work
• This is great. The tool will free us from doing this kind of silly work and let us concentrate on the things that really matter.

Only one question remains. Is such a tool already available or would we have to roll out our own?

Out of town till Monday

I'll be out of town until Monday (11 Dec). I'm not sure if I'll have Internet or Mobile access while I'm away. In case I dont have, I'll get back on Monday.

Bangalore Barcamp 2 Roundup

Here is a roundup of some of the events during the barcamp. There was so much stuff going on that there were quite a few times when I wanted to be in two or three places at the same time, and ended up missing some interesting stuff.

The first presentation was by Sandeep Singhal from Sequoia Capital. He talked about what they look for in the companies in which they invest, and proceeded to give ten points that they look for — addressing large markets, good team DNA, clarity of purpose, rich customers, insane customer focus, they are pain killers, think differently, agility, frugality and they disrupt the market.

It seems to me that VCs are pretty interested in investing. On the second day, Kiruba had a session on Proto.in. I had mentioned Proto.in previously in my blog, and its coming up next month in Chennai. It's a chance for entrepreneurs to pitch their ideas to a group of VCs in ten minutes by showing a demo/prototype.

There were a couple of talks on social entrepreneurship as well. These were based on Dr.C.K.Prahalad's book, "The fortune at the bottom of the pyramid". Harinath Pudipeddi discussed social entrepreneurship in the healthcare industry, while Siva Prasad spoke about microfinance.

By the way, there is a business plan competition for social entrepreneurship called Genesis which is being conducted by IIT Madras, MIT and a few others. The contest takes place through Jan and Feb. The schedule is available on the website.

Jonathan Boutelle had an interesting session on how they scaled Slideshare. Slideshare is a place where you can upload presentations and they share them, embed them in blogs and so on. It's positioned as YouTube for Powerpoint, and the whole thing was done using Ruby on Rails and Amazon S3.

It was done with just a few people (less than 10) in Delhi in six months. Another cool thing is that they serve the flash content directly from S3. This means that if there is a huge spike in viewing presentations, all that traffic is handled by Amazon. In fact, even if the slideshare site goes down, you can still view presentations embedded in blogs, because they come straight from Amazon.

A theme that I noticed was that there were quite a few companies using Amazon services. It was something that I didn't expect. Apparently, a number of photo sharing sites use the S3 service to store their data. I met Manish from Picsquare and he said that they stored all their high-res photos on S3. He also mentioned that some other companies kept their data backups on S3! I had no idea that Amazon services were so popular.

Aditya Mishra from TCS had a session on innovation in large IT services company, where he made an interesting point about what innovation means for a services company. After all, someone else gives the requirements, and you deliver for those requirements, so what does it mean to be innovative? He gave a number of examples where TCS has been innovative with business models and development processes. He made a point that the entire outsourcing industry is a result of TCS innovation in succeeding with outsourcing in the 1970s when no one believed that such a thing could be done.

There were a couple of discussions that I had about Agile. Marco Janson from Thoughtworks had an introductory session on Agile, but I also had some interesting coversations with a lot of people from thoughtworks, as well as Sowmya from Aditi. There were some more sessions on agile, mostly by various people from thoughtworks.

There were a few Python related sessions. I attended one by Anand Chitipothu on web.py. One of the interesting things was that Anand is doing a rewrite of Infogami. web.py was initially written by Aaron Swartz for Infogami.

Aashish Solanki had a session on his new site YBangalore.com, which is a site to have user generated content about Bangalore. There were a number of other sessions on other sites that have been developed or are in beta. I didn't attend many sessions on Day 2, as I was in the corridor discussions that were taking place. As a result I missed many sessions.

Another session from Day 1 was Jay Fichialos talking about how they implemented HackDay at Sabre. Sabre is the company that owns travelocity and a number of travel and hotel related sites. Jay got the idea for HackDay after talking to Chad Dickerson at the first Bangalore Barcamp. Chad had just finished the HackDay for Yahoo at that point. Jay then implemented HackDay at Sabre and it was a huge success.

Taylor Cowan, also from Sabre had a session on an experiment he is doing with Sabre Labs. Sabre has huge databases of information on hotels availability, car rentals, airports, flights and so on. The idea is to make all this data available via an open REST API. Once the API is released, developers will be able to take this data and create their own mashups. This is very exciting. The whole thing will launch at opencontenttravel.com sometime in mid-Jan.

Another interesting discussion was with Jace, Shreyas, Amit Pande and Harish about Bangalore culture and sociology.

Shreyas also had a presentation on RadioVerve, a site that presents streaming music of independent indian music. You should check it out.

I also met Manoj Govindan and Ravi Mohan in a discussion on cellphones, dungeons and dragons, and wargaming. I last met Ravi back at Agile India 2005.

Finally, I gave a presentation on the stuff that I'm currently working on. I'll elaborate more on this in the next post.

A big thanks to the organisers and sponsors who did an awesome job in setting up this barcamp!

Back from barcamp

Returned from Bangalore today. I was there over the weekend to attend the second Bangalore Barcamp (my first). It was really cool. Lots of interesting sessions, lots of interesting people. Expanded post coming up tomorrow.

A case study in poor management

If you dont follow cricket, skip to the lessons to learn section below

India got thrashed by South Africa in the cricket match a couple of days ago. Well, no surprise there. The Indians have always struggled in South Africa. What was a surprise was that the BCCI (the Indian cricket board) vice-president, Shashank Manohar came out and said that if it were up to him, none of the players would be paid a single penny. Quoting:

Manohar said the board was not to be blamed for the team's poor performance. "We are administrators and we have done our utmost to provide the players with the best of facilities and support staff. It's the job of the players to deliver on the field, not ours. We at best can take corrective action and measures."

This is pathetic for a number of resons. If Mr.Manohar feels that the players are not good enough, the BCCI should take responsibility, because they are the ones who selected the players. If Mr.Manohar feels that their technique on fast pitches is suspect, the BCCI has to again take responsibility, because they prepare slow pitches for domestic games. When will the BCCI learn that you cannot 'get acclimatised' to fast pitches with one warm-up game, and that it takes years of playing on these pitches to learn to handle them? Mr.Manohar is trying to wash his hands off the matter and put all the blame on the players, when in fact, the majority of the responsibility falls on the BCCI.

Lessons to learn

This is a perfect case study of what not to do. Here are some lessons to be learnt.

How many times have you heard a manager complain that the people under him are not good enough? Pretty often I'm sure. Managers, remember that you hired the people. If they are not good enough, you should take responsibility for it. Now, it is perfectly okay to make hiring mistakes. Everyone does it. But never say that "the person is not good enough", say instead "I made a hiring mistake", and then move forward from there.

Sometimes you don't hire the people under you. Maybe you moved to a new team and have to work with the existing team members. Can you then say that the people are not good enough? No. As a manager, one of your responsibilities is to make sure that the right people are doing the right work. When you take over a team, the first step is to make sure you've got the right people, and take steps to correct the situation if you dont.

Here is a situation: You have a great developer whom you promote to a Project Manager. Management requires a different set of skills to development, and the same person who was so good with development now struggles with management. Do you find this situation familiar? Different situations require different skills. These skills dont magically appear on people as the situations come up. The manager should have an idea of the skills that will be required, and have a personal development plan to help the team members acquire the skills that they are missing. Most managers do not do any personal development and then blame the team member when a situation arrives that the person is not equipped to handle.

Another golden rule is that you always back your team. Within yourselves, you can sort out any issues that need sorting out, but to the external world, you are together. You celebrate together, and you face the music together. When the going gets tough, you face the music on behalf of the team. You never, ever, abandon the team. Yet, when facing the heat, we often see managers who turn round and push the blame on the team. If you stick with the team, you will earn trust and respect, otherwise the team members will only remember you as the weasel who backstabbed them.

Management is unique because you are responsible for the results even though the actual work is done by someone else. A great manager helps others achieve results. The key word here is 'help'. Not 'forces', not 'commands', but 'helps'. How few managers there are who have that attitude. Jack Welch said it best — Before you become a leader success is all about growing yourself. Once you become a leader success is all about growing others.

This post is a part of the selected archive.

Selected archives available

Its almost three years since I started this blog. Thats a long time. I was going through my archives when I realised that its impossible for anyone to find the good posts amidst all the other posts. So I sat down and collected some of my favourite posts into a selected archive page. Check it out.

Flexibility vs Efficiency

This post by Dmitri kicked off a mini-debate about agility and rigidity. The debate is about whether a developer should be interrupted for a day to perfrom some side job. Check out the posts linked above for the whole story.

This is a common enough issue that it requires further examination.

At the root of the issue is what I call the flexibility-efficiency dilemma. This tradeoff is, in fact, at the root of many issues of agile vs traditional processes. The essence is that if you want more flexibility, you trade in less efficiency and vice versa.

Example

You have ten packages that you want to send by post. Each package involves buying the cartons, collecting the items, packing the items, writing out the address, filling up forms, and finally going to the post office and delivering them. There are two ways to approach this problem.

Approach 1

1. Buy ten cartons
2. Collect all the items
3. Pack the ten packages
4. Write out ten addresses
5. Fill up the ten forms
6. Go to the post office and drop off all ten packages

Approach 2

1. Buy one carton
2. Collect items for one carton
3. Pack the carton
4. Write out the address
5. Fill up the form
6. Drop off carton at post office
7. Buy another carton, and repeat the whole process ten times

Which approach is better? Most people would say the first one. Hmm. First, we need to define what we mean by better. Let's rephrase that question. Which one is more efficient? I think we can all agree on the first one. Its a no brainer. Just the ten trips to the post office makes the second approach way more inefficient. Let us say that it took ten days to send the ten packages via Approach 1 and fifteen days via Approach 2.

Now, which one is more flexible? Ah. This time its the second one. Why? Because it is much better suited to handle unexpected surprises.

I'm going to add in a surprise now. Surprise: After five days, you are informed that you need to go overseas for an office trip right now. Packages sent using Approach 1: Zero. Packages sent using Approach 2: Three.

Lets try another surprise. Surprise: It seems that customs is not letting such large packages go through. You need to repack into smaller cartons. Approach 1 impact: All your ten packages come back in two days. You repack everything for another ten days. It takes twenty days to get everything through. Approach 2 impact: After two days, your first package returns. You repack and resend it. The unpacked items are packed into smaller cartons the first time around. It takes seventeen days to get everything through.

Apart from the above approaches, are intermediate approaches. You could make five trips to the post office, sending two packages each time. The net result of efficiency and flexibility will be somewhere between the two extremes.

Software processes

Surprisingly (if you think about it, not such a surprise), the two approaches have a direct correlation with software processes. Just a couple of quick definitions. In the next paragraph, when I say 'agile', I mean an Agile (captial A) process. When I say 'agility', I mean ability to handle unexpected requests. So high agility mean high capacity to take on unexpected process, not adherence to an agile process.

At one end is traditional waterfall, exemplified by an up-front plan and minimum adaptation to changing conditions. It can have high efficiency—provided the requirements never change and the software is understood perfectly.

At the other end is the ad-hoc process (bet you thought it would be agile at this end!). With ad-hoc process, everytime someone asks, you can just drop what you are doing and handle it. There is no process constraining you. However, efficiency is extremely low. You are multitasking so often that you never get anything done.

Somewhere in between, but more towards high agility, are agile processes. This is something like the five trip,two package per trip, example above. The idea is that you fix an iteration length. During the iteration, programmers are left alone to work. At the start of every iteration is a 'change point.' This is a point where stakeholders can come in and change the course of an iteration. By choosing shorter iterations, you create more 'change points' for more flexibility. Or you can choose a longer iteration for more efficiency. I'll write more about choosing iteration length in a future post.

FINALLY: The answer

With that under our belt, we can finally decide what to do about the problem. The situation is that one of your developers needs to be taken off for a day to implement a small feature for a sale. Should you do it? Scrum says no disturbance in the middle of the sprint. Joel says that doesn't sound agile to him.

What to do?

The simple answer: Ask the managers to decide if the sale is important enough to delay the other project and take a developer off if it is.

The complex answer: Your average manager is not Joel Spolsky. Most managers are under the mistaken impression that there will be no impact on the original project (hey, its only one developer for a day right?). Not all requests are really all that important. Be clear about the cost to the original project, because this is often hidden to them. Then go ahead and accept the request, but be _very_ careful that the process does not degenerate into a ad-hoc-handle-every-change-every-day process.

This post is a part of the selected archive.

Leaving today

I'll be leaving Singapore today at 2035 SGT, arriving in Chennai at 2200 IST. Only an hour to go before I leave for the airport.

Returning to Chennai

Last Friday was my last day of work here in Singapore. I will be returning to India sometime this week. I've been in Singapore for four and a half awesome years. So why am I returning? I just get this feeling in me that a lot of cool stuff is going to happen in India. So far, India has only been seen as a cheap destination for outsourcing, but that is starting to change. There are a lot of startups, interesting conferences and user groups that are being formed. I could feel the change in the environment when I was in Bangalore a year ago. Very different from when I was there four years ago.

Yesterday, I found that the second Barcamp Bangalore is going to be held on December 2-3. This is exactly what I mean. In the last year there have been Barcamps in virtually all the major cities. Add in stuff like the Agile India conferences, Proto.in, TieCon India.. the list goes on. Can you imagine that the Bangalore Python yahoo group has 230 members? A few years ago, I would have stuggled to find ten people interested in a language that is outside of what is used at work. Very cool. Its for stuff like this that I'm coming back.

Spree River

Spree River
Originally uploaded by Siddhi.

Berlin: Friederichstraße S-Bahn (Metro) Station over the Spree river

Do we need an Apgar score for software?

Daniel Read has a very thought provoking post on whether software needs an Apgar score.

First, I encourage you to read both the original New Yorker article and Daniel's post.

What is the Apgar score? If you don't know, you didn't read the article, so go back and read it. Okay, okay :) The Apgar score is a method used to determine the health of a baby at childbirth. The baby is scored between 0–2 on five factors: heart rate, breathing, muscle tone, skin irritability and skin colour, giving a total score between 0–10. The score is measured once at 1 minute after birth and again at 5 minutes after birth. A score of 7 or more is a healthy baby. Less than seven does not necessarily indicate an unhealthy baby, but doctors will look at the baby to be sure. Before the score was devised, determining the baby's state was a complex mass of subjective measures which often meant that the baby was not given treatment when required because some factor was overlooked.

The key idea behind the Apgar score is that it is simple. It is simple to calculate. It is simple in that it condenses a number of subjective factors into one number. It is simple in that an instant decision can be made as to whether the baby is healthy. The final Apgar score does not take into account the myriad complexities, but it is intentionally kept simple so that it can be used as a quick, shorthand method of assessing the baby's health. That's what it is—a quick, shorthand assesment.

Daniel asks a question—Do we need an Apgar score for software development? Something that can serve as a quick, shorthand assesment of the project's health. Current methods of determining a project's health are a complex mass of subjectivity. Maintainability? Security? Relaiability? Code coverage? Usability? Requirement satisfaction? Customer satisfaction? There is no effective means of quickly measuring the project quality in a way that would give a rough idea as to the quality of the software. An Apgar score could help a lot here. It need not be exact. It just needs to be somewhere in the ballpark so that a decision can be made as to whether the project is healthy or we need to investigate more.

I think this is an interesting idea. It may not be practical as yet for software, but is nevertheless thought provoking. Check out some of the comments on Daniel's post as well.

This post is a part of the selected archive.

Twenty five posts on agile

I was re-organising my bookmarks yesterday and I decided to collect some of them into a post. Here then, in no particular order, are twenty five posts on agile:

1. Ten-Minute Build
2. The Next Big Thing
3. Knowing when you're done
4. The Coad Letter: Strategy Edition, Issue 40, Working Together -- Lessons Learned From Fred
5. An XP Team Room
6. Using Continuous Integration? Better do the "Check In Dance"
7. The Satir Change Model
8. Being an effective Onsite Customer or Product Owner
9. Agile team room photos
10. Scrum Tools Roundup
11. Agile Adoption across the Industry
12. Testing: Coverage Reports Considered Dangerous
13. It's Not Just Standing Up: Patterns for Daily Stand-up Meetings
14. Risk To Pattern Table
15. Buildix: The Agile development platform on a disk.
16. Dysfunctional Agile, Agile-in-the-Large
17. Project risk reduction patterns
18. Sending People Home
19. Agile - Losing Sight of the Goal?
20. Agile, Top Down
21. No Best Practices
22. Baby steps for your project in the new year
23. A Pragmatic Project: Live In Concert
24. Badge of Merit
25. Unjust Deserts? [PDF]

This post is a part of the selected archive.

Proto.in, Chennai, January 20-21, 2007

What is Proto.in?

Proto.in is positioning to be something like the DEMO conference. From the Proto.in blog:

One question that came up over and over again, was the question of What this event is supposed to achieve and what is the goal that we are aiming for? It’s simple. We are not just an outsourcing destination. We have innovation happening and we just want to shout that out to the world, so that the investments are varied and will also take that into account.

The event is scheduled for the dates of January 20th and 21st. We are narrowing down on the venues and the final list of companies will be 30 of them, with ten minute slots to present and make their pitch. The three questions that basically need answering are:

1. Who are you?
2. What do you do?
3. Why do you want the money? or What can you do with more money?

This is a pretty exciting time to be in India. Lots of cool stuff happening.

Agile in the Communications of the ACM

I got my October 2006 issue of the Communications of the ACM in the post today. The featured topic of this issue is "Flexible and Distributed software processes", and a lot of articles feature agile software development, especially in the context of distributed software development. If you have ACM digital library access, you can read the articles online at the above link.

The leading article, Flexible and Distributed Software Processes: Old Petunias in New Bowls?" (ACM Digital Library membership required) has some interesting commentaries by well known experts. Although about global software development and distributed teams, much of it is also applicable to co-located teams. Here are some snippets

David Parnas: I have been hearing the term "software crisis" for more than 40 years. Clearly, it is not a crisis; it is a chronic problem. Each time someone uses the term "crisis," it is a preface to the announcement of a new miracle cure for the problem. Examinations of the cure usually reveal new words for ideas the have been tried before without much effect. This decade's crisis is global software development (GSD); this decade's miracle cure (for all the ills of the industry) seems to be a collection of methods known as "agile."

It is fashionable to speak of "grand challenges." The real grand challenge is not to find ways to to avoid producing documentation, but to find ways to produce useful documents—documents that take time but save more time. We will find that real agility comes from good design that is well documented in precise, lean documentation.

Barry Boehm: There are no one-size-fits-all solutions. The best way I have been able to find is to use risk as a way to determine where to go agile and where to go document-driven. Thus, for example, if you are developing a graphic user interface (GUI) for an unprecedented decision support system and want to document its requirements, the most frequent answer you will get from users is, "I can't tell you in advance, but I'll know it when I see it (IKIWISI)."

In such a case, it is a high risk to try to document the GUI in advance, and with a GUI builder tool, it is a low risk not to document it.

On the other hand, when you are outsourcing a relatively stable piece of business logic to a contractor 10 time zones away, it is a high risk not to invest in a significant amount of thorough documentation of the interfaces and protocols connecting the outsourced sofware to the rest of your software.

Giancarlo Succi: No one thinks that analysis and design are useless. But consider a system to dispatch tracing messages and other information to a group of trucks. This domain is likely to be alien to most software developers. In such a case, would it be better to first spend a lot of time doing the upfront design, and eventually writing the code, or to work incrementally, involving the end customer, interleaving some analysis, design, and even coding, so that the developers grow their knowledge of the system domain?

Matthew Simons: While the idea of story cards appeals to those with no desire for reading or writing technical documentation, the reality of the situation is that such scanty artifacts are rarely sufficient for development of anything beyond simple systems for highly accessible individuals or small groups of users. This scenario rarely applies to GSD. As the picture becomes more complex, we have found the story card-only approach quickly becomes inadequate.

While writing effective documentation is a good place to start, we have found it takes a lot more than that to deliver complex systems with distributed teams. Without the discipline that comes from the remaining agile practices (such as Test-Driven Development, Continuous Integration, Pair Programming, among others) good documents are nothing more than a step on the long and perilous path toward successful delivery.

Like I said, the quotes above are only snippets. The actual commentaries include a lot more context and information than I have provided here. Read the whole article if you can, it is well worth it.

This post is a part of the selected archive.

Agile is not a set of techniques

A few weeks ago, Steve Yegge had a post on Good Agile, Bad Agile which caused a huge flutter in the blogosphere. The reactions on the Extreme Programming list went from "this is the usual 'I don't understand agile, so I'll bash it' crap" to "listen to your enemies. They'll tell you things about yourself that your friends never will." with a whole bunch in between. The post even turned up on Slashdot and Joel. Wow. Steve then posted a followup titled Egomania itself.

I personally think that both the articles have some good parts and some bad parts. He correctly points out that agilists can sometimes be loud and dogmatic. This is true and I've written about it before: Agile is not XP.

Having said that, I really do think that agile principles from the Agile Manifesto have a lot of value. Most of the problems associated with agile come from the 5 pitfalls list. Just to summarize, here is the list in short:

1. Unfamiliarity
2. Top-down thinking
3. Culture change
4. Incomplete implementation
5. Silver bullet syndrome

When something starts to get popular, there are any number of managers who want to implement it to get immediate results. "This book says that if we write requirements on index cards instead of an excel sheet, we will be more successful." Obviously that is nonsense, but for a number of people, that is the take away message from agile. "Do X and you will be more successful," where X can be test first, pair programming, daily standups or what have you. You then have the manager forcing the team to have standups everyday, its 45 minutes long (Standups are supposed to be short, not more than 15 mins), and naturally everyone just hates it. The result? The engineers hate it, and the manager is disillusioned.

Don't obsess over techniques. If pair programming doesn't work for you, that's perfectly fine. Don't do it. Do what works. Maybe you are having a lot of success with daily standups. Continue to do it. A confession: I don't do pair programming either.

Techniques do not exist in isolation. There are a number of factors that contribute to the success of a technique: the people, the environment, the culture, the management. A technique might work great for Google, but be terrible in your environment. A technique that is terrible for Google might work great for NASA.

So if you are not going to focus on techniques, what do you focus on? I prefer to focus on principles. Being agile is not about writing requirements on index cards or doing test first development. To me, being agile is about valuing people, working collaboratively, improving continuously and delivering consistently.

Now, that is a lot easier said than done. A comment on Steve's post goes like this (the company mentioned is Google):

I worked on a team (at the same company as you) where we were essentially "forced" to use Scrum. I don't think a single engineer on our team found it useful, and I personally found it annoying, but we didn't really have a recourse for evaluation: I gave feedback that I didn't like it, but I'm not sure that everyone else was comfortable doing the same, and there weren't any of these wonderful double blind weekly evals. I also think its results were mis-used. The data that was derived from it (how much our team could accomplish in a week) was tainted and it shouldn't have been turned around and used to make decisions.

If the engineers didn't find it useful, the practices should have been changed. Continuous improvement of the process by the team is a cornerstone of agile. This is a perfect example of following the techniques and ignoring the principles.

Agile is not a set of techniques. It is the principles that matter more than the techniques. Focus on the principles, and do the techniques that work in your environment.

This post is a part of the selected archive.

Stairs

Stairs
Originally uploaded by Siddhi.

Near my office, late at night

Techcrunch on PayPerPost

A year and a half ago, I wrote a post, pointing to this cartoon on corporate blogging. I had said

While its meant to be a bit funny, I can see the day when something like this actually happens.

A couple of days ago, Techcrunch had a post about the controversial PayPerPost, where bloggers are paid to post product reviews and the company can mandate that they be positive. This kind of thing is nothing new, as it happens all the time in mainstream media (what, you though all those magazine articles were unbiased? See this), but it has generated a lot of controversy for 'tainting the unbiased nature of the blogosphere', whatever that means. Check out the comments on the Techcrunch page.

Bug Bash

Bug Bash is a techie comic strip that I've been following for a long time. It's absolutely hilarious, so if you haven't heard of it yet, take a look here.

Export django database to an xml file

Here is a simple python script to export your django database to an XML file. I haven't tested it out very thoroughly. It seems to work for the fields that I have in my model - CharField, TextField, DateField, IntegerField, PositiveIntegerField and ForeignKey. If you find any bugs, add a comment to this post.

# setup the environment
import os, sys
sys.path.append(os.pardir)
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"

class XMLWriter:
    """Helper class to write out an xml file"""
    def __init__(self, pretty=True):
        """Set pretty to True if you want an indented XML file"""
        self.output = ""
        self.stack = []
        self.pretty = pretty
        
    def open(self, tag):
        """Add an open tag"""
        self.stack.append(tag)
        if self.pretty:
            self.output += "  "*(len(self.stack) - 1);
        self.output += "<" + tag + ">"
        if self.pretty:
            self.output += "\n"
        
    def close(self):
        """Close the innermost tag"""
        if self.pretty:
            self.output += "\n" + "  "*(len(self.stack) - 1);
        tag = self.stack.pop()
        self.output += "</" + tag + ">"
        if self.pretty:
            self.output += "\n"
        
    def closeAll(self):
        """Close all open tags"""
        while len(self.stack) > 0:
            self.close()
        
    def content(self, text):
        """Add some content"""
        if self.pretty:
            self.output += "  "*len(self.stack);
        self.output += str(text)

    def save(self, filename):
        """Save the data to a file"""
        self.closeAll()
        fp = open(filename, "w")
        fp.write(self.output)
        fp.close()

import django.db.models

writer = XMLWriter(pretty=False)
writer.open("djangoexport")
models = django.db.models.get_models()
for model in models:
    # model._meta.object_name holds the name of the model
    writer.open(model._meta.object_name + "s")
    for item in model.objects.all():
        writer.open(model._meta.object_name)
        for field in item._meta.fields:
            writer.open(field.name)
            value = getattr(item, field.name)
            if value != None:
                if isinstance(value, django.db.models.base.Model):
                    # This field is a foreign key, so save the primary key
                    # of the referring object
                    pk_name = value._meta.pk.name
                    pk_value = getattr(value, pk_name)
                    writer.content(pk_value)
                else:
                    writer.content(value)
            writer.close()
        writer.close()
    writer.close()
writer.close()
writer.save("export.xml")

Note to self: Find a better way to colorize the source code

This post is a part of the selected archive.

Handling conflict

Conflict. It's that word again. We have all faced it and most of us would rather not discuss it, but sooner or later you will be faced with such a situation and it's helpful to be able to understand it. In this article, I will discuss what I understand by conflict, and how we can handle it.

Conflict - good or bad?

Let's face it. People are different - that's what makes us people rather than machines. People with different viewpoints will naturally see things differently, inevitably leading to conflict.

A complete lack of conflict represents a dangerous situation. One possibility is that everyone thinks and behaves the same way. This represents a lack of diversity in the group that can easily lead to not exploring alternate viewpoints and 'groupthink'. Another possibility is even worse - that team members are afraid to speak up, due to some underlying fear. Most commonly, those who present alternate viewpoints are told that they are not 'team players', and the person in question stops raising objections from then on.

To a certain extent then, conflict is essential. It represents the heterogeneity of the group and leads to the exploration of alternate viewpoints.

It is all too easy however, to get caught up in a situation where you are forever arguing with someone with no end in sight. This happens when good conflict degenerates into dysfunctional conflict. After a point, the arguments are no longer on the merits of each case, but tied into the egos of the participants. A decision at that point takes political connotations and is perceived as an attack against the 'loser' of the conflict, which in turn leads into rapidly deteriorating relationships between all those involved.

Thus we see that conflict can be both good and bad. If you have no conflicts at all, then that is a sign that you need more diversity. In the rest of this article I will only talk about dysfunctional conflict.

Alignment

The first step in conflict resolution is alignment. The most obvious form of conflict is when two individuals/teams/groups are working in opposite directions. There are two possible reasons for this. One reason is that the groups are deliberately sabotaging each other due to political reasons. If this is case, the conflict is just a symptom or some other root cause, and you need to go fix that root cause. You have bigger problems.

The other reason is where both teams think that they are doing the right thing, but because there is no common vision, they are going in opposite directions. Both are convinced that they are doing the right thing and wondering what is going on with the other group. The result is a conflict where both sides are convinced about their righteousness. Resolving these conflicts is simply a matter of aligning both groups to a common vision, which of course is easier said than done and is left as an exercise to the reader :)

Handling conflict

Even when you have two people working to the same goals, it is likely that there will be conflicts in the various ways to get to that target. How can these conflicts be handled?

Acknowledge the conflict: The first step is to acknowledge the conflict. Acknowledge both sides in the conflict. Listen to both sides and make sure that they both get heard. The worst possible course of action is to pretend that a conflict does not exist or ignore it and hope that it goes away. First, it will not go away. Second, people want to be heard. It makes then feel empowered and feel that they are valued. Ignoring the individuals in the conflict sends a message that you don't care for their issues. Simply acknowledging the conflict says that you care about what they are feeling.

Separate the problem from the person: The next step is to separate the problem from the person. In any conflict, both sides have something invested in their position, which ties in their viewpoint with their ego. As long as that exists, it is impossible to solve the problem without making it seem personal. By separating the problem from the person, you can attack the problem without attacking the person.

Attack the problem together: Having got the problem in front of you, you can all discuss the problem together. By solving the problem together, you ensure that both parties will find the final solution acceptable. This is in contrast to mandating a solution, where one party might not like the solution but might accept it in order to close the issue. Remember, resolving a conflict is not about closing the issue, but about a solution that everyone feels good about. If one or both of the groups involved is not satisfied with the solution, you can be sure that the conflict is still simmering under the surface and will come out with more force the next time.

Example: How not to handle conflict

Amy: We need some way to bring new team members up to speed
Bob: We need some documentation
Chris: NO! We don't need more documentation

Amy: Chris, Bob has a lot of experience at this, maybe you should listen to him
Later...
Amy: Chris, you are always arguing with the team. You need to be a team player, not a disrupter
Chris: #$@!#$

The above is a case of attacking the person. Chris is likely to be extremely unhappy at the 'resolution'. Relationships are going to go downhill.

Example: A better way

Amy: We need some way to bring new team members up to speed
Bob: We need some documentation
Chris: NO! We don't need more documentation

Amy: Why do you say that?
Chris: Nobody reads documentation
Amy: Okay, that could be a problem. Bob, what do you think?
Bob: Yes, but the current situation is even worse
Amy: What other alternatives can we think of?
Chris: How about mentoring?
Bob: I'm not sure. We don't have the time to mentor each new person
Chris: Mentoring is the only way to teach the details that get lost in documents
Amy: Maybe we can have a small light document that covers the core areas, and we have mentoring to transfer knowledge of the details?
Chris: Yeah, we could do that
Bob: Good idea

In this case, Amy acknowledges both the team members positions, then attacks the problem, not the person. This allows all three parties to work together as a team to solve the problem.

Handling deep conflict

In some cases, the conflict has already been going on for a while. Both sides have already developed deep mistrust for each other and have their positions firmly entrenched. Solving such a conflict is much harder. The same principles apply as above, but because of the mistrust between parties, they do not want to talk to each other. More effort is expended towards preserving their position than towards resolving the conflict. In such cases, the you will probably need to work with the parties individually, as they will probably not see eye to eye together. The key goal is to change the mindset from defending a position to working towards a resolution. Once that happens, the above principles can be used again.

Most deep conflicts are rooted in certain past events. By understanding the history, it is possible to understand the viewpoints of the two sides and bring them to a resolution.

Example of deep conflict

Developer: The management is out to squeeze every ounce of work out of us. They make us work overtime for no particular reason. I refuse to work this weekend.

Manager: The developers are so unreliable. They keep giving excuses about why something cannot be done. We need to monitor them closely and keep pushing them or nothing would get done.

Another example

Widget team: The application team gets all the support. All the good programmers are there. We never get any support from them. It's better not to work with them if possible.

Application team: The widget teams keep asking us for support. How are we to support them and still get our work done? We need to minimize interaction with those teams.

Conclusion

The next time you face a conflict, keep in mind these three guidelines

• Acknowledge the conflict
• Separate the problem from the person
• Attack the problem together

This post is a part of the selected archive.

Testing is...

James Bach provides a great definition:

Testing is the infinite process
of comparing the invisible
to the ambiguous
so as to avoid the unthinkable
happening to the anonymous

Keep a lookout for these languages

some articles are doing the rounds about the languages to learn, so I decided to add my two cents to the discussion. These are the languages that I have on my radar -

• Python: I've been using Python for my side projects for the last few years. Needless to say that I like it a lot.
• JavaScript: Essential if you are going to do anything related to browser client-side coding. With the popularity of AJAX, JavaScript is quickly becoming a language that you need to know.
• Ruby: Many of the ex-Java coders are moving to Ruby in a big way. The language has gotten another big push with the popularity of Rails. This is on my to-learn list.
• A functional language: Scheme and Haskell are popular choices. Even if you don't actually use it regularly, learning a functional language is great for completely changing the way you think about programming. One of those experiences was when I got hold of a Scheme interpreter and followed along SICP and HTDP. Both Scheme and Haskell are on my to-learn-more list.
• Erlang: This is on my to-learn list. I think concurrent programming is going to increase in importance and I've heard a lot about Erlang.
• Perl: I find it useful as glue in *nix environments, especially if I need to use it's awesome regex capabilities, but don't use it all that often. The syntax is a pain to remember.
• C/C++/Java/C#: Chances are that your organization uses one of these languages, so you need to know it. That's about the only reason I can think of. Java has some useful libraries and frameworks for it, so that's a plus. Not sure about C#.
• Maybe someday: Smalltalk !!

Factory factory factory

Hilarious spoof on over-engineered frameworks. I was literally rolling on the floor laughing, and the best part is that it is all just so true!

What skills do I need to be a good scrummaster?

Based on my experiences, my list would go like this -

Have empathy. This is the most important. Learn continuously. This is the second most important. Apart from these, you need

• Understanding of the organization and its culture
• Understanding of the team and its dynamics
• Understanding of the people on the team
• Facilitation skills
• Conflict resolution skills
• Ability to work with people
• Ability to coordinate with teams
• Ability to work with management
• Ability to solve problems and get things done

What does you list look like?

Steve Irwin Tribute

Why software processes are like exercise

Software processes are like exercise. Everyone knows that its the right thing to do, but most people don't do it.

Why don't more people exercise? Because its too hard and it involves too much change. You see, exercising involves things like 'determination' and 'discipline' and 'goals', and who has time for that? An entire industry has evolved around the fact that people would like to be fit but do not like to exercise. At one extreme is the quick fix solution - wear some gadget and get fit without doing anything. At the other extreme are the hardcore fitness people who spend hours in the gym and scoff at any fitness scheme that tries to water down exercise. In the middle are a whole lot of activities that try to make exercise more fun and meaningful. The idea is not to be the fittest that you can possibly be, but to be as fit as you can be, without feeling like giving up.

Software processes are exactly the same. Processes are the right thing to do, but its far too easy to not see results and give up in frustration. At one end are the silver bullet providers - do X and your project will deliver itself. At the other end are those who live and breathe process, the process gurus, the people who have paparazzi following them all around.. well ok, maybe no paparazzi.

The agile early adopters are high up on the agility scale. The mainstream following behind are low on this scale. The big challenge for the agile community is to transfer the benefits of agility to the mainstream.

One way to do that is to take a bunch of processes that are radically different from what they are used to doing and ask them to follow them all. Some will try it out, get overwhelmed by the change, and give up in frustration. Others will resist the change. Still others will quit and go with what the silver bullet providers have to offer. A few will cross over. Those who didnt make, well, the fault lies with them - they are weak and have no discipline, and no wonder their projects failed.

Another way is to provide a transition path from zero agility to full agility. Pushing companies to go all the way, all at once, just makes them give up completely. This is bad. Companies should adopt agility at the point that they are comfortable. The idea is to maximise benefit, without feeling like giving up. That way, companies know that they can take the next step, and get better results, if they want to, or else they can stay where they are comfortable if they so desire. Just like exercise.

What would this transition path look like? How would you define 50% agility? How would you reconcile this against different agile processes that advocate different things? What should a company be doing if they want to be semi-agile? I don't have answers to these questions yet. What do you think? Is this possible? I would love any comments and feedback.

This post is a part of the selected archive.

5 dangers when adopting agile processes - and what to do about them

The last couple of years have seen agile software development 'cross the chasm' from the early adopters to the mainstream. This is great news for agile, but there are some dangers lurking in the background. Here are five points that companies need to watch for when adopting agile processes

1. Unfamiliarity: When early adopters practice agile, they really understand what agile is all about. Mainstream companies are less likely to understand the principles of the process. They've heard about 'this agile thing' and want the same benefits on their projects.


2. Top-down thinking: Most organizations practice top-down management, where managers make decisions and staff follow them. Agile processes work bottom-up, where the team is empowered to take many decisions, and the team is responsible for changing the process. This can be uncomfortable for many managers.


3. Culture change: Agile processes not only demand a change in they way software is developed, but a change in culture. Agile processes value a culture of openness, cooperation and collaboration. The mantra is "get work done", instead of "cover your ass".


4. Incomplete implementation: Companies that implement agile processes sometimes change the process out of convenience, without an understanding of the process. 'We don't do retrospective meetings because they waste a day of work', or 'Everyone knows what to do, a daily standup is not required' are some forms of process change that are instituted under the guise of 'tailoring the process'. Tailoring the process is good if you know what you are doing, but can lead to disaster if it is done just for convenience.


5. Silver bullet syndrome: Agile processes are not a silver bullet. They will not magically deliver your software, cure all ills and create world peace. There are many components to a successful project, and the process is just one of them. You still need a good team to do the work - in fact a good team is even more important in an agile process than in traditional processes. There are many components that determine project success or failure - management support, effectiveness of the process, quality of the team, familiarity with the domain and technology, to name a few. Agile can help with the process, but don't ignore the other components.

Right, so what can you do about it? Here are three ideas to help you get started with agile

• Learn about agile: The best thing to do is to learn about agile. I mean really learn about it, not read a single article in a magazine about how agile is the next big thing. There are lots of great online resources for learning. Check out agile websites, blogs covering agile or agile groups on Yahoo! Groups. You will find lots enthusiastic people just waiting to help out.


• Start small: Start small, refine, repeat. Implementing an agile process is not something that can be done throughout the organization in one shot. Accept that the first few months will be unproductive as everyone comes to grips with agile, its culture and its values. Choose a small project as a prototype, and iteratively refine the process. When you are comfortable, move on to another project, then another.


• Examine the organization culture: A big stumbling block is reconciling the existing organization culture with the values of agile processes. Transitioning to agile involves change, and like all change it is easy to see things not work out at the start, get frustrated and return to known, comfortable ways of doing things. Take some time to learn the values of agile, and how it can be incorporated into the organization.

This post is a part of the selected archive.

Interesting way to sell your company

From TechCrunch:

Online calendering company Kiko.com has decided to call it quits and put the site up for auction on eBay. They are also offering to export or delete user accounts.

You know that the action in the web space is starting to get hot when everyone starts talking about startups that closed down.

Final batch of photos uploaded

I've uploaded the final batch of photos for now. The new France photos here (new photos are at the end, starting from the fireworks), and the fireworks photos here. The fireworks are from Singapore National Day 2003, France Bastille Day 2006, Singapore National Day 2006 and the Singapore Fireworks Festival 2006.