Proto.in registrations open

A few weeks ago, I joined the Proto.in organising team. When I knew that this event is going to happen in Chennai, I had to be a part of it.

For all those interested in attending Proto.in as attendees:

Registrations for Proto.in are open!

Yes, Proto.in is open for the general public, so if you are neither a VC nor a selected company, you can still attend. Just go to the above page, click Technologist, fill up the form and you're done. Attending Proto.in is not free — this is a serious event for serious people — but at Rs.500 it's a steal. If you are a student, there is a special rate of just Rs.250.

Think about it. For Rs.500, you get

1. To meet the entrepreneurs
2. To meet venture capitalists
3. To meet like minded people like yourself
4. ... all in one place!
5. And of course, see thirty companies present demos of their products

Our aim is to nurture the entrepreneurial and innovation ecosystem in India, and doing that means providing a forum where different people can interact. And if you are interested in technology, come and see what the upcoming trends are.

Here is the registration link again.

SVN Revision 200

Today I checked in revision 200 into the source control tool — SVN in this case. I'm particularly happy because it means that I'm checking in frequently at the right level of granularity.

Now, frequent checkins don't actually mean anything much, but as a rough rule of thumb it indicates that code is being changed in small, independent increments. This in turn is important because it is a measure of the flexibility of the design. When the design is inflexible, changes involve touching lots of code and may take a few days to complete and checkin. A flexibile design makes it possible to perform changes one at a time.

Just for fun, here are some statistics

Revision milestones

Revision	Date
1	September 17, 2006 (Start as a hobby project to learn Django and AJAX)
50	October 18, 2006
70	November 16, 2006 (Return to Chennai, start full time work)
100	November 21, 2006
150	December 15, 2006
200	December 27, 2006

Revisions per month

Month	Number of checkins
September	27
October	40
November	69
December	62

Catalyst

Update: For more on Catalyst, see the Catalyst home page.

This post is an overview of Catalyst, the project management tool that I'm currently writing. The reason I'm putting all this online is to a) get feedback and b) to clarify my own thinking. I wrote a small story where I explained my vision for Catalyst and another about my experiences developing Catalyst. This post will follow on from those.

What is Catalyst?

Catalyst is a tool to manage software development for small teams, distributed teams and agile teams.

Catalyst is designed to make project details visible to all those involved in the project so that everyone knows what is happening.

Aren't there hundreds of project management tools already?

YES! The funny thing is that they are not suited very well to small, agile, distributed software development. You would think that at least one would be suitable, but I couldn't find that one.

About six to eight months ago, when I was working in my previous company, I was struggling to coordinate everyone involved in the project. That's when I started investigating to see if there was any tool that was useful and I found that most of them were not.

Catalyst features

My requirements were simple.

1. Easy to install
2. Easy to use:
   • Could I create 20 tasks, estimate them and assign them to team members easily?
   • Could I see the tasks in progress and adjust the estimate easily?
3. Did it support distributed teams? If two teams were involved in the planning would changes made by one propagate to the others?
4. Did it support all the stakeholders in the project — Dev team, QA, project manager, management, customers and anyone else involved?

When I tried various tools, what I found was

• Many tools were hosted solutions only
• Some had server and database requirements that I didn't have (IIS + SQL Server, for example)
• Many were extremely complex. Some companies even provided training to use the tool. Thats too complex for me.
• Lots were unusable. The 20 task test was often extremely painful to do.
• Many were designed with large teams in mind
• There was no way to synchronise teams doing planning in different locations
• The tool was often meant for only the dev team to use, so managers or customers still didn't have visibility

That's when I decided to write my own tool. So how is Catalyst different?

1. It is dead easy to install. Catalyst does not need any web server, it has one built in. No need for a database engine, thats built in too! Just run one executable and the server starts and you are ready to use it!
2. Easy to backup or migrate. Have you ever tried moving tool data from one machine to another? In Catalyst, all the data is stored in one file. Backup that file to backup your data. To migrate, install Catalyst on a machine and copy the data file over.
3. Catalyst does only project management. Many tools do everything — project management, user management, bug tracking, requirements tracking, the works. Catalyst does only project management. Not only does it keep everything simple, but there are already good bug tracking tools around, so why reinvent bug tracking? Use the best tool for the job. Besides its not a good idea to ask people to migrate all their data from the existing bug database into your tool.
4. Web API for easy integration. Catalyst has a web services API through which you can access and change data. So if you want to write a script that displays the project status in your custom dashboard, you can do that.
5. Participant/Observer model. A login can be either a participant or an observer. Participants can view and edit project details. Observers can only view them. This allows you to give the team a participant login, while giving customers and managers an observer login. Now everyone can view and follow the project. The tool is no longer just for the dev team but supports all stakeholders involved.
6. Auto-synchronisation. If many team members are on a page and one changes some data, it gets automatically refreshed for everyone else. This is to support distributed teams doing planning.
7. Easy to use. In many tools, operations are arranged by function, so adding a task means going to "Requirements" and adding a task, then going to "Team" and assigning to a user, the clicking "Planning" and estimating it, and so on. In Catalyst, operations are arranged by usage patterns. Thus, the "Planning" screen allows you to add a task, assign, estimate and schedule it, all in one screen. The "Dashboard" screen shows you current iteration status, and allows you to update estimates on one screen (because those operations are usually done together just before or during a daily standup).

So that's a summary of what Catalyst is (at least what I'm hoping it will be). Any questions or feedback? Leave a comment here.

Using python decorators to implement guards

One cool technique that I learnt while going through the Django code was using python decorators to implement guards.

What are guards?

Take a look at this bit of pseudo-code

if condition1:
  if condition2:
    {do_something}
  else:
    return error2
else:
  return error1

This is a common pattern where you do something provided condition1 and condition2 are false. The problem with this code is that it is difficult to seperate out the core logic of the function contained in {do_something} and the error handling code in the rest of the function. Another disadvantage is that the condition is at the top of the function, while the failure action is at the bottom. This makes it difficult to correlate the condition with the failure action.

The solution is to refactor the code to use guards.

if not condition1:
  return error1

if not contidion2:
  return error2

{do_something}

Guards are the conditions at the top of the function. They act like security guards — If the condition passes you go through, otherwise you leave the function. It is now a lot easier to see the conditions and the failure actions, and you can easily identify the code logic block by just skipping past the guards.

Python decorators

Python has a decorator feature that allows you to modify the function that it is applied to. Here is an example:

def decorate(fn):
  def _decorate():
    print "before calling"
    fn()
    print "after calling"
  return _decorate

@decorate
def myfunction():
  print "in function"

What we have is a function myfunction that prints the string "in function". To this, we apply the decorator 'decorate' (denoted by the @ symbol). decorate is itself a function that takes one function as a parameter and returns another function. In this case, it takes fn as a parameter and returns _decorate. Everytime myfunction is called, it will actually call the returned function, in this case _decorate. _decorate prints "before calling", then it calls the original function, then prints "after calling". Here is the output

>>> myfunction()
before calling
in function
after calling

Implementing guards using decorators

We can now see how guards can be implemented using decorators. Let me take a real example that I've encountered — my admin page. My tool has an admin page. In order to access this page, you must be logged in, and you must be an admin. If you are not logged in, you need to be redirected to the login page. If you are not an admin, an error message should be displayed. This is how the code would normally have looked

# check for login
try:
  user = request.session["user"]
except KeyError:
  # redirect user to login page

# check for admin
if not user.isAdmin():
  # display error page

# show admin page
...

Here is a version using decorators

# login decorator checks whether the user is logged in
def login_required(fn):
  def _check(request, *args, **kwargs):
    try:
      user = request.session["user"]
    except KeyError:
      # redirect to login page
    # user is logged in, call the function
    return fn(args, kwargs)
  return _check

# admin decorator checks whether the user is an admin
def admin_required(fn):
  def _check(request, *args, **kwargs):
    user = request.session["user"]
    if not user.isAdmin():
      # return the error page
    # user is admin, call the function
    return fn(args, kwargs)
  return _check

@login_required
@admin_required
def admin(request):
  # show admin page
  ...

This is how it works. We first have the admin function. All it does is implement the admin code. We decorate it with the admin_required and login_required decorators. When the admin function is called, it first enters the login_required decorator function which checks for login. If the user is not logged in, it redirects to the login page, else it calls the function. The function passed to login_required is the admin_required decorated function. So if login passes, it calls the admin_required decorator, checks for admin. If the user is not an admin, it displays an error message, else calls the function, which in this case is the original admin function.

See how neatly the guards are separated from the core logic. The admin function contains only the core logic, while the list of guards is neatly arranged as decorators. This method also has another advantage — it is easy to apply the same guards to other functions. Take a look at this

# no decorators
def login(request):
  ...

@login_required
@admin_required
def admin(request):
  ...

@login_required
def dashboard(request):
  ...

@login_required
@project_permission_required
def view_project(request, project):
  ...

Now each function only implements the core logic, while all the guard logic is taken care of by the decorators. It is easy to see the core logic and easy to see the guard conditions applied for the function. If I want some other function to have a guard, I can just add a decorator to it without touching the core logic. Best of all, the code is self descriptive and very easy to read.

This post is a part of the selected archive.

Rapid Web App Development — My experiences while developing Catalyst

My talk at the barcamp was titled "Rapid Web App Development — My experiences while developing Catalyst". Catayst is the name of my project management tool. The presentation was created using a modified version of Eric Meyer's S5 presentation tool. The template allows you to create the presentation content in XHTML and style it using CSS. It's really awesome, and I feel a lot more comfortable with it than Powerpoint or OpenOffice. However, there is no place to upload it. Another problem of putting the slides online is that it lacks the voice over and discussion that actually took place. Since the slides by themselves are pretty meaningless, I've added some commentary below.

---

The goal: Write a web application with one person, one month of work to get to version one. That's what I'm trying to do with Catalyst. I've spent about 80 hours working part time on Catalyst. At that time, it was only a hobby project for me to learn Django, and AJAX. I've worked about 100 hours on it full time since I returned to India. I'm hoping to get to beta before the end of the year.

Catalyst is a project management tool for small, agile and distributed teams. If you have ever been frustrated with the lack of visibility into what everyone else is doing in the project, especially with multiple offices, then Catalyst is for you. But this presentation is not about Catalyst. It is about what I have learnt while attempting to develop an application in a month.

There are six points that I want to discuss, split into two areas. The first area is the philosophy, and the first point is that constraints are good.

The more constraints one imposes, the more one frees one's self. And the arbitrariness of the constraint serves only to obtain precision of execution — Igor Stravinsky

Most of us think of constraints as bad, something that limits our freedom, but constraints can be good. Take the example of the one month limit, an arbitrary constraint. Why develop version 1 in a month? Why not six months or a year? I could have chosen a year and done a lot more features, but the one month constraint forces me to select the most important features and implement them first. In this case, the constraint helps me to focus on what is essential. Constraints can also drive innovation and spur creativity.

Second is the 'less is more' philosophy. There are two angles to this. The first is that the fewer requirements you have, the less you have to design, code and test. In that sense, it obviously help you finish faster and get the product out. The other way of looking at it is simplicity. If you look at Catalyst, it does not have an integrated bug tracking system like many project management systems. Why? By eliminating bug tracking from the list of features, I retain the focus on project management. Not only does it simplify Catalyst, but the users can continue to use their favourite bug tracking tool. In my opinion, this combination is actually worth more than having integrated bug tracking within Catalyst.

The second area I want to focus on is execution. This is the actual development part. How do you speed it up? My favourite is the combination of expressive languages and powerful frameworks. Catalyst is written in python and django. Here is an example: In the project dashboard screen, we need to get all the tasks that are a part of the interation that is currently in progress. The python code to do this is just six lines, with no SQL to be written!

The fifth point is to do smart testing.

There are two ways to write error-free programs; only the third works. — Alan Perlis

You'll never write error free code and since there is only one person, you have to rely on automated tests. Javascript code is unit tested using jsUnit. I am looking to start using the Django extensions to python's unittest module for testing the server side code. Another tool I want to learn is Selenium for automated system tests. Of course manual tests have their place too. I am using catalyst to manage my development work, so that kind of dogfooding helps in areas like usability and finding bugs via basic exploratory testing.

Finally, use libraries! The less code you have to write, the easier it is. Catalyst uses Dojo and Mochikit on the client side. Dojo has a bunch of widgets that really ease development, while Mochikit has a really cool DOM creation API. Catalyst uses both. PIL is used on the server side to generate charts. Apart from this, a lot of repeated code has been refactored into libraries. This is general good practise of course, but its often not done. See how the constraints force you to develop better?

To summarise, here are the six points again

1. Constraints are good
2. Less is more
3. Expressive languages
4. Powerful frameworks
5. Smart testing
6. Libraries

Performance reviews reviewed

Harvard Business School's Working Knowledge newsletter has an article by James Heskett titled What's to Be Done About Performance Reviews? that is worth reading. James first tries to clarify why we have performance reviews in the first place:

Perhaps a more important issue is the objective of the review itself. Is it to weed out poor performers? To recognize the so-called A players? To provide the basis for compensation decisions? To provide clues to future opportunity within the organization? To map out an individual plan for personal development? All of these? Too often this is unclear. Is it any wonder then that managers, many of whom receive little or no training in how to do it, conduct the task of reviewing performance so poorly?

Most of the performance reviews that I have been involved in have not gone as well as planned. Why?

Their goals were not well defined: Why exactly were we conducting the review? The reviews were used to determine salary, promotions, past employee performance and future employee development. With virtually everything riding on one review, it is unsurprising that most of us hated it, much like a final exam at school.

They are too infrequent: A recurrent theme in the comments is that performance feedback should be a continuous process between the manager and the reports, not a once a year event. The feedback should be a conversation, not a test. Employees also need feedback at the time of the event, not many months after.

They are too backward looking: The reviews that I have been involved in have tended to be too backward looking. They spend far too much time analysing what happened in the past and not enough time about what should be happening in the future.

Some interesting comments. Jon Clemens says:

In my experience, the nature of reviews at most companies is overly burdened with administrative headaches involved with passing documents back and forth, knowing when things are due, and getting access to timely, relevant information. This in large part contributes to the standard practice of pain that is associated with many company's reviews.

Thad Juszczak says:

Too often I have seen managers tell employees "good job!" and the employees have interpreted that as "outstanding performance!" Managers should not be waiting for the once or twice a year period when feedback and performance results are formally recognized. If either the managers or employees are "surprised" during the formal review process, then someone was not doing their job or listening.

Anonymous says:

The most effective model I have had to implement was Deming-philosophy-based and assumed that everyone did a satisfactory job and did not tie performance to reward. However, we did add to that skill-based pay or project/performance goals (3-5) that would support the organization or department strategic initiatives or self-development. Also, good performers were able to get promoted. This is when we achieved the most growth in our department and organization; employees were put in control of whether or not they achieved their goal and therefore their reward. Performance issues were addressed in real time, not once a year at review time.

And finally, B.Greenspan says:

I have known many colleagues in other organizations where the review systems are essentially "fear based." How can a person get a helpful review when the manager is more fearful of his or her review?

I'll close up this post now. The original article has 93 comments plus a closing summary so do check it out.

The role of documentation in agile projects

A commonly misunderstood aspect of agile project management is the role of documentation in agile projects. The agile manifesto says:

Working software over comprehensive documentation

This is misunderstood to mean that agile projects do not have documentation. Discussions about agile often revolve around this point, and statements of the form "Agile says no documentation, so lets stop documenting and we'll be agile" are heard once in a while. At the other end is the "Agile says no documentation, so it's just ad-hoc coding" camp.

Agile processes are not about eliminating documents. Agile processes are about delivering working software. The line in the manifesto is a reference to waterfall processes.

A traditional waterfall project has phases: Requirements, Analysis, Design, Coding, Integration, Testing. For the sake of example, lets say that each phase takes a month to complete, so the whole project time is six months. Three months into the project, we have delivered the requirement spec and design documents, but no software. By the fourth stage we have some software but it's not integrated and hence unusable at this point. Only at the end of the sixth stage — the end of project — do we get any working software. In the meantime we have delivered lots of documents, but no software. Agile projects should deliver software right from the start instead of delivering only documents for five out of six months. That is what is meant by working software over comprehensive documentation. Deliver the software, not the documentation.

Having said that, agile teams also produce do less documentation internally. The rest of this article will explore why that is so.

There are two main purposes for documentation in a project:

1. Documents are a form of communication between the various parties involved in the project
2. Documents are a point of reference on the various aspects of the software system

A detailed spec is supposed to communicate exactly what the software is supposed to do, so that consulting the spec is as good as consulting the customer. A design spec is a form of communication between the designers and developers. A user manual is for communicating between the developers and the users. Documents can also be used for reference. If a new member joins the team, she can just read up the documents to figure out what is happening. The maintenance team can read the design doc to figure out the code.

Traditional processes are reliant on documents because it is the only form of communication. Once the spec is signed off, communication with the customers slows down to a trickle. Secondly, groups are isolated from each other. Developers never get to speak to the customers directly, so documents are the only way for them to learn what the customer is thinking.

Agile processes turn things around. Continuous communication with the customer is a key principle of agile projects. Since there is constant communication, the need for documentation-as-communication reduces. There is still a need for some lightweight documentation, but not anything too detailed.

Secondly, with frequent releases, communication can be around the software. What this means is that instead of looking at the requirements for reference, you simply make a release and get feedback from the customer.

Finally, agile teams are cross-functional. Groups that were previously isolated, such as analysts, testers and developers, sit and work together. Again, this increases the amount of communication between groups, thus reducing the need for documentation.

Not all projects have great levels of communication. These projects require more documentation. Other projects can make do with less documentation because communication is better.

We see that reducing documentation is not the cause, but the effect. Teams that reduce documentation in an effort to be agile are getting the cause and effect mixed up. The key is to adjust the communication level and match the documentation to your needs, not the other way around.

This post is a part of the selected archive.

Software Project Management — A short story

Since returning to Chennai, I've been working on a project management tool for small, distributed and agile teams. To illustrate my vision for the product, here is a small story. Do you find any part of the story familiar? Are there any characters that you can relate to? Maybe some of this stuff happens in your project. Solving this kind of problem is what I'm setting out to do.

---

The characters

Geoff (Male, 42)

Geoff is a manager at the LoansNow bank. He wants Quality Software Inc to develop an IT system to automate some processes. This is the third time that Geoff is trying to get this system built. The first two times were complete disasters and left a bad taste in Geoff's mouth.

Quote: “If I dont ask for lots of reports, I'll never know what the hell is happening to my project”

Susan (Female, 38)

Susan is a senior manager at QoS Inc. She spends most of her time soothing the nerves of her customers. ‘Yes, you will have your software on time and on budget,’ she tells them, but she doesn't believe it herself.

Quote: “The developers have gone dark again! What do I tell the customer now?”

Alex (Male, 31)

Alex is the project manager for the LoansNow project. Alex finds himself in a tough position, stuck between an irrational customer and the unpredictable developers.

Quote: “Hey! I just remembered something else that we need to do”

Barbara (Female, 29)

Barbara is a senior developer. She has years of experience developing solutions for a number of clients, right from when she was 15. She thinks that software development will be so much better if there were no customers.

Quote: “Not another change in the requirements!”

Kevin (Male, 30)

Kevin is a senior developer leading the second team in Chicago.

Quote: “Why does the other team always mess up?”

---

“Susan, I still haven't got my status report.” Geoff's voice thundered through the phone.

“Alex is working on it” I replied. “You'll have them by this evening.”

After the call, I stopped by Alex's cubicle. Alex is the project manager for this project. We call him the Scrummaster.

“How is the report going?” I asked.

“I'm almost done. Why does he want the reports every week? Susan, it's insane! I spend two days of the week preparing this damn report, and I'm not sure that he even understands what is on it.” Clearly, Alex isn't happy.

Both of us know that the time spent preparing the report can be better utilized in actually managing the project, but what are we to do? We are the third company that Geoff has given this project to. The previous two attempts were complete failures. Not only were they over the schedule, but the delivered system was unusable. Geoff had no idea what was going on till the system was delivered. Geoff is under pressure to make sure that the project goes through smoothly this time, and one thing he has learnt the hard way is that he needs visibility into the project.

I take a look at the report. It contains a bulleted list of tasks completed this week. Visibility. It's that ugly word again. I sympathize with Alex. After all, I spend a couple of days every week writing up a report to email to my boss back at headquarters. It's a pain, but they need to know what is happening.

“How is the project going?” I ask.

“Alex!” A voice rang out from the distance. It was Barbara, our technical lead. Barbara looked angry. She quickly walked down to Alex's cubicle.

“Alex, you won't believe it. It's the Chicago team again. Kevin didn't do the multiple edit feature. He thought that we were going to do it. I thought we made it clear in yesterday's call that it was reassigned to Kevin. Alex, this is crazy. We have a release every two weeks, and these mistakes take up valuable time.”

Alex looked at me. I knew what that look meant. Just a couple of releases ago, a vital feature was not implemented because of a misunderstanding between the teams.

“Let me think about this again,” I said and went back to my cubicle. This was a recurring problem and no solution had been found yet. I decided that writing things down might help. I got myself a coffee and headed to the conference room. Fortunately, it was empty. I took a marker and started to write:

• Communication problems
  • HQ needs to know how the projects are going (once a week)
  • Geoff needs to know what is happening (once a week)
  • Alex needs to keep track of the project (daily)
  • Barbara and Kevin need to synchronize their teams to the current project status (daily)
  • Any changes made by Barbara, Kevin or Alex needs to be propogated to everyone else on both teams (immediately)

Below that I list out our current practices and problems with them:

• Current practices
  • Weekly status report to Geoff
    • Takes two days a week out of Alex's schedule
  • Weekly status report to HQ
    
    • Takes a couple of days a week out of my schedule
  • Daily call between the team here and in Chicago
    
    • Great for status, but details tend to get missed out or misunderstood
  • Excel sheet maintained by Alex
    
    • Alex seems to be the only one who sees this, the rest of the team hardly open it
    • The situation is even worse in Chicago, they hardly ever open this file
    • Everyone thinks they know what they should be working on, but they don't synchronise with the spreadsheet
  • Chart maintained by Barbara's team
    • It's only visible to the team and anyone else who can physically walk there, which rules out Geoff, HQ and Kevin's team

I think for a while. Then I add another two lines below that

• Excel sheet is not visible enough
• Chart is not visible enough

Dang! It's that visibility word again. What a mess. I finish my coffee and look at the whiteboard again. "This is stupid," I think to myself. "Just look at the effort spent in synchronizing everyone and it doesn't even work very well".

I take the digital camera by the side of the whiteboard and take a picture of the board. I email it to my home account. I also take the opportunity to check my mail. There is a mail from Kevin. He is not happy either. He blames our team for the misunderstanding. He suggests that Alex mail out the final task sheet daily after the call. That sounds like overkill. I shut down my computer. It's been a tiring day.

Back home, I take a shower and cook my dinner. Once dinner is over, I take some time to think about the problem again. There is only one project, so why is it so hard to keep everyone in sync? Then it hits me. The DRY principle. The DRY principle is short for Don't Repeat Yourself. It makes sense that if there is only one project, there should be only one authoritative source of information concerning that project. Since everyone needs information about the same project, we can just point to this source and they can get information out of it. Why are we creating lots of different sources and reports and struggling to keep everything consistent?

I take out my notebook and make some notes:

• Keep project information DRY
  • The information should be accessible online. This is for those who cannot be physically present.
  • The information should cover the needs of the various parties. In this project, that would be Geoff, HQ, Alex, Barbara, Kevin, the two teams and myself.
  • The information should be easy to change. That will make sure everyone uses it.

I am on a roll now. I add few lines below that:

• As a side bonus, neither Alex nor I would have to prepare reports, leaving us free to concentrate on our work
• This is great. The tool will free us from doing this kind of silly work and let us concentrate on the things that really matter.

Only one question remains. Is such a tool already available or would we have to roll out our own?

Out of town till Monday

I'll be out of town until Monday (11 Dec). I'm not sure if I'll have Internet or Mobile access while I'm away. In case I dont have, I'll get back on Monday.

Bangalore Barcamp 2 Roundup

Here is a roundup of some of the events during the barcamp. There was so much stuff going on that there were quite a few times when I wanted to be in two or three places at the same time, and ended up missing some interesting stuff.

The first presentation was by Sandeep Singhal from Sequoia Capital. He talked about what they look for in the companies in which they invest, and proceeded to give ten points that they look for — addressing large markets, good team DNA, clarity of purpose, rich customers, insane customer focus, they are pain killers, think differently, agility, frugality and they disrupt the market.

It seems to me that VCs are pretty interested in investing. On the second day, Kiruba had a session on Proto.in. I had mentioned Proto.in previously in my blog, and its coming up next month in Chennai. It's a chance for entrepreneurs to pitch their ideas to a group of VCs in ten minutes by showing a demo/prototype.

There were a couple of talks on social entrepreneurship as well. These were based on Dr.C.K.Prahalad's book, "The fortune at the bottom of the pyramid". Harinath Pudipeddi discussed social entrepreneurship in the healthcare industry, while Siva Prasad spoke about microfinance.

By the way, there is a business plan competition for social entrepreneurship called Genesis which is being conducted by IIT Madras, MIT and a few others. The contest takes place through Jan and Feb. The schedule is available on the website.

Jonathan Boutelle had an interesting session on how they scaled Slideshare. Slideshare is a place where you can upload presentations and they share them, embed them in blogs and so on. It's positioned as YouTube for Powerpoint, and the whole thing was done using Ruby on Rails and Amazon S3.

It was done with just a few people (less than 10) in Delhi in six months. Another cool thing is that they serve the flash content directly from S3. This means that if there is a huge spike in viewing presentations, all that traffic is handled by Amazon. In fact, even if the slideshare site goes down, you can still view presentations embedded in blogs, because they come straight from Amazon.

A theme that I noticed was that there were quite a few companies using Amazon services. It was something that I didn't expect. Apparently, a number of photo sharing sites use the S3 service to store their data. I met Manish from Picsquare and he said that they stored all their high-res photos on S3. He also mentioned that some other companies kept their data backups on S3! I had no idea that Amazon services were so popular.

Aditya Mishra from TCS had a session on innovation in large IT services company, where he made an interesting point about what innovation means for a services company. After all, someone else gives the requirements, and you deliver for those requirements, so what does it mean to be innovative? He gave a number of examples where TCS has been innovative with business models and development processes. He made a point that the entire outsourcing industry is a result of TCS innovation in succeeding with outsourcing in the 1970s when no one believed that such a thing could be done.

There were a couple of discussions that I had about Agile. Marco Janson from Thoughtworks had an introductory session on Agile, but I also had some interesting coversations with a lot of people from thoughtworks, as well as Sowmya from Aditi. There were some more sessions on agile, mostly by various people from thoughtworks.

There were a few Python related sessions. I attended one by Anand Chitipothu on web.py. One of the interesting things was that Anand is doing a rewrite of Infogami. web.py was initially written by Aaron Swartz for Infogami.

Aashish Solanki had a session on his new site YBangalore.com, which is a site to have user generated content about Bangalore. There were a number of other sessions on other sites that have been developed or are in beta. I didn't attend many sessions on Day 2, as I was in the corridor discussions that were taking place. As a result I missed many sessions.

Another session from Day 1 was Jay Fichialos talking about how they implemented HackDay at Sabre. Sabre is the company that owns travelocity and a number of travel and hotel related sites. Jay got the idea for HackDay after talking to Chad Dickerson at the first Bangalore Barcamp. Chad had just finished the HackDay for Yahoo at that point. Jay then implemented HackDay at Sabre and it was a huge success.

Taylor Cowan, also from Sabre had a session on an experiment he is doing with Sabre Labs. Sabre has huge databases of information on hotels availability, car rentals, airports, flights and so on. The idea is to make all this data available via an open REST API. Once the API is released, developers will be able to take this data and create their own mashups. This is very exciting. The whole thing will launch at opencontenttravel.com sometime in mid-Jan.

Another interesting discussion was with Jace, Shreyas, Amit Pande and Harish about Bangalore culture and sociology.

Shreyas also had a presentation on RadioVerve, a site that presents streaming music of independent indian music. You should check it out.

I also met Manoj Govindan and Ravi Mohan in a discussion on cellphones, dungeons and dragons, and wargaming. I last met Ravi back at Agile India 2005.

Finally, I gave a presentation on the stuff that I'm currently working on. I'll elaborate more on this in the next post.

A big thanks to the organisers and sponsors who did an awesome job in setting up this barcamp!

Back from barcamp

Returned from Bangalore today. I was there over the weekend to attend the second Bangalore Barcamp (my first). It was really cool. Lots of interesting sessions, lots of interesting people. Expanded post coming up tomorrow.

A case study in poor management

If you dont follow cricket, skip to the lessons to learn section below

India got thrashed by South Africa in the cricket match a couple of days ago. Well, no surprise there. The Indians have always struggled in South Africa. What was a surprise was that the BCCI (the Indian cricket board) vice-president, Shashank Manohar came out and said that if it were up to him, none of the players would be paid a single penny. Quoting:

Manohar said the board was not to be blamed for the team's poor performance. "We are administrators and we have done our utmost to provide the players with the best of facilities and support staff. It's the job of the players to deliver on the field, not ours. We at best can take corrective action and measures."

This is pathetic for a number of resons. If Mr.Manohar feels that the players are not good enough, the BCCI should take responsibility, because they are the ones who selected the players. If Mr.Manohar feels that their technique on fast pitches is suspect, the BCCI has to again take responsibility, because they prepare slow pitches for domestic games. When will the BCCI learn that you cannot 'get acclimatised' to fast pitches with one warm-up game, and that it takes years of playing on these pitches to learn to handle them? Mr.Manohar is trying to wash his hands off the matter and put all the blame on the players, when in fact, the majority of the responsibility falls on the BCCI.

Lessons to learn

This is a perfect case study of what not to do. Here are some lessons to be learnt.

How many times have you heard a manager complain that the people under him are not good enough? Pretty often I'm sure. Managers, remember that you hired the people. If they are not good enough, you should take responsibility for it. Now, it is perfectly okay to make hiring mistakes. Everyone does it. But never say that "the person is not good enough", say instead "I made a hiring mistake", and then move forward from there.

Sometimes you don't hire the people under you. Maybe you moved to a new team and have to work with the existing team members. Can you then say that the people are not good enough? No. As a manager, one of your responsibilities is to make sure that the right people are doing the right work. When you take over a team, the first step is to make sure you've got the right people, and take steps to correct the situation if you dont.

Here is a situation: You have a great developer whom you promote to a Project Manager. Management requires a different set of skills to development, and the same person who was so good with development now struggles with management. Do you find this situation familiar? Different situations require different skills. These skills dont magically appear on people as the situations come up. The manager should have an idea of the skills that will be required, and have a personal development plan to help the team members acquire the skills that they are missing. Most managers do not do any personal development and then blame the team member when a situation arrives that the person is not equipped to handle.

Another golden rule is that you always back your team. Within yourselves, you can sort out any issues that need sorting out, but to the external world, you are together. You celebrate together, and you face the music together. When the going gets tough, you face the music on behalf of the team. You never, ever, abandon the team. Yet, when facing the heat, we often see managers who turn round and push the blame on the team. If you stick with the team, you will earn trust and respect, otherwise the team members will only remember you as the weasel who backstabbed them.

Management is unique because you are responsible for the results even though the actual work is done by someone else. A great manager helps others achieve results. The key word here is 'help'. Not 'forces', not 'commands', but 'helps'. How few managers there are who have that attitude. Jack Welch said it best — Before you become a leader success is all about growing yourself. Once you become a leader success is all about growing others.

This post is a part of the selected archive.

Selected archives available

Its almost three years since I started this blog. Thats a long time. I was going through my archives when I realised that its impossible for anyone to find the good posts amidst all the other posts. So I sat down and collected some of my favourite posts into a selected archive page. Check it out.

Flexibility vs Efficiency

This post by Dmitri kicked off a mini-debate about agility and rigidity. The debate is about whether a developer should be interrupted for a day to perfrom some side job. Check out the posts linked above for the whole story.

This is a common enough issue that it requires further examination.

At the root of the issue is what I call the flexibility-efficiency dilemma. This tradeoff is, in fact, at the root of many issues of agile vs traditional processes. The essence is that if you want more flexibility, you trade in less efficiency and vice versa.

Example

You have ten packages that you want to send by post. Each package involves buying the cartons, collecting the items, packing the items, writing out the address, filling up forms, and finally going to the post office and delivering them. There are two ways to approach this problem.

Approach 1

1. Buy ten cartons
2. Collect all the items
3. Pack the ten packages
4. Write out ten addresses
5. Fill up the ten forms
6. Go to the post office and drop off all ten packages

Approach 2

1. Buy one carton
2. Collect items for one carton
3. Pack the carton
4. Write out the address
5. Fill up the form
6. Drop off carton at post office
7. Buy another carton, and repeat the whole process ten times

Which approach is better? Most people would say the first one. Hmm. First, we need to define what we mean by better. Let's rephrase that question. Which one is more efficient? I think we can all agree on the first one. Its a no brainer. Just the ten trips to the post office makes the second approach way more inefficient. Let us say that it took ten days to send the ten packages via Approach 1 and fifteen days via Approach 2.

Now, which one is more flexible? Ah. This time its the second one. Why? Because it is much better suited to handle unexpected surprises.

I'm going to add in a surprise now. Surprise: After five days, you are informed that you need to go overseas for an office trip right now. Packages sent using Approach 1: Zero. Packages sent using Approach 2: Three.

Lets try another surprise. Surprise: It seems that customs is not letting such large packages go through. You need to repack into smaller cartons. Approach 1 impact: All your ten packages come back in two days. You repack everything for another ten days. It takes twenty days to get everything through. Approach 2 impact: After two days, your first package returns. You repack and resend it. The unpacked items are packed into smaller cartons the first time around. It takes seventeen days to get everything through.

Apart from the above approaches, are intermediate approaches. You could make five trips to the post office, sending two packages each time. The net result of efficiency and flexibility will be somewhere between the two extremes.

Software processes

Surprisingly (if you think about it, not such a surprise), the two approaches have a direct correlation with software processes. Just a couple of quick definitions. In the next paragraph, when I say 'agile', I mean an Agile (captial A) process. When I say 'agility', I mean ability to handle unexpected requests. So high agility mean high capacity to take on unexpected process, not adherence to an agile process.

At one end is traditional waterfall, exemplified by an up-front plan and minimum adaptation to changing conditions. It can have high efficiency—provided the requirements never change and the software is understood perfectly.

At the other end is the ad-hoc process (bet you thought it would be agile at this end!). With ad-hoc process, everytime someone asks, you can just drop what you are doing and handle it. There is no process constraining you. However, efficiency is extremely low. You are multitasking so often that you never get anything done.

Somewhere in between, but more towards high agility, are agile processes. This is something like the five trip,two package per trip, example above. The idea is that you fix an iteration length. During the iteration, programmers are left alone to work. At the start of every iteration is a 'change point.' This is a point where stakeholders can come in and change the course of an iteration. By choosing shorter iterations, you create more 'change points' for more flexibility. Or you can choose a longer iteration for more efficiency. I'll write more about choosing iteration length in a future post.

FINALLY: The answer

With that under our belt, we can finally decide what to do about the problem. The situation is that one of your developers needs to be taken off for a day to implement a small feature for a sale. Should you do it? Scrum says no disturbance in the middle of the sprint. Joel says that doesn't sound agile to him.

What to do?

The simple answer: Ask the managers to decide if the sale is important enough to delay the other project and take a developer off if it is.

The complex answer: Your average manager is not Joel Spolsky. Most managers are under the mistaken impression that there will be no impact on the original project (hey, its only one developer for a day right?). Not all requests are really all that important. Be clear about the cost to the original project, because this is often hidden to them. Then go ahead and accept the request, but be _very_ careful that the process does not degenerate into a ad-hoc-handle-every-change-every-day process.

This post is a part of the selected archive.

Leaving today

I'll be leaving Singapore today at 2035 SGT, arriving in Chennai at 2200 IST. Only an hour to go before I leave for the airport.