Django code coverage for doctests

As a followup to my post on code coverage for your Django code, Eddy Mulyono shows how you can get code coverage working with Django when you use doctest for unit testing. Check it out here.

Advertising to the new consumer

Russell Buckley has a nice post pointing to a new Microsoft video about a "break up" between the old advertisers and the new consumers.

Old advertising was all about bombarding the consumer with messages, but what the new consumer wants is to have a dialog. Now, this is not news if you have read the Cluetrain Manifesto or any of the stuff that followed it, but it probably is news for a lot of traditional advertisers.

The new advertiser will be talking to their customers via blogs, message boards and social networks, especially if their target market hangs out online.

Silver Catalyst 1.0 has been released!

Some links to get started

• Download Silver Catalyst here
• Check out the new, revised documentation (PDF file)
• Head over to the list of online resources
• Of course, you can also buy a license for additional team members

Another thing: This release of Silver Catalyst is beta license compatible. So if you helped out during the beta test and got a license at the end of it, you can use the same license with this release and it will work. Thats US$1,568 in value. Just to say thanks for beta testing the tool.

Proto.in 2007 SE

So the Summer Edition of Proto.in is coming up in July. Like the first edition of Proto.in, we will be selecting a bunch of startups to come down and present their prototypes to an audience of VCs and other folks.

First, nominations have opened, so if you are a startup looking for funding, customers or partners, then head over to the nomination page and nominate yourself. You can nominate yourself again, even if you nominated the last time around, as long as you weren't selected last time.

Secondly, we are looking seriously at the feedback we got from the first edition of Proto. As the first event of its kind, there were a number of things that happened well, but I'll be the first to admit that there were a number of things that could have been better.

This is a good time for me to say that the Proto blog for the second edition is up, so take a look.

One of the things we have been rethinking is the entry fee for the event. This time around we are going to charge Rs.10,000 for startups instead of the Rs.5,000 last time. In exchange, we hope to bring in more value for the startups this time. How? I'll post more about that in the days leading up to the event.

And thats the point really. The focus is in making sure that startups get good value for what they paid for. There has been a lot of brouhaha about the meaning of non profit, and whether we are making loads of cash and retiring to the Bahamas etc. Just to clarify, while I wish I was retired in the Bahamas, I am unfortunately still in Chennai. I'll just point to Vijay's reply (see comment #16). I don't really want to spend any more time on this topic because its kind of lame.

Coming back to the startups. Our main focus is in ensuring that the startups are happy. Which means satisfying what they want. Startups basically come to Proto for three things - funding, customers and partners. So we need to bring to them some investors and a high quality audience. If there are 200 people in the audience, we would like it to be 200 people who would like to do business with the startups, rather than 200 spectators.

We fell short on this on two counts last time around. First, the audience was mostly spectators. Second, the audience preferred to stay in the main hall through the event. We had hoped that a majority of the audience would go to the networking hall and meet with the companies, but they mostly decided to stay in the main hall instead.

This was pretty much a surprise to us all, because all the pre-event debate was about what to do if the whole audience hogs the networking hall after a good presentation and no one remains to see the next presenter.

We are thinking about how we can tweak the format this time to encourage the audience and the startups to mix.

One of the options is to have half an hour breaks after every few presentations. This would allow everyone to mix around without having the guilty feeling that you are missing something in the main hall. Obviously if we go this route, then either we have to go to two days of presentations, or cut down on the number of companies presenting. There is no way we can finish on time otherwise.

Another idea that is doing the rounds is having stalls for each company. That would act as a point of contact between the audience and the startups, and generally allow for more interaction. The issue then is of finding enough space to setup twenty or twenty five stalls.

If anyone has more ideas, post them here.

Coming back to the entry fees for startups. Some have said the entry fees are really high (and are getting higher this time..). We've debated this within ourselves a lot.

DEMO, on which we have modeled Proto, charges a startup US$18,500 as an entry fee. That's seven and a half lakhs when converted to Indian Rupees. Read that number again - seven and a half lakhs. For the first Proto, we charged Rs.5,000 and this time it will be Rs.10,000.

The same goes for the audience fee. It costs US$2995 to attend DEMO. It costs US$2495 to attend Techcrunch20. We charged Rs.500 (US$12.50).

Sure the numbers are not directly comparable, and the value provided for your money is different, but it still puts in perspective that Proto is very very affordable compared to equivalent events.

There is of course a lot more to write about, but this post is big enough, so other stuff will have to wait till later. Any comments welcome.

Mobile Monday Chennai

Some updates on Mobile Monday. It's coming up this Sunday, 6th May at the TeNeT hall in IIT Madras.

Time: 10 AM

Venue:

TeNeT - ADI Seminar Hall,
CSD Block, 3rd Floor,
IIT Madras.

Check out the wiki for the speaker and attendee lists: http://www.momochennai.com/

Do you buy comics?

If you do, then Fantagraphics Books is giving a 20% discount on all their books.

Unlike other discounts, this one is different and rather unique. To avail of the discount, you have to add their user to your network in one of the popular social networking sites: MySpace, Flickr, Orkut, Bebo, Tribe.net, Yahoo 360 or AIM Pages. Once you add them to your network, just mention your nickname and the website and you get 20% off all books purchased.

I took the opportunity to order Volume 1 and 2 of Usagi Yojimbo, Like a Velvet Glove Case in Iron, a couple of new issues of Mome and Bête Noire #1.

Upgrade to new blogger

I upgraded to the new blogger interface yesterday, which means that I finally have access to labels. Over the week I'll be labeling most of the 300+ posts in the archives. If you are reading this via a feed, it may happen that a whole bunch of old posts lands up in your feedreader. If that happens you can just ignore it as its only the labeling going on.

Chennai Air Show Photographs

I've uploaded a bunch of photographs from the Chennai Air Show that happened at Marina Beach yesterday. The photos are here:

http://www.flickr.com/photos/siddhi/tags/airshow/show/

Code coverage for your Django code

One of the things that I like about Django is that it allows you to run unit tests on the code with relative ease. So I have a bunch of unit tests now, and I was looking to expand it. For that I first needed to know which parts of the code had good coverage and which parts had no coverage. I needed to integrate code coverage with the unit testing framework. In the end, it involved modifying a few files. This is what I did:

1. First, I downloaded Ned Batchelder's coverage.py module.
2. Next, get this script to colorize the coverage output. I saved it as coverage_color.py
3. Put both files somewhere on the python path. I put it at Python24\Lib\site-packages
4. Now we need to modify the django unit test runner to include coverage. Head over to your django\test directory and edit the simple.py file
5. At the top, add the following line:

   import os, coverage, coverage_color

6. Scroll down to the run_tests function. You will see a line like this:

       unittest.TextTestRunner(verbosity=verbosity).run(suite)
       destroy_test_db(old_name, verbosity)

7. Modify it to read like this:

       coverage.start()
       unittest.TextTestRunner(verbosity=verbosity).run(suite)
       coverage.stop()
       if not os.path.exists(settings.COVERAGE_DIR):
           os.makedirs(settings.COVERAGE_DIR)
       for module_string in settings.COVERAGE_MODULES:
           module = __import__(module_string, globals(), locals(), [""])
           f,s,m,mf = coverage.analysis(module)
           fp = file(os.path.join(settings.COVERAGE_DIR, module_string + ".html"), "wb")
           coverage_color.colorize_file(f, outstream=fp, not_covered=mf)
           fp.close()
       coverage.erase()
       destroy_test_db(old_name, verbosity)

8. What that does is to record the coverage when running the tests. It then creates a directory for putting the HTML output, creates the colorized version of the source and dumps it into the output directory. At the end of everything, it cleans up the coverage data
9. We now need to configure the COVERAGE_DIR and COVERAGE_MODULES settings. Open your Django settings.py file and add the following lines:

   COVERAGE_DIR = "scripts/build/coverage" # Where the HTML output should go
   COVERAGE_MODULES = ["apps.catalyst.views", "apps.catalyst.models"] # The modules that you want colorized

10. Save and run your Django unit tests. After running, you will have a html file for each module in the specified directory with the colorized coverage output

Check out the output: The lines in red were not executed by any of the unit tests. Looks like there is a function that I'm not testing at all!

Mobile Monday Chennai

Update: See here for the latest update.

Varun Krishnan is organising the first Mobile Monday event in Chennai. Mobile Monday is something like Barcamp in the sense that its a tech gathering, except that the event is only 3 hours long and the topics revolve around mobility.

Mobile Monday, as its name states, is supposed to be on a Monday, usually after work. The upcoming Chennai event, however, will be on a Sunday (some people call this Mobile Monday on a Sunday or MMS). Mobile Monday is also supposed to be once a month, usually the first Monday of the month. The Chennai edition will be once a quarter instead. If there is demand, the frequency will be increased.

So here are the details

Date: April 22nd, Sunday (2 weeks from today) Update: The date has been changed to sometime in May
Time: 10am to 1pm
Venue: Mobile Worx office

The schedule is three tech talks, 20 minutes each and three product demos, 20 minutes each followed by lunch.

I've signed up for a talk on Location Based Services. I'll be talking about a couple of projects that I did while I was in Singapore.

If you are interested in giving a talk, head over to the Speakers wiki page and add your name there.

Barcamp Bangalore 3 Roundup

I was in Bangalore on Saturday and Sunday attending the third Barcamp to be held there. Here are bits from the event that I found interesting. This is a long post, you have been warned :) Note that the following is not in chronological order.

Python Conference

I met a few guys from the BangPypers groups - Anush, Anand, Kushal, Swaroop and Sidu. Apart from the Python conversations, I also asked if anyone knew what was happening on the planned Python conference. No one had much information, so I'm not really sure whats happening. Is the Bangalore python conference going to happen?

The Knowledge Foundation

The Knowledge Foundation is the name of the group that is behind various unconferences (plus Proto.in) in Chennai. We had a discussion on how to spread the unconference movement to other cities and other non-tech settings. While Barcamps have been a big success in India, there has not been much activity outside of a few tech events like Barcamps and Mobile Mondays. TKF has been working on that in Chennai where apart from Barcamp we've had Blogcamp, Wikicamp, Proto.in and Rooftop Film Festival (that reminds me - I have to upload the RTFF photos tomorrow). Coming up are another edition of Proto, Mobile Monday and Shoot. Quite a few people were enthusiastic about having small gatherings on other such topics. It would be great to see more unconferences on a whole range of topics in various cities.

Silver Catalyst

I talked to a few thoughtworkers about Silver Catalyst - Amir, Sidu and Vivek. Amir had recorded my session and had also interviewed me at the second barcamp bangalore back in December. I also met Umesh and a couple of other guys who had also attended my session in the previous barcamp. It was interesting to see how the tool had progressed in the three months between the barcamps.

Sidu took some time out to point me towards Mingle, which is a tool developed by the new Studios division of Thoughtworks, scheduled to release in June. I wanted to discuss Silver Catalyst a bit more with Vivek, but I lost him in the crowd somewhere.

Organisational Culture

There were three sessions that I thought touched on the same topic of culture from different angles. The first was by Savita Kini on Products vs Services in Indian companies. The second was by Satish about whether soft skills training was required. The third was by Shourya on the Bambi code jam initiative at GE.

Savita pointed out a number of challenges and opportunities facing Indian companies. Shourya made a good point in the discussion asking why we think that products are superior to services. Some of the points raised were profit margin and scalability. I'm with Shourya here that I'm not convinced that products are always better than service. Profit margin depends on the product. The XBox for instance is sold at a loss. Mobile phones have very thin profit margins. Scalability is also not a great reason as it all depends on the market. If you put out a product in a small market, there may not be much scope for scalability.

In the end, I think the biggest reason that product companies are not as prevalent is because of the organisational culture. Large companies are by nature risk averse. Since there are a lot of profitable service companies, we tend to follow them and not take the risk of doing something new. One of the cool thinks about Barcamps is that it gets a lot of product startups together.

That ties in with the second talk about soft skills training in IT companies. Soft skills are certainly required. The problem arises when the training is imposed rather than sought. Sukumar made a good point that knowledge can never be given, it can only be taken. I dont think a one day training can build teams for instance.

Satish had a good point which, to me, got to the heart of the matter. Fishermen work in teams because they know that if they don't, all of them will have trouble with dinner. The result of success and failure is right in front of them. They don't need to take a training class to realise that. The problem in large organisations is that there is a disconnect between the work and the outcome. Developers who do the work cannot see the outcome of their work, either in the form of customer satisfaction or bottom line impact. Thus they associate with the closest outcome that they can see - the salary - and start working for the money instead of the company.

A startup culture is like the fisherman team. Everyone is aware of pretty much everything and they know the outcome of their work. The question is can we get a startup like culture in a medium or big company?

Which brings us to the third talk by Shourya on Bambi, a codejam initiative taken in GE. He explained how he got the idea from the Yahoo Hackday in the first Barcamp Bangalore, and implemented it in the form of a codejam at GE. He explained how there was trouble convincing the management about holding it. Finally when it was held, it produced one IP plus a number of other cool, but not so useful projects. The second time around, the management actually asked for it to be held and it was held with some changes in the format.

A codejam, 20% time to work on your own project and other such initiatives are 'cool' initiatives. The problem is that they do not have direct ROI consequences. However, the actual dynamics are a lot more complex. Such initiatives for instance may (or may not) help attract smarter programmers, or reduce attrition, or build up a specific company culture. It may fail if the current programmers are not the type to take part in such events. So the success depends on both the current culture as well as the future culture.

Another problem is that some companies don't really believe in it. Someone was talking about 20% time in their company and how you can work on what you want provided you finish the main work. Since work pretty much takes the whole time, plus a bit more, this is as good as maintaining the status quo. The point of 20% in Google is that you can spend that time even if you are working on something else. They are okay with the fact that the main work will take a bit more time. So that again points to a difference in company culture.

So how can these forces be resolved in managing the current culture or creating a new culture? Thats the big question, but the discussions were interesting.

Proto

There was a discussion on the venue for Proto. A lot of people from Bangalore wanted the next edition of Proto to be held in Bangalore. This was a completely ad-hoc session held under the trees (an ideal unconference session!). [Off-topic digression: When a projector is involved there is a clear speaker/audience divide. Get everyone sitting in a circle and it becomes a discussion with everyone pitching in their views.] Back on-topic, there was a good discussion where it was decided to have the next edition in Chennai with interested people from Bangalore joining the team, and the session after that will be in Bangalore.

Barcamp Bangalore 2 DVDs

Amir from Thoughtworks had recorded a lot of sessions from Barcamp Bangalore 2. It came to 8 DVDs in all after editing. Kesav brought his set of DVDs for us to make a copy. The problem was that it takes a long time to copy all eight. Finally, Kushal volunteered to make copies for us. Thanks Kushal!

Quake deathmatch

A few of us had a Quake deathmatch using OpenQuake on Saturday evening. Jace, Shreyas, myself and a couple of other guys were there. It's been many years since I played Quake so it was great fun. For some inexplicable reason OpenQuake would sometimes stop reading the horizontal axis of the mouse. The mouse would work fine in the menus, but in the game I could only look up and down and couldn't turn left and right. Quiting and rejoining the server fixed the problem, but the frag count went back to zero. It happened thrice in all. Highly frustrating. Absolutely no idea what caused it.

Dinners

A dinner was organised at the end of Saturday. I didn't attend it because I went to meet a few guys at the Forum. At the dinner were ex-thoughtworker and AI hacker :) Ravi Mohan, Bharat Guruprakash, Manoj Govindan, Rajesh Babu and myself. A number of interesting topics were discussed like machine learning, silver catalyst, the ongoing drama at the scrumdevelopment group that has been tragically comical, work at thoughtworks and subex, thoughts on the CSM, agile in India, applying agile to embedded systems, PhD, research, entrepreneurs, dungeons and dragons, board games and so on.

We had to catch our train on Sunday night. Ramesh Babu joined Sukumar and me for the evening. He had hosted Syed, Kiruba and Sukumar on Saturday night. Kiruba and Syed had a dinner appointment so we split up at the station. They returned 5 minutes before the train was scheduled to leave, running and sweating, and then the train left half an hour late :)

Photography

I met Mahesh Shantaram at lunch on Sunday. Mahesh is a documentary photographer based in Bangalore. He has some very nice photographs that you can see online here. Amogh had a session on conservation with stories based on his photographs. He goes into the forest every weekend to take photographs. We later caught up with him at Cafe Coffee Day after the event was over to have a discussion about the Shoot event thats coming up and also if we could join up with him on a trip sometime.

Sessions

A few quick words about the sessions that I attended.

Motvik showed a demo of their WWIGO product. They showed how to use a mobile phone as a webcam and transmit data via bluetooth to the laptop where it was streamed over skype to another person. Inactiv explained their Activ Mobs service, a group SMS service for groups of friends. There was a presentation about Bigswerve by Raj Bala. What they do is to go through the comments at various websites and generate a network based on comment patterns. Comment patterns an example of implicit networks where the network is not formally defined and a comment automatically implies a level of engagement with the site. A few thoughtworkers gave a presentation of Continuous Integration and Cruisecontrol.rb. Sowmya Karmali had a session explaining the Mifos project. Marek Tuszynski had an event on How to organise an open event. Marek was the organiser for AsiaSource and a few other events. I think it started early or I messed up the timing because I really wanted to attend but when I went there the session was just wrapping up. There was a nice session on usability by Hari and Ram.

Bangalore Barcamp 3

I'm off to Bangalore this evening to attend the third barcamp at bangalore. I'm hoping for some good discussion and meet up with interesting people. I'll post a roundup of the event when I get back on Monday.

Little Green Guys With Guns

I found out about Little Green Guys With Guns, a play by email game through Ian Hickson. I played through the tutorial and it looked pretty fun, so I've joined the newbie game thats running. If you're into this sort of thing, take a look at LGGWG and join in. It should be fun!

Python decorators and generators: An introduction

I gave an introduction to python decorators and generators at yesterdays ChennaiPy meet. I've uploaded the presentations to the files section of the yahoo group (OpenOffice 2 format).

More on python decorators

A few months ago, I wrote an article on Using python decorators to implement guards. When Parthan from ChennaiPy was preparing for talk on decorators due to be given at the FOSS event at NIT Calicut, he mailed me for a few more examples of decorators. He's put up the examples along with an explanation of each example over here. If you are looking to learn more about python decorators, check out the blog post.