Friday, January 04, 2008

Economies of experience and Agile development

We always hear about economies of scale and scope. Do we ever think about how do these two concepts apply to software development? Let's consider economies of scale first. If we get more developers, we maybe able to negotiate lower rate from our vendor. Another way to look at it is the larger the project size, the lower the rate per hour of development. What about economies of scope? When we develop software, we could design it in a way so that we can end up with a set of reusable components. We could then use these reusable components and build our next software for an adjacent/related domain quicker. Building the first application will cost more. The successive applications should cost less. Thereby we achieve economies of scope. Here is one more reason to ask your boss to invest in OO on your project.

OK, you probably already know this. The reason I brought this topic up is to introduce another related concept that I call economies of experience. I don't hear anyone talk about this in the software industry. In manufacturing, there is a similar thing called experience curve. How exactly does economies of experience apply to software development? When we hire developers for a project, we all look for individual experiences in a developer with the technologies and in the domain the project is. The longer a developer works with a technology and in a domain, the more productive he/she is with that technology and in that domain. We, however, never think about experience at a team level. Developers maybe individually very productive, but when they work as a team it is completely different ball game. Like an individual, a team also needs nurturing to be able to unlock its fullest potential. In the abscence of active coaching and guadanice, a team can end-up crashing and burning.

Every team has a unique DNA. The DNA emerges over time. The longer a team works together, the more productivet the team gets. That is why Agile/Scrum uses a drag factor (negative productivity) in estimation. We increase the total estimate by this drag factor. A team working together for the first time may start at 40% drag or 60% productivity. Usually, it takes a year for a team to reduce drag to 0 or reach 100% productivity. Anytime a team member leaves or a new team member joins, productivity of the team goes back down.

In traditional project management, resource planning is done using staggered or just-in-time approach. We bring a developer on board only when we have a need for his/her skills. The thought there is that we would save money by engaging people only when they are needed. However, this (the savings) is seldom achieved. The further along the project, the higher the impact on team productivity for adding a new person to the project. That is why Agile projects make a reasonable effort to recruit the full team upfront with people who will be able to wear multiple hats. In the long run, the savings come from higher overall team productivity and utilization.

Conundrum of fixed price contracts for software development service?

I get this from our prospective clients a lot-

I need to know how much it is going to cost me to build my software.

The conundrum. I understand where they are coming from. After all budget is always finite. We do not feel comfortable committing to buying something without knowing the total costs upfront. While it is possible for fixing the price of many goods or services, it is very difficult, if not impossible, to do so with software development service. The simple reason is that the estimates are as accurate as the requirements. It is not possible to define everything one wants in his software with all the details. That is why buyers try to put many clauses in the contract to move the risk over to the service providers. On the other hand, the service providers try to push the risk back to the buyers by requiring the spec to be detailed and complete. We all know words are imprecise. Hence both parties end up with a contract that helps none. So what is the remedy?

Agile philosophy is in rescue. The concept is simple. The cost driver of software service is time. Since the budget is fixed, the total development time is automatically fixed for a specific rate. The only thing that we can play with is the scope of the work. Yes, I know who wouldn't like to have everything, but the reality is we need to restrain our wants. How do we do it?

Enter the world of no long-term commitment. The client creates and manages a prioritized list of things/features that he wants in the software. In Scrum, we call it product backlog. The developer works his way from the top of this product backlog, and delivers working software every 1-4 weeks (iterative incremental software development). That way the client is sure that he is getting the most value for their bucks. Yes, we can still give our clients a high level costs estimate in terms of number of sprints it might take to complete the project with an understanding that it would change as the project progresses. As the developer makes no commitment to any effort estimate, so does the client make no commitment to any budget amount. Both work collaboratively to manage the scope (the product backlog) to meet the goal and the budget. Every two weeks (assuming the development cycle is 2 weeks), the client can change the course of the project based on the current state of the working software in progress as well as the business conditions.

What should you do? I know this could be unsettling for many who are used to the old ways of managing vendors and projects. Believe me, if you try it out on your next project, you would not want to do it any other way. It is truly liberating when the client and the service provider work collaboratively to achieve a common goal without worrying about legal documents.