Wednesday, March 25, 2009

Orlando Scrum Gathering 2009

I just came back from the gathering last week. It was different this time from the past gatherings in that this time the event was graced by the presence of industry heavy hitters like Ron Jeffries (Co-developer of XP), Jim Copline (The author of "Advanced C++" that helped me shape my understanding of C++ and OO early in my career), Alistair Cockburn (famous for his book "Writing Effective Use Cases" and Agile process called "Crystal"), Dr. Mark Paulk (led the work on CMM at SEI), and last but not least Gregory Balestrero (President and CEO of PMI). As you can see from the lineup, Scrum Alliance has done a great job of inviting these people and initiating an industry wide collaboration around software development process.

What to expect in the coming months? Scrum Alliance has engaged Dr. Paulk to do empirical research on Scrum. He is going to focus on Agile and Scrum practices and help the Agile community to understand what practices really contribute to the success of projects. With the inclusion of PMI, we would see more and more practices being included in the PMBOK. There is a growing trend among PMPs to adopt Scrum and Agile practices. A group of PMI members has already formed a pmiagile Yahoo group to promote agile practices within PMI. This group has been officially recognized by the PMI.

What's Scrum community grappling with? From the types of discussions I heard at the event, it seems the adoption of Scrum has come a long way. The top things now-a-days on the minds of most Agile teams are,

  1. User Story vs Use Case. It was thought that User Stories are going to replace Use Cases. However, it seems people are still using Use Cases (especially on large projects) in at least some light weight format along with User Stories as they lack high level contexts. Jeff Payton's "User Story Map" technique to address this gap in User Story is becoming popular. He was at the gathering and offered sessions on it. Alistair is a vocal advocate of continuing to use Use Cases. Surprise, surprise...:-) He wrote an article on User Stories vs Use Cases for the Lean magazine (I thought I could get the article from Alistair's site or from the magazine's, but I could not).

  2. Agile Architecture. Jeff Sutherland and Jim Coplien had a session on how architecture can help a team becoming more Agile. There is a trend among Agile teams to only rely on continuous refactoring to have a suitable/consistent architecture emerge over time. They lose sight of the importance of starting on the right foot with some upfront "architecture thinking." Jim is working on an new architectural pattern called DCI (data, context and interaction) that could set the OO developers free from the shortcomings ("algorithm" part of the stories is usually scattered among many domain entities/models and get in the way to becoming Agile) associated with MVC.

  3. ScrumBut. Most teams can be considered ScrumBut, a termed coined by Ken Schwaber (I think) to point out how teams are not taking full advantage of Scrum practices. While the intent of ScumBut is to help team identify where they have improvement opportunities, it is creating confusion and frustration in the minds of the practitioners in that if there is such thing called "Pure Scrum" (which seems to be an illusive target). The whole confusion around the term ScrumBut (which gathered more negative, almost derogatory, connotation than it was intended for) lies in its definition based on a check list of mechanics. In fact, it was brought up in the panel discussion at the gathering that the definition of Scrum as found on the Scrum Alliance site as 3 roles, 3 rituals, and 3 tools hide the true essences of Agile captured in the Agile Manifesto. The mechanics are there to help teams get started, but they, by no means, are "sliver bullets." Each team needs to find its unique practices (customized through "inspect and adapt") that help them incrementally deliver working software that maximizes value for the customers.
Alistair nicely summed up the underpinnings of Agile process movement this way,
Self-awareness (focus of Crystal), self-discipline (focus of XP), and self-organization (focus of Scrum).

Thursday, March 12, 2009

Best Practices for Amazon Web Services (Amazon Cloud)

We run our ScrumPad (A Web-based project management tool) on Amazon cloud (AWS). We have been using Amazon cloud for last year and a half. We did not have any issue with it until recently, when we found out that we had lost shell connection to our production server. Although the application was working fine, we went into a panic mode realizing that we won't be able to do anything, should we encounter any problem. Once we calmed down from our initial panic attack, we quickly searched on the net to see if anyone experienced the same issue and how they recovered from it. We did not get any hit on it. There is always first time for everything. We thought to ourselves that the solution is easy with AWS. We would just bring up another EC2 instance and point to the new instance. Right? Not so fast. We found out that we were not using elastic IP. That means we need to make DNS change, which takes up to 24 hours to propagate and may result into some data loss and/or service downtime.

This incident helped us to think through how we should be using AWS and here is what we have come up with:

1. Always use elastic IP (behaves like a static ip) to point to your EC2 instance. You can then quickly swap instances without needing to make any DNS changes.

2. Always use EBS (elastic block storage) instead of the storage that comes with an EC2 instance to store your Database data . You can then quickly reattach your data store to a new instance as well as you will not loose your data in the event of EC2 instance failure and have your data automatically replicated.

3. Spread your EC2 instances to different availability zones and regions instead of putting them all in the same availability zone (a physical data center). You can then protect your application from single data center failure.

4. Backup (incremental snap shots) your database from EBS to S3 (Simple Storage Service), which is automatically replicated across multiple availability zones.

5. Create a custom AMI (Amazon Machine Image) for your application from a pre-configured public AMI. You can then bring up a new instance quickly (in less than 15 mins) on-demand and even completely automate server provisioning.

6. Rollout your new release to a new instance(s) when you need to change your AMI (i.e., when you upgrading your DB, OS, Application Server, etc.) and then reattach your elastic ip and EBS to this new instance. You can then quickly rollback to the old instance should you encounter any problem with the new release.
Figure: How to transparently switch to a new instance

7. If budget permits, use cloud management tool like RightScale.

With a few practices in place, you can have your own virtual data center (on Amazon cloud) that was only available to large corporations.

Thursday, March 05, 2009

Are You Getting The Most Out of Your Daily Scrum?

I was recently out on a six-week trip to our offshore development center in Bangladesh. When I came back, I found out that the Scrum team I am coaching has changed the format of the Daily Scrum. They were doing the Daily Scrum in the traditional stand-up format in front of a physical task board. However, we always felt that the Daily Scrum was not as effective as we want it to be. We tried different things in the past to make the most out of our Daily Scrum. Nothing seemed to work, and hence always fell back to the basic format.

The team is now using a projector to use the electronic task board available in ScrumPad during the Daily Scrum. I, to be honest, was surprised and a little skeptical at first. The team felt that this new format is helping everyone better follow what others are saying in the Daily Scrum. It seems the verbal communication is being reinforced by the visual elements. It remains to be seen that how long we can continue to follow this new format.

One of the challenges of traditional Daily Scrum is to really "listen" to what others' are saying. An effective daily Scrum should allow everyone to understand how individual work is contributing towards achieving the team's Sprint goals. Anytime the team falls out of sync, it should become obvious in the Daily Scrum. And the team should take appropriate steps to recover/compensate. Thus the Daily Scrum works as "a daily feedback loop" to keep the team on the same page. It is easier said then done though.

How effective is your Daily Scrum?