Sunday, November 30, 2008

7 Practices to Agile QA

If you are transitioning to Agile/Scrum and wondering how your existing QA practices need to evolve in the new world, here are 7 practices that you should consider implementing.

1. "In-cycle" QA. Traditionally QA team is a separate team and . QA testers should be integral part of the development team. They should be working very closely with the developers on stories in the same sprint/iteration. If QA testing is done in the next sprint for stories implemented in the previous (out-of-cycle), the team will always find itself doing a catch-up and end up accumulating "technical debt." As a rule of thumb, you would need 1 tester for every 3 developers. However, everybody on the team should be willing to jump in and help with QA testing. When planning, make sure to adequately account for QA activities. It is OK to take on fewer stories in the beginning to find the team rhythm before trying to increase the velocity.

2. Implement 5 Quality Gates. Traditionally QA is seen as just performing functional/system tests. But, software development is a highly integrated process. The quality of each activity- requirements, design, and coding, could impact the amount of effort needed in testing. If quality issues that could have been detected and prevented during these other activities are left for testing to catch them, there may not be enough time to fix the issues. Even worse, it may not be found until after deployment. QA needs to expand beyond just testing and should include the following QA gates.

Fig. 1: Quality Funnel
  • Quality Gate#1- peer review of requirements. Make sure stories meet INVEST (independent, negotiable, valuable, estimable, small, and testable) criteria as much as possible. This could be an on-going activity as part of pre-planning/product backlog grooming and not tied to a specific sprint.
  • Quality Gate#2- peer review of design. Make sure the design is in-line with the architecture guideline/style and appropriate alternatives are considered.
  • Quality Gate#3- peer review of codes. Make sure appropriate unit tests are written, coding standards are followed, and above all ensure codes follows design.
  • Quality Gate#4- continuous integration. Make sure code is being integrated and automated tests are run on a continuous basis and any issues that surfaces are addressed immediately.
  • Quality Gate#5- automated functional tests. Make sure QA testers are testing using automated scripts as much as possible.
3. Target 90% test coverage with automated testing. Even though test coverage does not tell anything about the quality of tests, it tells you where you need to focus more than others. Without test automation, it would be difficult to accommodate adequate testing in a sprint.

4. Inspect and adapt. Do not just stop at identifying the technical cause of a bug. Understand where (the quality gate) this bug could have been caught and improve the associated process.

5. Definition of "Done." Include all quality gates as part of your definition of "done."

6. Fix all know bugs first. Traditionally bugs are triaged based on severity as well as other works in the pipeline. The danger of doing this is that a seemingly minor/cosmetic bug could result into a large cumulative technical debt over time. Fixing bugs as soon as they are found can be less costly and time consuming than to wait for the right time to fix them.

7. Prioritize test cases. The intention behind this practice may not be apparent at first. The idea is that tests cost time and money. You can spend inordinate time in tests without getting any incremental/marginal value. Knowing how much testing is enough as well as what tests are more important than others not only will save time but also will ensure quality of testing. In order to understand what to test first, you could map your stories along two dimensions- frequency of use and risks of having bugs. The stories that gets used more often and also would have high risks should bugs are encountered needs to tested first and more.

I would be interested to know how you are doing QA on your Agile projects.

Sunday, November 02, 2008

Stockholm Scrum Gathering 2008

I got back from Scrum Gathering in Stockholm a couple weeks ago. This was my first time in Sweden. It was colder than I expected. I went there over the weekend before the conference so that I can see some places around. Unfortunately, my luggage did not arrive with me. That kind of restricted my movement in that cold weather. Also, I realized that my session was the first one on the first day of the conference. I needed to prepare for my presentation...:-)

Overview of the Gathering. The conference was sold out. It was good to see a growing popularity of Scrum among the European companies. People came from many countries. I met people from UK (Conchango guys), Netherlands (Xebia), Denmark, Finland, Norway, and Germany (Siemens). The gathering was graced with the presence of Ken Schwaber and Jeff Sutherland. As always, the participants made the the conference come live through great interactions at the keynote speeches, and breakout sessions, as well as by self-organizing at the Open Space. The highlights of the conference from my perspective were the two great case studies presented by Xebia (Distributed Scrum) and (Enterprise rollout of Scrum at, the discussion on Agile Contracting at the Open Space, and Q/A session by Ken and Jeff to end the conference. Here are three pictures from the gathering,

Serge from Xebia on Agile contracting at an Open Space
2. Jeff and Serge on the role of product owner
People are mingling during a break

High Level Theme.It seems contracting is getting a lot of attention these days as more companies are adopting Scrum. The traditional contracting is based on waterfall model and it does not work for Agile projects. We expect to see some templates on Agile contracts to emerge over next year or so. Another overarching issue that everybody is grappling with is "Scrum" verses "Scrum Butt." Both Jeff and Ken talked about how to move from "Scrum Butt" to "Scrum." Jeff talked about the Nokia Test for teams to determine where they stand on Scrum maturity. Scrum adoption is a continuum and teams move along that continuum over time. All team starts at ScrumButt. Teams gradually move from "Scrum Butt" to "Pretty Good Scrum" to "Good Scrum" to finally "Great Scrum." In Jeff's word, hyperproductivity is achieved by "Great Scrum" teams. I am not sure about hyper productivity is sustainable in the long term though. Agile/Scrum development process is intense as it is. The goal is to find a sustainable pace that helps the team to maintain a sane lifestyle.

My Session on Agile QA. I will be honest that I was a bit nervous about how many would show up at my session. To my surprise, there was a great turnout at the session. A lot of good conversations happened during the presentation. I'll write a separate blog on the topic. Everything went well except that I was caught off guard when someone commented that we were doing "waterfall" when I was sharing how we were doing QA at a large government project as a case study on Agile QA. The "waterfall" sounds like a curse word specially at an Agile conference...:-) However, my key take away from my session was that I need to put more thoughts on how Agile QA would scale. In fact, scaling Scrum is not specifically an issue with just QA activities, it is an overarching challenge for all large projects. It is just that QA makes it more difficult. I plan to write about scaling Scrum in the future.

What's Next. Xebia case study showed how distributed Scrum and offshoring can produce great results. Agile/Scrum is increasingly becoming a critical element for making offshoring work. I look forward to seeing more case studies on the use of Agile/Scrum to make offshoring work at the next Scrum Gathering in Spring 2009. As for myself, I hope to have a session on Agile architecture and scaling Scrum.

Overall, I enjoyed the conference. The opportunity to visit beautiful Stockholm in itself was worth the trip. I would love to go back again. It is always difficult to visit a place alone. However, I enjoyed my walk-around the old city. I wish I had more time to visit the islands. Next time...:-) Here are four more pictures of Stockholm,

1. Bustling evening in the alley of old city of Stockholm
2. The Royal Guards at the Swedish Palace
3. The sun setting in the alley

Finally, this picture is for people in Bangladesh, do you recognize the logo in the picture?