Why is "Continuous Deployment" challenging for most companies?

Continuous Deployment (CD) is one of the elements of Lean Startup model and the most controversial one. In simple words, it is an act of rolling software codes to live servers (a.k.a production) as frequently as many times a day. It is an extension of the popular and critical Agile software development practice known as Continuous Integration (CI).  In CI, a development team continuously integrates each other's work and run a suite of automated tests with every change in the code repository (a.k.a. check-in). This allows the team to catch any bad changes as soon as they are introduced so that they can be fixed quickly. Eric Ries took this practice a step further at his company IMVU. At IMVU, they move codes to live servers as many as 50 times a day, a high mark set for others to follow. To be able to pull this off, you would need technical maturity and product management discipline. However, as much as I love to be able to implement it at my company, I see a few issues with extending CI to become CD at most companies.

Review by the product owner. Unless you are a one-man team, you would want your changes reviewed by the product owner before you make it available to the users. In Lean Startup world, most likely the product owner role will be with someone from the customer development team. At least, the product owner is not someone from the product development team. Even the team is co-located, and there is daily interaction between the development team and the product owner,  the product owner should review the changes before  we ship them out the door. The product owner would want to make sure the changes are what he/she asked for. Usually the product owner would have feedbacks for the team. So, it is not clear how bypassing this human review is going to work. The larger the team, the more important the review becomes. If the team is geographically distributed, it is even more important. If you outsourced the development, I don't think you would feel comfortable with changes making to the live servers without you reviewing it.It is not clear how the product owner role fit into this CD at IMVU.

Risk vs reward. Yes, we cannot eliminate all bad changes or bugs through tests. However, I am not sure what incremental value we would get from CD beyond CI. With CD, yes, you may catch some bugs that may still slip through CI sooner (assuming you have a lot of users, which may not be the case for an early stage startup) and it may be easier to track down (there are other ways to reduce debug cycle) the bad codes since only need to work with smaller set of changes. But, at the same time, CD increases business risks (not just ours, but also for the end users), even if we can back out our changes with the flip of a switch. For most business applications, the reward would not outweigh the risk. In fact, it is a good practice to perform deployment during a "scheduled maintenance window" (even if you can do hot deployment) that the users know about so that their is no surprises. However, I can see the reward outweigh the risk for IMVU and similar services.

Non-backward compatible database changes. Database changes are always the most difficult to deal with, and requires more scrutiny and would not be good candidate for CD. In other words, it is difficult to back out (undo) changes if things go wrong. With iterative, incremental development, we encounter these changes more frequently. It is not very clear how IMVU handles these type of changes. It seems from the comments on their blog on CD, they handle the DB changes out of band. That means these are not part of CD.

Although I do not see any significant value in CD to live server beyond CI, I definitely see the value in CD to test/integration servers. In fact, we recently started doing this. It is not exactly CD, more like daily deployment to test servers for the product owner to review and provide feedback. This helps us to be able to deploy changes to live server every iteration (every 2 weeks).

Are you doing CD? Am I missing something?

Retrospective on ScrumPad Product/Market Fit

As I am beginning my journey of Lean Startup with our ScrumPad service, I plan to share my experience along the way.

The first stop in the journey is "Product/market fit." For the uninitiated (I am not an expert either, learning by doing), all it means is to prove that there is a large enough market for the product or service that I am building. The goal at this stage of a startup lifecycle is to identify/define that market with specific details- who are the users, why they like our product, and the business model that would work the best, and a repeatable sales model.

The best way to adopt the Lean Startup model is to first look back to understand where we are (since it's been 18 months that we are working on ScrumPad). During this time, we did not apply the Lean Startup model (were not aware of it). We only focused on product development using Agile software development model (the only good thing we did). Customer development was largely absent. I haven broken the retrospective in two parts- 2008 retrospective, and 2009 retrospective. And I have structured it in light of Lean Startup for easier understanding.

Version 1- Free ScrumPad: 2008 Retrospective
What was our hypothesis?
All started with a itch of ours. We looked for a tool that fits our needs. We found two- Basecamp and Version One. But, none was what we were looking for. After trying both, we had decided to build one that we liked. So, since we liked what we had built, we thought there must be more like us who would also like ScrumPad.

How long did we spend to validate this hypothesis?
6 months.

What did we do during this period?

• We primarily focused on adding functionality (product development, not customer.). We thought more functionality = more satisfaction = more users. 
• We were releasing incremental version of ScrumPad every two weeks using Agile/Scrum process.
• We did not do anything on the customer development front. Did not do any marketing. Did not talk to customers for feedback (did not get out of the building). 
• We completely deferred design thinking that design and usability is secondary to functionality. We thought once we have the functionalities in, we could always add design and improve usability.
• We did not charge for our service. We thought we need to first build out the full product (not thought about an MVP) before we can start charging.
• We did put some basic tracking- Google analytics as well as some custom ones (i.e., number of projects, users, and overall project activity index that we defined). It helped us detect some early issues with signup and getting started with ScrumPad. 

What was the result?
We did get a constant stream of SEO traffic from search engines. This traffic resulted in a trickle conversion 1 to 2 every week. Received some unsolicited feedback, primarily from those who liked it. However, when we started charging in early 2009, most of the users from 2008 stopped using the service.

  New   Bounce Avg. Time   Pages 
Visits     rate     on Site       per visit  Conversions
3,521     48%   00:02:50            2.61          5%

What did we learn?
Our product is better than the opensource counterparts. However, as a paid service, we did not learn anything about whether we have a market yet as a paid service.

Version 2- Commercial ScrumPad: 2009 Retrospective 

What was our hypothesis?

If we shift focus from functionality to user-interface and usability, there is a market for our service for a $21/user/month with single pricing model (no tiered pricing). I did a pricing analysis of our competitors (I must admit my class on pricing for my MBA from UNC helped).

How long did we spend to validate this hypothesis? 

All of 2009.

What did we do during this period?

• We changed our logo, updated the UI, and continued to tweak the interface to improve usability. 
• However, we spent a significant amount of time in refactoring (rework) to fix the sprawling code-base that resulted from the quick piling up of functionality. 
• We started actively asking for feedback. We in fact incorporated a short exit interview during the cancellation process. 
• We started charging for our service. 
• We still did not do any CPC marketing. We wanted to focus on word of mouth. We added affiliate program instead. 

What was the result?

We continued to have a constant stream of SEO traffic from search engines. Even with pricing in place, we saw significant improvements in all metrics except trial conversion rate. We saw a drop in trial conversion rate (by 1%. I think it makes sense since its no longer free). With "trial to subscription" business model, we finished the year with a few paying customers, but no new customers through our affiliate program.

                                                                                                           

 New     Bounce    Time      Pages per        Conversion
Visits  rate Avg.   on Site      visit         Trial   Trial to Paid

   7,578    38%    00:04:23      3.72           4%          4%

What did we learn?

Since we have customers who are now paying for our service, there exists a market who are willing to pay for our service at the price-point we are offering now. However, we do not know who they are and how to reach them, and why they like our product over our competitors. Also, the metrics suggest that we have room to optimize the conversion rates through A/B testing of landing page and other aspects of the site and the service. Affiliate program was too early to add.

In hindsight, I believe what we achieved in 18 months or so, we could have learned even more in 6 months or less using Lean Startup techniques. More on what's next from here. 

Three Startup triplets that I like

Recently Dharmesh Shah blogged about Startup triplets- startup advice in exactly three words. I thought what a great idea to collect all advices from all great entrepreneurs in one place. I was not alone. It is evident in the responses it got on the net- 1434 tweets and many comments, and still counting.

I always like the number 3. 3 goals, 3 ground-rules, 3 action items, 3 options... I find it easier to deal with a list of 3 of anything. Any list longer than 3, I lose interests. In the spirit of 3, I asked myself,  if I had to choose only 3 triplets that I think are the most important to me as an entrepreneur, what would those be? So, here they are and why, in no particular order.

Just do it. The famous Nike tag line. It captures the timeless essences of entrepreneurship. The fundamental difference between success and failure. We need to start somewhere. Without the beginning, there is no being. We will make mistakes. We will fail. But we will know that we tried, which is lot better than then living with "what ifs." Kevin Dewalt, summed it up nicely on his blog and I quote,

Entrepreneurship tests us like few things in life. Jobs are trivially easy in comparison. Capital markets have a brutal way of exposing our own personal flaws in blinding bright neon signs before our eyes – frank feedback you just can’t get any other way

Build, measure, learn. The essence of "Lean Startup." If we can take the leap of faith, now we have this growing body of shared experiences about how to go about doing a startup that would give us the greatest chance of success.


Support customers maniacally. This is our ultimate goal. Anything else we do are means to this end. I recently stumbled on a post by the guys over at Scout. I could not agree more with them that we have an unfair advantage over larger incumbents when it comes to customer support. It really is easy for us to be very responsive to our customers' needs. It opens up the door to connect with our customers at a personal level like nothing else can. What can be better than this for "customer development."

What's yours and why?

Retrospective from a first-time entreprenure

It's been almost three and a half years since I started my "entrepreneurial" journey. I did not blog about my experiences so far. Everything has been written about the topic. I feel that I have nothing new to add to these shared experiences. Well, reinforcement is always good, even though everything has been told before. So, here it goes.

Getting used to no-paycheck is difficult. I quit my fulltime job to focus on my startup, Code71, (IT outsourcing services) in early 2006. I was too excited the first few months to notice that paychecks stopped coming. When I first noticed, the reality of entrepreneurship hit me. I must admit that I was a little uncomfortable in thoughts that it could take a while before I would see any paycheck again. As I had become self aware, I began to notice that I would look at interesting job posts from time to time and would toy with the idea of applying. It took me a good 6-9 months to actually stop looking at job postings (which I call a symptom of withdrawal from not receiving regular paychecks).

Brother and wife as co-founders. I started my business with my brother and my wife. Yes I know, you might be thinking what are you crazy? In my defense though, I involved them not because my relationship with them, but purely based on what they brought to the table. However, it has not been easy, I must admit. With my brother, we gradually started to step on each other's toes. He would complain that I do not treat him as a partner like I would have if he weren't my brother. And I would say the same. It got to a point where we decided to end our business relationship to save our personal. As for my wife, we are still in the business together and married...:-) But, it constantly add spice- good and bad to already eventful days of a startup business.

Emotional roller-coaster. I was aware (from other entrepreneurs) of the stress that comes with starting and running a business, yet I did not realize the extent of it until I experienced it firsthand. It is almost like mood swing. At times, I would mistake it for a split personality disorder. One day I feel I am at the top of the world, everything is looking great. The next day it would be completely opposite so much so that I would be thinking what have I gotten myself into? The worst thing was that it would spill over and impact my team. It was particularly difficult when the team is half way around the world and missing the context.

Clients amplify emotional roller-coaster effect. We work with startups and small businesses. As a result, they experience the same emotional roller-coaster themselves. This has "amplifier effect" on my emotional state- good and bad. When they are feeling great, I am doubly happy. When things are not so good for them, things are even worse for me.

Saying "No" is difficult. I would say "Yes" to any opportunity (even it did not fit with our long-term goal- technology or industry that we want to focus on, or agree to terms that we would not usually agree to) that come my way. How could I say "No" to a "potential revenue source," right? I always thought I could minimize the undesirable aspects of an opportunity while making the most out of it including short-term revenue. Sometimes we will sacrifice in the sort-term for a long-term benefit (the proverbial "carrot"), which never materializes. This got us into some situations that would leave a bad taste in our mouth. Even worse, it jeopardized our relationship with clients. I plan to say more "No" than "Yes" in 2010.

Product vs Service. When we started, we only focused on providing custom software development services. I did not plan to get into building a product ourselves. Things change. Business model evolves. We found ourselves building a SaaS agile project management software called ScrumPad in a year and half into our business. Having a taste of both service and product businesses now, I could say I like product more than service business. It is not so much because one is easier than the other. Both pose their own challenges. But, with service, it always feels like I am at the mercy of my clients, which is not a great feeling to have all the time. Whereas with a product, I feel that I have the freedom to pursue my vision and hence control my destiny. Now we are at a cross-road in our business. It remains to be seen that in 2010 how well we could make a successful transition to a product company following  lean startup framework (thanks to Eric Ries). However, I feel that we should continue with our service business in some form and fashion just because it helps us learn certain aspects of the business that could be applied to our product business. This should work to our advantage over our competitors who are pure product companies. After all, products are packaged as services (SaaS) now-a-days. I know this is contrary to popular belief, and investors would not like us for this.

It's not a sale until you get paid. I come from a technical background. Sales is something I am still trying hard to learn. It is still a work in progress. In the beginning, when I would have a conversation with any potential client, I would take any positive conversation as a sign for a sure sale (closed deal). After many disappointments, now I am aware of the complex dance that goes on during a sales process. I learned it is not a sale until you actually sign the contract. In fact, I do not even consider that a sale until I get paid.

Client feedback can be harsh. We ask our users to provide feedback every chance we get. I did not realize that it would be difficult to handle harsh feedbacks though. Fortunately, we get more positive than negative feedbacks. However, it made me realize that there exists a symbiotic relationship between entrepreneurs and their products. Every entrepreneur put hearts and souls into their products that is probably we as users are not always mindful of. Yes, some products are better than others. But all of them get the same love and care from their creators. I am now more respectful in giving feedback about any product that I use. You should be too.

Are you an entrepreneur too? Please drop a few lines to compare notes.

They stole my startup idea

I recently came across a series of posts by Steve Blank on how couple of his ideas were stolen. This reminded me of somewhat similar thing happened to me a few years ago. I want to get it off my chest by sharing it here. So, here it goes.

In 2004, I was toying with a business idea that I had for sometime and finally decided to put it on paper and get some feedback. I was still working a full-time job then. The idea was about leveraging the growing popularity and penetration of mobile phones in the villages of Bangladesh (that's where I am originally from). Although the poor farmers were using the phones to find out prices in the market, but it was limited in scope. The farmers were still going through the middlemen who were taking advantage of the price arbitrage that existed because of lack of market information available to the farmers. So, I thought what if we enabled a market place over this growing network of mobile phones and help cut out the middlemen. Fittingly, I called my venture " Grameen Bazaar." I thought who would be better to seek advice from than the person behind the company called "Grameen Phone" that, in partnership with the "Grameen Bank" (the organization behind the micro lending), brought the phones to the poor. My expectation was to see what he thought about the idea and possibly convince him to get involved in some capacity. So, I contacted him by email and sent him my biz plan. Then scheduled a phone call to discuss my idea. I was excited to talk with him on the phone only to be disappointed. He was less than encouraging. He was busy with other ventures that he would not have time to help me out. So, what I did as most wanna-be entrepreneur would have done- shelved my idea and moved on.

Fast forward a couple years. It was 2006, I finally quit my job to plunge into my first entrepreneurial venture (Code71 and ScrumPad). I had to travel to Bangladesh (that is where I setup our offshore development center). At that time I came across this new company called "Cell Bazaar." I naturally got interested and did some digging around. To my surprise, I found out, you guessed it right, that the founder of the company sure enough was the younger brother of the person I had spoken to about "Garmeen Bazaar." I was sad to learn this, not because they pursued my idea, but because I was not acknowledged in some manner- privately or otherwise.

I was happy to see someone working on an idea that I also thought worth pursuing. I subscribe to the idea that Seth always preaches- spread ideas, don't protect them. I understand that the fact of the matter was I just had an idea and did not act on it, and someone did. I guess I just wanted to see a little professional courtesy extend to me when they had decided to pursue the idea by dropping me a line-

"Hey, My brother decided to pursue the idea you shared with me. If you are still interested, let's chat again."

Am I being too sensitive about this whole thing? What do you think?

Lessons learned as a SaaS startup product owner

It's been almost 2 years since we (Code71) embarked on a journey of building a SaaS product called ScrumPad. I have been playing the role of a product owner (PO). Although I have played the PO role on projects/products for our customers, being a PO of your own product is quite different. I thought it would be good to take some time and jot down my experiences so far. So, here it goes:

Minimum viable product (MVP). Initially I did not really think in terms of let's build an MVP and put it out there for feedback & iterate. We took on this initiative to meet our needs based on our dissatisfaction with the choices available in the market at the time. So, I gave a free reign to my team to build out features. We were stung by that "SNT" bug as Dharmesh puts it. The results was, you guessed it, a sprawling a set of features. When we realized that we had piled on too many features too quick, it took us sometime to cleanup the clutter. It still is not easy, but I am getting better at restraining my and my team's enthusiasm around new features.

Usability and performance. We were focused on features rather than usability in the beginning. We thought getting the features right is more important than making them feel right. Boy, were we wrong! It hurt us more than anything else. We are still working on getting this right. I realize that this is an on-going effort. You are never done with usability and performance. In the Web 2.0 world, it is in the front and center of any product development. I now have at least one story in each sprint that focuses on usability and performance.

Brand. I thought a nice logo and a design theme can always be added later. So, I continued to put it off until when I realized lack of it is causing inconsistency in user experience. When we added a well thought out design with logo, the impact was clear. I wish we did this at the beginning. We could have saved some rework and retained some early adopters. A product without a design theme is like a boat without a radar.


Idea, Build, Product, Measure, Feedback, Learn. Yes, this is the "lean startup" cycle introduced by Eric Ries. This is Agile applied to Startups. Although we were doing Agile/Scrum even before we started working on our ScrumPad product, it was limited to only our engineering process. I came across the "lean startup" framework when I met Dave McClure and Erik Ries this September during the "Geeks on a Plane" trip to Europe. I realized what is missing from what I am doing as a PO is driving product development decisions based on the rigor of "measurement-based learning." We had a basic tracking (e.g., basic Google tracking, custom "project activity index") setup. It was not anything close to what Dave describes in his AARRR (Acquisition, Activation, Retention, Referral, and Revenue) framework for startups. Even the minimal tracking (project activity index) we put early on helped us quickly detect a potential issue we had with our initial setup process after registration. However, I always was torn between working on "a product feature" vs. "a tracking feature." Always the former would win over the later. Now, I realize it is not one or the other. They are one and the same. I need to rollout a feature along with the ability to track and measure its impact on one or more AARRR metrics.

Charging early. When we decided to open the door to our service for others, we offered it for free. It did not feel right to ask people to pay for a product that is in early development phase and not stable. I thought I would not get people to use our product and provide us "useful" feedback if I charged. The reality was quite the opposite. Yes, we saw people singing up for our service. We were excited. We were getting some feedback, but none took the time to point out what was working and what was not. So, we continued to focus on thnigs that did not matter to our users. Sure enough, when we finally thought we were ready to start charging for our service, the users fled in all directions. It was demoralizing for us, but interesting to me to experience it first-hand the importance of "charging early, and often." It took us some time to build our user base back again. This time with the fees in place, we see a stark difference in the quality of feedback. It is because now the users have a skin in the game.

What kept us alive? As you can see I made all the mistakes I can make as a first time entrepreneur PO. However, I believe we did a few things right that helped us survive our mistakes so far. First, we were putting out a new release every two weeks since we started working on ScrumPad. Second, I was able to muster up courage to stop adding new features and focus on re-factoring to get us back on track. Last but not least, I ignored the temporary inconvenience to users when we were making changes to the look and feel incrementally, even if it meant some parts of our site looked completely different from another.

Now with 2010 around the corner, I am ready to drive our product development pivoted in the lean startup concepts. I'll report back on how it is going some time next year.

Are you a PO of a startup SaaS product? I would love to compare notes. Please drop a few lines.

Why open-source project management tools do not flourish?

Unlike other open source tools, the open-source project management (PM) tools are miles behind from their commercial counter parts. I have always wondered why? Most of the projects have very low development activity, if at all. Did you wonder why too? Recently, I attended a DC Scrum user group meeting where we were discussing why we chose the tools that we are using to manage our Agile projects. Suddenly, I had this "aha" moment.

If we think about it for a moment, we will realize that we all have different ideas as to what to expect from a PM tool. We all know what some of the core features should be, but we most likely would disagree on how to go about implementing them. These differences are not technical in nature. We can be very opinionated about the details of these requirements. It is like choosing what to ware. As a result, most project management tools embody the view of the development team, the product owner to be specific. As a user, we choose the one that most closely matches our view of the world. Even if we somewhat like one, we would move on to something else and would not have the patience to wait for the tool to catch up to our taste. We act like we have ADD when it is about choosing a PM tool.

Since these open-source projects do not have a passionate product owner, there is no single ownership of a cohesive view to pursue. As a result, a developer will start working on implementing one because he has this unique idea about what this PM tool should look like, but will find it difficult to have other developers to agree with that view. Worse, he will find it even more difficult to find users who will like his view and support his effort by actively using the tool. Heck, even the developers would not use their own tool to manage their project. We know an open-source project would need an active user base to sustain and flourish.

The only way to have a project management tool to survive is a team's persistence with the tool in the early stage of the development. This is only possible in the commercial world. Each of these companies are pursuing their unique vison of what a PM tool should be in the hope that there is a large enough user base (who subscribes to this vision) to sustain their development. Thanks to all these companies sticking to their guns irrespective of the diverse nature of a very discerning users group (that is us).