Tag: Supply chain management

What is Validated Learning and how can it be applied to Kanban?

I’ve been reading the Lean Startup by Eric Ries and he has introduced me to the concept of Validated Learning. Validated Learning is the practise of effectively measuring the accuracy of assumptions and using the results of the Validation to understand whether the assumption was correct and if so, continue onto the next test. If not, decide whether your strategy, assumption or feature needs to be improved or to change direction.

The main thing that I found interesting was an example of how this could be integrated with Kanban, a practise that I am an avid user of since launching Mousebreaker.com in 2008.

The example spoke of the founder of Grockit, an online teaching site that enables students to learn either socially with other students online or individually. They assumed that a user story was not actually delivered until it had been confirmed as a success through measurement of Validated Learning.

In addition, the feature is aligned to the product’s strategy and therefore had a set of assumptions associated with it. The process of Validated Learning either proves or denies the assumptions.

If it is found that a feature has not been a success and actually improved the product, then that product is removed. In addition, the feature is aligned to the product’s strategy and therefore had a set of assumptions associated with it. The process of Validated Learning either proves or denies the assumptions.

In the example, Grockit uses A/B Testing and cohort analysis to validate the success of the feature being delivered by the development team.

A/B testing allows you to test different versions of pages or parts of pages to different proportions of your users to test a metric such as registrations or visits to sections of your site – although you can measure much more than this. You can then determine which version works best and then make the winner available for all users.

Cohort analysis is very similar but takes place a step before the A/B Testing. You may use a number of different marketing channels using a number of different messages, for example. Each user that comes from one type of channel/message would be associated with that channel/message and then tracked throughout their lifetime on your product. This would allow you to understand the cohort group’s lifetime value for example, so that you can understand the value of those messages or that channel.

You can use each of these methods exclusively or together and measure against the metrics that matter to your business. What you are looking for is actionable data, something that up you can make a decision on or understand the value of a change.

I really like this concept; however it does seem to go against the basic principles of Kanban. Kanban is an agile methodology that is used, partially, to eliminate waste during the build process of a product through practises such as limiting work in progress and kaizen – the constant improvement of the process.

My question is whether removing a feature that is not deemed a success as wastage; and if it is, could that have been avoided?

Whenever I have built a web site and we have launched a new feature, we have measured and iterated based on the data collected until it has worked – a form of validated learning. Only, of a feature has not worked, I have seldom, if ever, removed this feature. My main thought on this is that it would have been a waste of the time to develop the user story.

Perhaps I would have removed the feature if it had a detrimental effect on the metrics used to validate the functionality.

I would be happy to do this as long as it informed decision making in the future. This is what Eric Ries is arguing. A user story or feature that is being developed is actually a test of a theory or an assumption. You are assuming that this new feature will improve the product and will improve a core measurement to your business.

If it does not do this, why keep this feature live? If it does not deliver enough improvement, then why keep it live? As long as these lessons learned helps to change the strategy or assumptions around your business, you should be able to reduce wastage as your strategy should be directly influenced by validated learning through the use of your product.

I really like this concept and will keep this in mind when I next get the opportunity. I will definitely be posting about it once I have some personal evidence of this in practice.

Please comment if you have used Validated Learning in practise and how you have found it in the comments below.

A Practical example of Feature Driven Development

Feature Driven Development (FDD) is often theorised about on many web sites with blog posts, articles and essays being published on a regular basis and this blog post will give you a much needed practical example of it in use.

One article that is worth pointing out is DZone’s Introduction to Feature Driven Development. This is part one of a two part article describing a theoretical project and a theoretical team and the first three of five steps to achieving Feature Driven Development. It is extremely well written and gives you some really good insight into what is needed.

In my experience, I find that when you take these theories and methodologies and apply them to real life situations and projects, that they need adapting and shaping to fit with what you are trying to deliver.

An example of this is when I was leading the Product Development across five different web sites and one development team. The way in which I had implemented the Kanban methodology was different for each site due to different stakeholders and different commercial strategies needing to be delivered.

Anyway, back to a practical example of Feature Driven Development.

The example that I am using is the build of Mousebreaker, a casual gaming site that utilised a mixture of Kanban and Feature Driven Development to quickly and effectively deliver a new web site with a new code base in 28 days.

Traditionally, my approach had always been to gather all requirements, build the infrastructure, then the code, and finally the front end for a web site.  This information gathering and the writing of functional and technical specs can take a long time to complete. Then, when the development begins, the whole spec needed to be delivered before the site could launch. By which time 6 months has passed and requirements may well have changed and what is delivered is not necessarily what the business or the market needs.

Feature Driven Development tries to get around this by defining the requirements as features, then the business owners and development teams prioritise these features into a backlog of work and then the developers deliver these features in the order that offers the most business value.

One thing to note is that there is some pre-work that needs to happen before development can start. The general technical approach needs to be agreed; technologies need to be discussed, terminology needs to be agreed and basic development, testing and live environments need to be created.

In addition, certain standards would have been discussed such as coding, SEO and accessibility standards and any automated testing. In addition, any front/back end frame works that will be used as well should also be discussed.

If you look at the Mousebreaker site, you will see that the primary user function is for the user to play flash games. So the first feature that was worked on was that the user needs to be able to play a game on a web page.

At first, the developers approach was to start building a database infrastructure that could be used for the whole site. They were also wanting designs for pages etc. You need to be careful here as that is not what was required by the feature. All the feature required was for the user to be able to play a game. Nothing else.

So to deliver this feature, all that was required was a static html page with some embed code that would allow a user to play a game. The game needed to be in a web facing folder.

Once complete and tested, the feature can then be released.

The next story was that a user needs to be able to play all games on a web page.

This is where the database gets created and the initial html page is turned into a template. Again, the developers only needed to create a database that delivers the feature’s requirement.

In the meantime, while the initial features were being delivered, the designers were working with the development and business teams to deliver the designs for the site. There was a further feature for the site to have a premium look and feel that eventually would need to be delivered which could be applied to the site around the templates that were being delivered.

This felt a little back to front, but you need to remember that we were delivering features in the order of business priority.

As the features kept being delivered, the site quickly started to take shape. Throughout the development, the business representatives were always attending the stand ups and were constantly making decisions on scope of work and what would be required for launch.

We found that the close collaboration between the business and the development team was the most effective way of managing scope and ensuring that what the dev team delivered is what was expected.

I have applied this form of Feature Driven Development many times and I find that it really works. You do need buy in and effort from the business owners, and you do need to make sure that the developers do only focus on what is required to deliver the features rather than architecting a full solution before understanding all the requirements.

This allows more of a front to back development process. Where the features take priority over the implementation. One thing that I would like to point out is that there would be occasions where future considerations are sometimes ignored or put to one side to get functionality out and that this may result in refactoring work further down the line.

The thing to remember is that you will have already delivered the highest business value functionality required at that point and that the business will understand that any refactoring work should also have a value and then a discussion can be had about options and whether this work needs to be done or not. If it does and it takes a longer period of time, then allowances should be made for this.