The Lean Enterprise Experiment Canvas
This is a guest post from Eric Klaassen of Bloom, a consulting firm that helps companies grow online. We first met Bloom late last year in South Africa, and they’ve been pushing the envelope of applying Lean Startup concepts to big, established companies.
The success of the lean start-up methodology is increasingly resonating in large enterprises. Companies like Intuit, Amazon, GE and many others are implementing key principles of the lean start-up in order to deal with the complexity of their markets and the increasing speed of disruptive innovation.
Our company, BLOOM, has the main objective to grow companies online. While doing so, we see companies achieve great results using the lean start-up principles and tools. However, a challenge that especially the larger companies face, is translating the lean start-up theory to an enterprise setting. As the table below depicts, many differences between start-up and enterprise are at the root of this challenge.
One method that has proven to be very applicable in both start-ups and large enterprises that are among our clients is the Lean Analytics Cycle. This process provides the much-needed rigour in translating fundamental business problems into metrics that matter, creating hypotheses in order to test them, and driving change in the business from everything you learn.
The fact that the lean analytics cycle can also be applied effectively in large enterprises triggered us to translate it into a tool, based on Javelin’s Experiment Board.
Lean Enterprise Experiment Canvas
Although the canvas is designed to be self-explanatory, it could be helpful to have guide to use the canvas in your team. The text below explains how to use the lean enterprise experiment canvas.
1. Define most important metric and draw a line in the sand
Your most important metric is the one that allows you to track how changes in your products and services impact your business goals. For a start-up this means finding product-market fit, and a sustainable business model.
For a large company, the most important metric differs between departments. Customer service might focus on customer retention, where the IT department steers on number of roadmap items delivered. What is the one metric that helps your department contribute to the overarching goal of the company?
Three criteria help choose the one metric that matters: the business you are in, the growth stage of a company and the audience.
Once you have decided the metric to focus on, it is important to define the current value of the metric before you start experimenting (=set the baseline). If you don’t know what the current value it is, go find out. If you can’t find out, develop the instrumentation to do so.
With the metric and its baseline in mind, you need to set a target value for this metric in order to manage expectations across the team—in other words, you draw a line in the sand for everyone to see. Make sure the target is ambitious, edging on the uncomfortable. Better set high goals and not fully reach them than to aim low. This does not mean that small achievements (e.g. completed an iteration) shouldn’t be celebrated. Celebrate quick wins to boost team morale, but never lose sight of the work that’s still ahead. And if the goal proves impossible: remember that the line is set in sand—not stone. If achieving it proves too easy or too hard, you can change it in a later stage.
Additionally, we suggest you define a control variable to keep track of. This is advisable as experimentation comes with a certain level of risk. This way we ensure we are not improving one KPI at the cost of business as usual. For example: removing a complaint form will bring complaints down to zero, but it surely won’t improve customer satisfaction.
2. Identify and prioritize issues from your customer’s perspective
Once you start analyzing your product or service, you will often identify different problems. This number grows exponentially with the added complexity of having multiple business units, product, market segments and customers. Therefore it is important to keep in mind that not every problem is equally important.
A problem is phrased from customer’s perspective, not only forcing the team to clarify exactly the problem at hand, but also making it easier to share with other departments.
Before a singular observation is deemed an actual systemic problem, it should first be supported by patterns in data, customer feedback or additional anecdotal evidence. When support is found, it can be identified as a (validated) problem.
As to not waste scarce (development) time, efforts should be directed at problems with both high potential and high importance. High potential means that there is a lot that can still me improved; high importance means that the problem has a large business impact. Prioritising problems this way has a beneficial side effect: because you are able to show how important a certain problem is, it is much easier to obtain buy-in from stakeholders.
3. Define possible solutions
Defining the solutions should be done in cross-functional teams. All employees are able to provide valuable insights against the backdrop of why a certain problem exists and how it might be solved. The CEO has a strategic high-level overview, the customer service representative understands the most important complaints, and a developer might know how to solve a problem from a technical point of view. Including more than one function in defining the solution, limits the chance that the genius to your local problem has a negative impact on the business as a whole.
It is important to keep in mind that solutions can be of a more incremental or more radical nature. The low-risk incremental experiments often receive more support from the organization and its leadership. This is a common trap: by experimenting with incremental solutions, you can only climb towards a local optimum. To also allow identification of the radically more effective solutions, experiments for solutions should be a mix of iterative improvements and larger leaps. The iterative improvements help you climb to the top of the mountain that you are on, and the leaps ensure you find the highest mountains.
4. Decide on the test method which allows for maximum learning with minimal amount of resources
A significant part of waste prevention lies in the determination of the minimal effort needed to validate a solution. For start-ups this is often relatively easy; they can move fast and break stuff.
In contrast, large enterprises need to be more careful as there is more at stake (e.g. their reputation). When building an MVP as a test method, keep in mind that the minimal version of a feature should actually solve the original problem.
In addition, the test is only useful if it enables you to act on the results. Relevant options for minimum viable features for start-ups, could also be useful in the enterprise. In some cases, more creativity and care is required.
5. Define success before actually running the test
There is a strong cognitive bias to look for positive results in a test. Make sure to define success criteria upfront in order to prevent yourself from being overly optimistic after the test. Try to phrase your success criterion as: “During the test, I expect strong signal from at least X% of visitors/customers”.
6. Get out of the building
As indicated before, experimentation in start-ups and enterprises serves two different purposes. Start-ups are looking for a sustainable business model, and experiment to find product-market fit. Enterprises already have a customer base and execute a repeatable and scalable business model.
For an enterprise, the goal of getting out of the building changes towards finding out what provides the most additional value to your existing customers. Methods to do so in enterprises include:
- Customer interviews
- Analytics
- NPS surveys
- Focus groups
- Usability research
7. Analyze the results and check if you moved the needle
This is a critical step in the process. After you have run the test it is time to see if you actually moved the needle. Based on the test results you have three options
- Pivot, try to solve a different problem
- Persevere, go to the next column and try again
- Declare a success and implement full solution
It would be great if all tests are an amazing success, but in reality tests will be invalidated. This is not a bad thing at all. At the very least you prevented a lot of time wasted fully implementing a solution without validating it first. If a solution is invalidated we can choose to focus on a different problem (pivot/give up), or we can think of a different solution. Because we defined the problem as both important and with high potential, you probably want to do the latter.
If a test is successful it is time to scale up and create a lasting solution involving a larger team and tighter integration with the existing business.
Additional resources
The Bloom team has created a Slideshare presentation that annotates the steps described above. We’ve embedded that presentation below so you can see how they move through the Lean Experiment Canvas at each stage.
The enterprise experiment canvas was developed based on our experience applying lean in large enterprises. By publishing our tool we hope you can bring the approach to your organization as well, so feel free to download and use the template.
If you have any questions regarding the content, or if you want to share your experiences from applying these tools in practice, use the comment form or reach out directly at any time
Follow
Leave a Reply