Last week, Paul Dowman launched the Toronto Startup Metrics Meetup. It was a great chance to talk Lean Analytics with dozens of founders and analysts in the city, and the Bitmaker Labs offices were the perfect venue for the evening event.
Here are the slides, for those who asked.
Paul Graham has a simple definition of a startup: an organization designed to grow fast. Steve Blank adheres to a different one: an organization formed to search for a repeatable and scalable business model.
They’re both right, in a way. And that has important implications for entrepreneurs
Many people think a startup is a small business. It’s not. A small business is a small business, and if you want to change that, you’ll have to follow both Paul and Steve’s advice: start searching for a sustainable business model that can grow fast.
Let’s talk about Paul’s view first. He explains that while tech companies aren’t necessarily startups—Dell and Microsoft aren’t growing rapidly—startups might necessarily be tech companies. That’s because fast growth happens either from a big market shift, or from a dramatic lubrication of a market. Technology causes both huge shifts and copious amounts of lubrication.
Put another way, startups move fast downriver, either because they’re on a really fast river, or because they have a motorboat and everyone else is in a canoe. Graham thinks (and I agree) that to achieve fast growth, most of the time you need a low-cost, short-cycle-time mechanism. And that’s usually software, because humans make bad processes. As Andreesen said, software eats everything. People retire; software gets upgraded.
Okay, on to the other definition. Startups are searching for a scalable and repeatable business model. To me, that means it’s sustainable without the direct involvement of the people who created it, because founders don’t scale.
A startup is not a lifestyle business. A startup does not mean consulting. It does not mean bespoke.
Etsy is sustainable; one-off quirky knitted sweaters are not. Etsy is software; sweaters are slow, messy, hard-to-scale wool. Sustainable, to me, means that as the business grows, the incremental cost of acquiring additional revenues or customers descends towards zero.
A few months ago, I wrote about a hairstylist who’s setting out on her own. She’s a small business owner. She can learn plenty from Lean methodologies about how to improve her business, just as Fortune 500 companies can improve their competitiveness by using Lean approaches. But she’s not a startup by either Steve or Paul’s definition.
Since Paul mentioned a barber shop as an example of a not-startup, I wondered, “what would a hairstylist need to do to become a startup?” They’d have to be searching for a sustainable business model that could grow fast, probably because of a new market or a lubricant that changes the economics of an existing one in an important way.
Consider some of the ways a hairstylist might try to launch a startup, based on their domain expertise. They could focus on a market that they feel is growing quickly; or on a technology that makes new products possible.
The source of your growth will influence the kinds of metrics you measure, and which you consider riskiest, and how much of your product or service needs to be built before you can test it. Some of these business models are SaaS; some are e-commerce; some are user-generated content. Each makes money in different ways: subscriptions; transactions; advertising. And all of that drives the One Metric That Matters for these companies.
If you choose to innovate by focusing on a growing market (fast river), then most of your risk is about whether you can capture that market. You’ll need a really good go-to-market strategy, because you’re not really doing anything new (you’re still cutting hair, just not in the traditional walk-in hair salon way.) You’ll care about exclusivity, partnerships, and existing competitors going after the same blue ocean.
Exclusivity means you’re the only one on the fast river. Partnerships mean that others already on the river help you get there. And competitors reduce your advantage, since they have the same fast waterway as you. The longer you can stay on a fast river, the better—but these days, it’s a fair bet that competitors will quickly copy you. Worse, several Internet giants (Google, Facebook, Twitter, etc.) are really, really good at fast rivers, since they run the underlying infrastructure. They shape the riverbed, and can divert the river’s course or slow the river to their advantage—think Twitter changing its APIs, or Facebook blocking an app’s access to its graph.
(It should be noted that neither home visits nor AirBnB haircuts are growing fast enough to qualify for Paul’s “designed for rapid growth.” That kind of growth takes something truly new, and often exploding because of network effects: the phone system, the Internet, credit cards, television, the printing press.)
If you choose to innovate by focusing on a disruptive technology (the fast boat in this analogy), you’re going to care more about whether you can build and run it; whether it will work and easily replace existing solutions; and whether it will really go as fast as you think it will.
The “can you build it?” question is true only of genuinely innovative technology, and often your adoption will come from patents or the barriers to entry that you create because of your invention. Your risk is that your Arduino/Dyson/Kinect Frankenscissors might not work.
Invention is a special segment of the startup world, and its value comes from an ability to demand high margins because of the advantage the product conveys to those who use you. For example, a chip manufacturer that increases network performance tenfold charges far more than the cost of making a chip, because of the value it conveys to network infrastructure that uses it.
The “will it work?” question is about substitution. If people won’t consider your offering a viable substitute, it’s no good. If Über cars weren’t widely available at launch, they wouldn’t be a viable alternative to taxis—which is why Über paid drivers $30 an hour to sit idle when they first launched. They basically rented a city’s population of limos until their demand took off. A good rule of thumb is that something must be ten times better than the alternative. Humans have intertia, and incumbents have inertia, and you need to overcome both.
The “speed” question is about being able to achieve the volume and cycle time you expect. Let’s say you think you can do a perfect haircut in 30 seconds because of your new hair-cutting invention, which will give you a 60-fold increase in productivity. You’re still running a salon. Your business hasn’t changed (much.) But there’s real risk that some other element of your business model (say, reservation management) becomes a bottleneck that prevents growth.
If you’re doing both (new idea in a new market—fast boat on a fast river) you’ve got even more risk, because you have to focus on market fit and product fit. The best companies do this:
But it’s taken an amazing amount of dexterity for these companies to stay around, and they’ve had to adjust constantly. Microsoft nearly steered their motorboat into the shore when the Web happened, and it took a draconian intervention by Bill Gates to get them back on course. David Allen explains this history in his consequences of state interference and non-interference.
Coming upon this scene about two years ago, in late 1995, Bill Gates found that his Microsoft was caught flat-footed. The company’s cash flows were tied to desktop computing, but there was a fundamental shift underway, toward networked computing. Remarkably, among many such performances by the man, he turned his leviathan virtually “on a dime” and steamed it off in relentless and vigorous pursuit of the Internet.
In other words: Beware the founder claiming to have a fast boat on a fast river.
It should be obvious that I’m not suggesting anyone go launch Airbnb for haircuts and the other business models I’ve mentioned here. These are bad examples. That was precisely the point of Paul’s post:
…if you want to start a startup, you’re probably going to have to think of something fairly novel. A startup has to make something it can deliver to a large market, and ideas of that type are so valuable that all the obvious ones are already taken.
That space of ideas has been so thoroughly picked over that a startup generally has to work on something everyone else has overlooked.
The issue here isn’t that a hairstylist couldn’t launch a startup. It’s that the realm of hairstylist-related innovations is relatively small, and unlikely to yield something fast-growing in a big market. That’s why good ideas are scarce, and investors snap them up for what seem like unreasonable valuations. A speedboat on a fast river is a rare thing indeed, and they all want to get on board.
So consider this corollary: Invest in the founder who’s found a fast boat on a fast river, and whose idea seems daft at first and obvious in hindsight.
Many entrepreneurs delude themselves into thinking they’re building startups, when in fact they’re building businesses. There’s nothing wrong with building businesses. In fact, it’s amazing. The hairstylist isn’t a startup founder, she’s a small business owner, and that’s fine. She has many of the same problems, but she isn’t girding herself for fast growth or searching for a sustainable, repeatable model.
What a lot of entrepreneurs who run lifestyle businesses don’t get is this: if they become a startup they’re quitting their job. They’re shutting down the current business (or finding someone to run it for them as a cash cow) and launching an entirely new one. None of the companies in my nonsense example above are a hairstylist—they’re a reservation platform, or a haircutting machine, or an iPhone app.
Until they mentally “quit” their lifestyle-oriented, slow-growth, unsustainable, unrepeatable jobs, they’ll never turn their businesses into startups. And that’s one reason why it’s so hard for small business owners to become startup founders.
They need to change jobs.
Last week was Strata, O’Reilly’s conference on Big Data, ubiquitous computing, and new interfaces. While most of the discussion focused on the event itself—and the widespread adoption of Big Data—I got a chance to talk Lean Analytics with the Silicon Angle crew who were covering the event.
The Montreal GROWTalk event is a pretty packed lineup, with each speaker only having 20-30 minutes to explain a few aspects of their business or idea. I’m presenting a quick introduction to Lean Analytics, before heading off to Superconf the following day for a workshop.
Here are the slides from the event.
You don’t know what book you’re writing until you’ve written it.
As most authors will tell you, the act of writing a book—particularly on a fast-changing, widely-discussed topic—is both a creative and a destructive one. You don’t know how little you know until you start to structure your thoughts.
As Ben and I have been working on the book (now topping out at over 200 pages on our various hard drives) we’ve come up with some basic stages and gates that most startups go through from initial idea to successful exit. These map—loosely—to established frameworks like those from Eric Ries, Steve Blank, Ash Maurya and Dave McClure. We’re not sure if we have it right, yet, but this is our current working model.
We’d love some feedback on whether this is right or wrong. Obviously, some startups will enter the Lean process at different stages, or clear certain gates before others; but as a general model, this is what we’re thinking.
Now that we’ve built it, it’s time to measure and learn: what’s wrong with it?
As we’ve shared the idea of Lean Analytics, we’ve obsessed—as most authors do—with naming. When I co-wrote Complete Web Monitoring a few years ago, I talked to a friend of mine, author Mitch Joel, about its initial name: Watching websites. Ultimately, my co-author Sean and I decided that name would be misinterpreted as a book on online video streaming, and went with the more descriptive (but less sexy) title Complete Web Monitoring. When Mitch heard this, he shook his head in dismay: he thought that by clarifying what the book was, we’d dramatically reduced its audience to a particular subset.
Every author wants an Airport Book. This is the kind of book that fills the racks in an airport bookstore. It makes you feel smarter for having bought it. It’s packed with little gems you can share at conferences. And it’s often got a compelling title: Made To Stick; Switch; Lean Startup. One-word titles, with few syllables, that have a verb in them, do well.
It turns out there is, and isn’t, a lot of research behind a successful title. There’s plenty of data on book sales, of course—so much that publisher Lulu even has a title-scorer app that uses research on bestsellers to predict (in a tongue-in-cheek way) what chance a title has of becoming a bestseller.
The advent of the electronic book, and its accompanying electronic bookstore, helps collect data on what people browse, and can give a publisher early indications of success in real time.
Electronic bookstores have also polarized purchasing: getting featured in online stores is key to success, as is getting mentioned by the Oprah of your particular industry (or, for that matter, Oprah.) Unfortunately, the choice of title, cover art, and other aspects of a book remain somewhat of a dark art for traditional publishers. This New York Times piece explains it in detail.
Most in the industry seem to see consumer taste as a mystery that is inevitable and even appealing, akin to the uncontrollable highs and lows of falling in love or gambling.
…
Eric Simonoff, a literary agent at Janklow & Nesbit Associates, said that whenever he discusses the book industry with people in other industries, “they’re stunned because it’s so unpredictable, because the profit margins are so small, the cycles are so incredibly long, and because of the almost total lack of market research.”
As analytics types, we don’t like this. My biggest concern is the bubble in which we live and write. Consider another book in O’Reilly’s Lean series, Ash Maurya’s Running Lean. I’ve told plenty of people about it; it’s doing really well, and Ash is in high demand as a speaker and subject matter expert. When a potential reader comes from the startup world, they get the title immediately. But outside our world? They think it’s a book on jogging to lose weight.
No, seriously: if you asked a million random people what the book “Running Lean” was about, what do you think they’d say? Here’s a post from a car tuning forum.
In our case, many people I’ve talked to in the business world (that is, in established businesses that are well past finding their product/market fit) think “Lean Analytics” is about doing web analytics on a shoestring budget, or about Business Intelligence with a minimum of technology. One even thought it was about lifelogging to shed pounds by tracking weight. None of those topics will leap off airport shelves. This presents us with a dilemma:
It’s a tough balance to strike. It’s one we hope to understand through analysis and experimentation, but even then, this is difficult: we mostly know how to survey startup types. Tim Ferris did some in-store testing before choosing his title (making many of us wonder how many hours he squeezes into four hours.)
He took a book about the same size, put a bunch of different covers on the books, put them in the new non-fiction book section, then just sat back and observed people for the next few hours, watching their reactions. An overwhelming percentage, something like 300% more people picked up The Four Hour Work Week title than the others.
…
Ferriss and his writing team came up with 12 alternate names. To break the deadlock and to help finalize and write the great book title they eventually came up with, Timothy Ferris ran a Google Adwords campaign. ” He bought ads for relevant keywords for all twelve potential book titles and tracked which titles performed the best. The clickthrough rate for The 4 Hour Work Week was by far the highest, so that is what his book is called.”
As far as using Google Adwords to write a great book title, Ferriss proved it’s possible and as Zeigler reports, “a smart and novel approach to write a great book title. “Google Adwords is a cheap and real time focus group.”
Ben and I have a number of ideas for titles that might work. But they’re just ideas—hypotheses to be tested. We’ll try a variety of titles with different audiences and see what works, because that’s what analytical types do when confronted with uncertainty. Not sure whether we’ll camp out in bookstores, but ads and surveys seem like a good start.
On that note, we have a survey going right now with 5 name options. Previously we had many more, and ran a survey on new sign-ups for the book. After 70 or so survey completions, we took the five highest ranked names and updated the survey.
Please take a look here: help us name our book!
The survey takes no more than a minute to complete, and will be one of the data points we use in picking a final name for the book. Thank you!
As we write the book, there’s plenty we’re not including. For example, there are several things that affect what metrics you should look at: the kind of company you are, the stage you’re at, your audience, and how your buyers think about their purchases. But rather than go into detail on all of these, we’re just picking two; the others are falling on the editing room floor, which, in this case, means the blog.
Here’s something we aren’t putting in the book, but which is still interesting—and IMHO not talked about enough. It’s the way in which your customers are accustomed to buying a particular product category.
The metrics and KPIs a startup tracks will vary depending on the stage that the company is at, and the business model. A late-stage SaaS startup might care about minimizing churn and uptime, while an early-stage mobile gaming company may be obsessed with gameplay mechanics and time in game per day.
Who are your target customers? It sounds like an easy enough question, but it’s not. If you’re selling a big-ticket item to an enterprise buyer, you’ll be wading through purchasing and processes; by contrast, if you’re selling a cheap (or free!) product to an individual consumer, you’re looking at a quick, simple, highly impulsive purchasing process.
Marketers have names for these “buying mindsets.”[1]
A straight rebuy happens when the buyer is familiar with the product and the buying process, and has generally chosen the brand—booking a familiar flight using the normal carrier, for example.
An app store purchase, 1-click buying, or a recurring billing model is an example of a straight rebuy.
This is the kind of “loyal customer” behavior marketers crave. The purchase is nearly instinctive, and you care much more about whether their behavior deviates from expectations; how long the activity takes; and any signs that their loyalty might be waning. A user auto-renewing their annual service, or placing an e-commerce order from a known online merchant, is engaging in a straight rebuy.
A modified rebuy happens when the buyer is familiar with the product or service, but is reconsidering some aspect of their purchase. For example, they might be flying to a city their normal airline doesn’t serve: they have expectations, but are open to change and looking for cues.
A Dell buyer who’s selecting a new notebook model is in a “modified rebuy” mindset, comparing with constraints but still relying on familiarity.
Here, you care about ways of upselling or unseating an incumbent. Buyers are more engaged than when they’re simply doing things by habit, and you have a chance to change their behavior in ways that will help you. Someone switching photo-sharing or messaging services might be in this group.
Finally, a completely novel buy happens when someone is purchasing in an entirely unfamiliar category, or something with considerable impact or uncertainty—booking a family vacation online, or buying a house. They have little or no experience with this purchase, and you can create their buying criteria or introduce your brand to them.
When a prospect uses a comparative shopping site, or an independent ratings company like Consumer Reports, they’re making a completely novel buy.
This is generally the case when you’re a truly disruptive startup, trying to define a product category; or when making a one-time purchase towards which the buyer has little emotional attachment, such as a dishwasher.
For a completely new purchase, SEO/SEM and placement on ratings sites matter much more; you’re fighting for a customer in the court of public opinion, where you have far less control over (and visibility into) the purchasing process.
[1] http://fds.oup.com/www.oup.co.uk/pdf/bt/palmer/im07buyer.pdf and
http://www.buseco.monash.edu.au/mkt/dictionary/sss.html have some additional lists of marketing terms.
Yesterday, I had an interesting conversation with, of all people, my hairdresser. Those who know me realize there isn’t much hair to dress, so it was an unusual event to begin with. As it turns out, she’s just bought a one-person barbershop and this was her last week at her current employer’s shop.
In hushed tones, under the sound of scissors, she told me about her marketing plans: “I just got my business cards printed and I’m making a Facebook page. I don’t get Twitter.”
I gave her a few ideas:
She asked me how I was coming up with all these ideas. And I tried to explain to her (in halting French), “well, I’m thinking about what makes someone change salons. Their friend gets a good cut; they’re visiting from out of town; they just moved. And that’s the moment when you can change their minds.”
She replied, “these are great! I’m going to do them!”
I asked her, “all of them? but why?”
“Because they’re free!” she answered. “Even printing my business cards cost more than some of these.”
I agreed with her: “This is the change that businesses have undergone in the past decade. The price of marketing has gone down to almost zero, if you can just find a way to be in the right place at the right time with the right message. But most small businesses still think about marketing as fliers, a Yellow Pages ad, and some business cards.”
“So I should do all of them, right?” she asked excitedly.
And I shot her down. “No,” I said, “you’re going to try them. You need to measure whether they work. Better yet, go home, grab a few beers and a pad and paper, and figure out all the ways that someone finds out about you and decides to try you. Then, for each of those, figure out ways that you might join that conversation in a way that doesn’t suck.”
She paused for a while, forgetting to cut what’s left of my hair, and I could almost see the wheels turning.
Once she came out of her daze I said, “and then figure out how to measure it. You have a barber shop to staff—you’re the only employee. You need to kill the bad ideas fast, and find someone else—preferably a piece of software or a friend—to do the good ideas repeatedly. You’re not a hairstylist any more, you’re an entrepreneur, because you bought your own business.”
She was clearly starting to get it—although she was probably also reconsidering her decision to strike out on her own.
So I said, “this is the difference between an employee and an entrepreneur. You’re not in search of the perfect haircut, you’re in search of a sustainable way to grow your business and bring in revenue.”
It occurred to me at this point that most small business owners define themselves by the work they do within the business (cutting hair, mowing a lawn, driving a truck, running a lemonade stand) and not by the customer need they’re addressing (a neat appearance that gets them promoted, a lawn that complies with town by-laws, relocation, learning about business.)
A worker becomes an entrepreneur when they re-frame themselves. They’re no longer the cog in the machine—they’re the architect of the machine. The watchmaker, not the watch. And that means they start asking what the machine is for, how it can be improved, what else it can do, and which other machines satisfy the need.
“In the end, it may even be that your customers don’t want haircuts, they want something else—fashion consulting, pedicures, personal training, someone to talk to. Or maybe you should be a travelling stylist who does work on demand for hotels. How would you find out about that and figure out what other needs customers have that you can help with? And how could you use those to become unique, or more efficient, or more profitable?”
The conversation reminded me that lean principles apply to everything, and are increasingly relevant to small businesses. Today’s one-person company has tools at its disposal that the Fortune 500 only dreamed of a decade ago. Entrepreneurs can make data-driven decisions, run experiments, segment customers, and iterate.
They just need to stop thinking of themselves as part of the machine, and start seeing themselves as its designer.
* 4% of the population moves on that day. No, there aren’t any trucks available. Go ahead, mock us.
One of the things Ben and I have been discussing a lot is the concept of the One Metric That Matters (OMTM) and how to focus on it.
Founders are magpies, chasing the shiniest new thing they see. Many of us use a pivot as an enabler for chronic ADD, rather than as a way to iterate through ideas in a methodical fashion.
That means it’s better to run the risk of over-focusing (and miss some secondary metric) than it is to throw metrics at the wall and hope one sticks (the latter is what Avinash Kaushik calls Data Puking.)
That doesn’t mean there’s only one metric you care about from the day you wake up with an idea to the day you sell your company. It does, however, mean that at any given time, there’s one metric you should care about above all else. Communicating this focus to your employees, investors, and even the media will really help you concentrate your efforts.
There are three criteria you can use to help choose your OMTM: the business you’re in; the stage of your startup’s growth; and your audience. There are also some rules for what makes a good metric in general.
We’ve found there are a few, big business model Key Performance Indicators (KPIs) that companies track, and they’re dictated largely by the main goal of the company. For online businesses, most of them are transactional, collaborative, SaaS-based, media, game, or app-centric. I’ll explain.
Someone buys something in return for something.
Transactional sites are about shopping cart conversion, cart size, and abandonment. This is the typical transaction funnel that anyone who’s used web analytics is familiar with. To be useful today, however, it should be a long funnel that includes sources, email metrics, and social media impact. Companies like Kissmetrics and Mixpanel are championing this plenty these days.
Someone votes, comments, or creates content for you.
Collaboration is about the amount of good content versus bad, and the percent of users that are lurkers versus creators. This is an engagement funnel, and we think it should look something like Charlene Li’s engagement pyramid.
Collaboration varies wildly by site. Consider two companies at opposite ends of the spectrum. Reddit probably has a very high percentage of users who log in: it’s required to upvote posts, and the login process doesn’t demand an email confirmation look, so anonymous accounts are permitted. On the other hand, an adult site likely has a low rate of sign-ins; the content is extremely personal, and nobody wants to share their email details with a site they may not trust.
On Reddit, there are several tiers of engagement: lurking, voting, commenting, submitting links, and creating subreddits. Each of these represents a degree of collaboration by a user, and each segment represents a different lifetime customer value. The key for the site is to move as many people into the more lucrative tiers as possible.
Someone uses your system, and their productivity means they don’t churn or cancel their subscription.
SaaS is about time-to-complete-a-task, SLA, and recency of use; and maybe uptime and SLA refunds. Companies like Totango (which predicts churn and upsell for SaaS), as well as uptime transparency sites like Salesforce’s trust.salesforce.com, are examples of this. There are good studies that show a strong correlation between site performance and conversion rates, so startups ignore this stuff at their peril.
Someone clicks on a banner, pay-per-click ad, or affiliate link.
Media is about time on page, pages per visit, and clickthrough rates. That might sound pretty standard, but the variety of revenue models can complicate things. For example, Pinterest’s affiliate URL rewriting model, which requires that the site take into account the likelihood someone will actually buy a thing as well as the percentage of clickthroughs (see also this WSJ piece on the subject.)
Players pay for additional content, time savings, extra lives, in-game currencies, and so on.
Game startups care about Average Revenue Per User Per Month and Lifetime Average Revenue Per User (ARPUs). Companies like Flurry do a lot of work in this space, and many application developers roll their own code to suit the way their games are used.
Game developers walk a fine line between compelling content, and in-game purchases that bring in money. They need to solicit payments without spoiling gameplay, keeping users coming back while still extracting a pound of flesh each month.
Users buy and install your software on their device.
App is about number of users, percentage that have loaded the most recent version, uninstalls, sideloading-versus-appstore, ratings and reviews. Ben and I saw a lot of this with High Score House and Localmind while they were in Year One Labs. While similar to SaaS, there are enough differences that it deserves its own category.
App marketing is also fraught with grey-market promotional tools. A large number of downloads makes an application more prominent in the App Store. Because of this, some companies run campaigns to artificially inflate download numbers using mercenaries. This gets the application some visibility, which in turn gives them legitimate users.
No company belongs in just one bucket. A game developer cares about the “app” KPI when getting users, and the “game” or “SaaS” KPI when keeping them; Amazon cares about “transactional” KPIs when converting buyers, but also “collaboration” KPIs when collecting reviews.
There are also some “blocking and tackling” metrics that are basic for all companies (and many of which are captured in lists like Dave McClure’s Pirate Metrics.)
A second way to split up the OMTM is to consider the stage that your startup is at.
Right away you need attention generation to get people to sign up for your mailing list, MVP, or whatever. This is usually a “long funnel” that tracks which proponents, campaigns, and media drive traffic to you; and which of those are best for your goals (mailing list enrollment, for example.)
We did quite a lot of this when we launched the book a few weeks ago using Bit.ly, Google Analytics, and Google’s URL shortener. We wrote about it here: Behind the scenes of a book launch
Spoiler alert: for us, at least, Twitter beats pretty much everything else.
Then there’s need discovery. This is much more qualitative, but things like survey completions, which fields aren’t being answered, top answers, and so on; as well as which messages generate more interest/discussion are quantitative metrics to track. For many startups, this will be things like “how many qualitative surveys did I do this week?”
On a slightly different tone, there’s also the number of matching hits for a particular topic or term—for example, LinkedIn results for lawyers within 15km of Montreal—which can tell you how big your reachable audience is for interviews.
There’s MVP validation—have we identified a product or service that satisfies a need. Here, metrics like amplification (how much does someone tell their friends about it?) and Net Promoter Score (would you tell your friends) and Sean Ellis’ One Question That Matters (from Survey.io—”How would you feel if you could no longer use this product or service?“) are useful.
Increasingly, companies like Indiegogo and Kickstarter are ways to launch, get funding, and test an idea all at the same time, and we’ll be looking at what works there in the book. Meanwhile, Ben found this excellent piece on Kickstarter stats. We’re also talking with the guys behind Pen Type A about their experiences (and I have a shiny new pen from them sitting on the table; it’s wonderful.)
Then there’s Feature optimization. As we figure out what to build, we need to look at things like how much a new feature is being used, and whether the addition of the feature to a particular cohort or segment changes something like signup rates, time on site, etc.
This is an experimentation metric—obviously, the business KPI is still the most important one—but the OMTM is the result of the test you’re running.
There’s business model optimization. When we change an aspect of the service (charging by month rather than by transaction, for example) what does that do to our essential KPIs? This is about whether you can grow, or hire, or whether you’re getting the organic growth you expected.
Later, many of these KPIs become accounting inputs—stuff like sales, margins, and so on. Lean tends not to touch on these things, but they’re important for bigger, more established organizations who have found their product/market fit, and for intrapreneurs trying to convince more risk-averse stakeholders within their organization.
A third way to think about your OMTM is to consider the person you’re measuring it for. You want to tailor your message to your audience. Some things you share internally won’t help you in a board meeting; some metrics the media will talk about are just vanity content that won’t help you grow the business or find product/market fit.
For a startup, audiences may include:
Let’s say you’ve thought about your business model, the stage you’re at, and your audience. You’re still not done: you need to make sure it’s a good metric. Here are some rules of thumb for what makes a number that will produce the changes you’re looking for.
There’s another important aspect to the OMTM. And I can’t really explain it better than with a squeeze toy.
Nope, this isn’t me. But sometimes I feel like this.
If you optimize your business to maximize one metric, something important happens. Just like one of the bulging stress-relief toys shown above, squeezing it in one place makes it bulge out in others. And that’s a good thing.
A smart CEO I worked with once asked me, “Alistair, what’s the most important metric in the business right now?”
I tried to answer him with something glib and erudite. He just smiled knowingly.
“The one that’s most broken.”
He was right, of course. That’s what focusing on the OMTM does. It squeezes that metric, so you get the most out of it. But it also reveals the next place you need to focus your efforts, which often happens at an inflection point for your business:
Whatever your current OMTM, expect it to change. And expect that change to reveal the next piece of data you need to build a better business faster.
(* with apologies to the excellent Café Baobab in Montreal, where I’m doing exactly that.)
When High Score House joined Year One Labs, we didn’t let them code for a month.
The story of how they found their target market is fascinating. It underscores one of the basic tenets of Lean: get out of your building and actually talk to people.
The modern world isn’t inclined to physical interaction. We have dozens of ways to engage people at a distance, and when you’re trying to find a need, they’re mostly bad. Unless you’re face-to-face with prospects, you won’t see the flinches, the subtle body language, and the little gasps and shrugs that mean the difference between a real problem and a waste of everyone’s time.
That doesn’t mean technology is bad, however. On the contrary, we have a set of tools for finding prospects that would have seemed like superpowers to our predecessors. Before you get the hell out of the office, you need to find people to talk with. If you can find these people efficiently, that bodes well: it means that, if they’re receptive to your idea, you can find more like them and build your customer base.
Here are some dumb, obvious, why-didn’t-I-think-of-that ways to find people to talk with, send surveys to, and grow your mailing lists and outreach efforts.
For startups, Twitter is a goldmine. Its asymmetric nature—I can follow you, but you don’t have to follow me back—and relatively unwalled garden means people expect interactions. And we’re vain. If you mention someone, they’ll come find out what you said and who you are. Provided you don’t abuse this privilege, it’s a great way to find people.
Let’s say you’re building a product for lawyers and want to talk to people nearby. Put keywords and location information into Twitter’s advanced search:
Right away, you’ll get a list of organizations and people who might qualify:
Now, if you’re careful, you can reach out to them. Don’t spam them; get to know them a bit, see where they live and what they say, and when they mention something relevant—or when you feel comfortable doing so—speak up. Just mention them by name, invite them to take a survey, and so on.
Another huge boon to startups everywhere is LinkedIn. You can access a tremendous amount of demographic data through your searches:
You don’t need to connect to these people, because you can just find their names and numbers, look up their firms’ phone numbers, and start dialing. But if you do have a friend in common, you’ll find that a warm intro works wonders.
Facebook is a bit more risky, since it’s a reciprocal relationship (they have to friend you back.) But you’ll get a sense of the size of a market from your search results alone, and you might find useful groups to join and invite to take a test or meet for a focus group discussion.
Google makes it really easy to target campaigns. If you want to promote a survey or signup somehow, you can do so with remarkable precision. In the first step of setting up an Adwords campaign, you get to specify the location, language, and other information that targets the ad:
Once you’ve done that, you can create your message. This is an excellent way to try out different taglines and approaches: even the ones that don’t get clicks show you something, because you know what not to say. Try different appeals to basic emotions: fear, greed, love, wealth, and so on. Learn which gets people clicking, and which keeps them around long enough to fill out a survey or send an email.
Some of this stuff seems blindingly obvious. But a little preparation before you get out of the office—physically or virtually—can make all the difference, giving you better data sooner, and validating or repudiating business assumptions in days instead of weeks.
