Step-by-Step Instructions

Click to print worksheet.

In this exercise, you will complete Step 2 of creating your experiment.

Problems it Solves

1. Easy to define success metrics

In this exercise you are going to get clear criteria to determine a successful experiment. These numbers will not be the wild guesses of yesteryear: they are solid hypotheses based on the work you've accomplished thus far.

2. What to do if your experiment is not a success

If you exceed your success criteria, it's easy to figure out what to do next. But what if you don't reach your success criteria? Or what if you come close, but don't quite get there? This exercise will tell you what to do next, no matter how your experiment goes.

What is a Success Metric?

A number that determines whether...

...an experiment is a success or failure.

In other words, this is the number that will help you determine which path you should follow. Should you continue on the path that you are on, or should you choose a new route? Should you pivot or persevere?

This exercise provides a rigorous way to define success metrics.

Previously, common knowledge suggested that founders take their best guess at a success metric for an experiment, and hope it was right. The thinking was, better to have some number than nothing at all.

However, what usually happened was that this number would be changed retroactively to fit the results we got. We'd change the number to justify continuing the experiment because, well, at the end of the day we just made that number up anyway, so it should be okay if we changed it.

The Success Metric Stoplight solves this problem.

First, your numbers are derived from actual data - your Victory data. Your Path to Victory work has told you precisely which hypotheses need to be right in order for you to achieve your Victory. You'll use that data to determine whether this experiment succeeds or fails.

Second, this Stoplight tells you exactly what to do given the data you get:

• If you exceed your success criteria, you know you are cleared to move forward.
• If you get close to your success criteria, but don't hit it, you will be able to iterate on your Offer until you can make a decision to pivot or persevere.
• You will also know the hard and fast stopping point. You will know when your experiment has been a failure - and what to do about it.

How to Define Your Success Metric

Take out your Path to Victory spreadsheet. Look at your Offer Conversion Rate for the channel you will use for your Offer.

Step 3

Write in your metric for your first experiment.

In my case, I am going to create a Facebook ad, so I'll want to measure the ad's Click-Through-Rate (CTR). Based on my Path to Victory exercise my Offer conversion rate for a successful ad campaign is 5%.

If I can achieve 5% or more CTR, I will be in the Green Light area: I can proceed to the next experiment.

Next, I'm going to determine my Failure condition, the Red Light area. This number is the Offer Conversion rate that, if I can't reach, even with continued optimization (i.e. iterating over and over again on my Offer), I would not be able to achieve my necessary Path to Victory metric of 5%.

In my example, I will put in a number that is just less than half of my green light metric. In other words, if I can't get 2% CTR, something has gone wrong with my Offer and it's very unlikely I'll be able to iterate on it sufficiently to increase my Offer Conversion rate to 5%. In this case, something would need to change, so my next step, like at any red stop light, would be to halt, and execute my Failure Protocol (coming up in the next exercise.)

If the metric I hit falls somewhere in between theses two numbers, I will consider this my Yellow Light area. This means "proceed with caution" or Optimize. At this point, I have the option to iterate on this experiment until I get it up to the 5% criteria from my Path to Victory, or until I determine that's not possible.

If after a few iterations I cannot get to 5%, I need to go back and change something. This means something has gone wrong and I won't be able to achieve my Victory via this path. I'll need to pursue a different channel or a different customer segment.

You can see here that these numbers are derived from real data - they are not random guesses. These are the numbers you need to hit in order to achieve your Victory!

You can also see that each number tells us exactly what to do next: Proceed, Optimize or Failure Protocol.

Next Steps

• Green Light

If you have reached your Green light criteria, your next step is likely to proceed to Currency Testing, the next phase of FOCUS. The Offer is no longer your riskiest assumption; you've validated customers are interested in solving the problem. Now you need to test if you can get Currency for solving it.

Note: just because you've reached your success criteria, doesn't mean you have to move forward. If it was easy to get there, you might want to see if you can optimize further to get an even higher conversion response.

• Yellow Light

If the results of your experiment fall in the Yellow Light range, there are a number of things you can optimize from your Offer Design worksheet:

1. You can change your Channel.

2. You can choose a different Product-Centric Message.

3. You can change your Call to Action.

In fact, you already have a second Offer designed if this one does not pan out. You can choose to pursue Offer 2 from your Offer Design worksheet.

• Red Light

If your experiment doesn't not hit your Yellow light criteria, you will proceed to the Failure Protocol, which you'll develop in the next exercise.

Recap

In this exercise, you've defined your success criteria for your Offer Testing Experiment, and determined what to do depending on the data it produces.

You've set clear next steps that will help you keep your experiment time-bound.

Finally, you've identified your fall-back plan by using your Offer 2 from the Offer Design worksheet, or identifying on factor to change from the many options you have under Channels, Problem-Centric Message or Call to Action.

What's Next

Before you run your experiment, you have one more question to answer:

What do you do if your experiment fails?

The Failure Protocol exercise will walk you through exactly what to do in this situation so you're back up and innovating again as soon as possible!