A Better way to Name a Company

tl;dr

We used clever survey questions to quickly test the memorability, spellability and emotional response of potential company names. Glad we did – we almost picked a shitty one.

Goddamn Domain Squatters

What should be the joyous process of naming a company, quickly devolves into:

Founder A: Are you kidding with me?! How is every single good domain taken!?
Founder B: Let’s make up a word. How about, “ooVooFoo.com?”
Founder A: Maybe “KillerKittenKites.co?”
Founder B: TwoGirlsOneCup…dot ly is available.

Most of the time we’re left with abysmal choices and forced to argue over which one sucks the least.

Some friends and I who were building a crowdfunding aggregator wanted to avoid all that, so we came up with a way to test our company/domain names with customer data. Here’s how…

Step 1: Crowdsource Ideas

SquadHelp is 99Designs for domain names. tweet-this-button That means you pay a couple bucks and SquadHelp users will find you some 500+ domain names, all of which are currently available. And if you don’t like any of them, you don’t pay anything.

Note: 99% of the suggestions are going to suck, but I always find that are good ones are worth the money, and they provide inspiration to come up with my own names.

Looking over the list of possible names for our crowdfunding aggregator, we chose our top 3:

  1. altFunder.com
  2. ChangeFunding.com
  3. ThingsWeStart.com

altFunder.com was our favorite name going in to the experiment, the next step of which was deciding…

Step 2: What to Measure?

After reading up on what makes for a good company name, we decided to measure the following characteristics of each name:

  1. Memorability – If users can’t remember your name, they can’t tell their friends about it. tweet-this-button
  2. Spellability & Hearability – If users can’t spell your domain, they could become someone else’s. tweet-this-button
  3. Associations
    1. Emotional associations – what feelings do these names evoke? For more on why this is important see Selling the Why
    2. Image associations – Names people automatically associate images with are more memorable.
    3. Competitor associations – The internet told us you want to avoid a name that gives your competition more clout. Seemed like reasonable advice.

Next up…

Step 3: Design the Experiment

Here’s a video tour of the survey we designed (i.e. why you’re reading this post):

If you don’t want to watch the video, here are the key questions we asked:

To test hear/spellability (try it):

1. Play this audio clip only once and type in the name of the website you hear:

What website did you hear? [input box goes here]

To test memorability:

After speaking & showing the domain names, we’d ask them a bunch of “busy work” questions (e.g. demographic questions, trivial math questions, etc.) to help them forgot the less memorable names:

12. What is the 9th word of the 1st paragraph on this page?

  • eat
  • faster
  • rubber
  • printing

With their minds distracted for a bit, we’d then ask:

Testing domain name memorability

Testing domain name memorability

To test Associations:

For each of the associations (e.g. emotional, image, and competitor) we asked them simple questions like:

  • What images come to mind when you hear this time?
  • How do you feel when you hear this name?
  • What organizations come to mind when you hear this name?

Btw, we used nsurvey for this – an open source survey tool that gave us the power & flexibility we needed. With the survey designed, it was time to…

Step 4: Run the Experiment

Pssst: If you like this blog post so far, you should subscribe (email or RSS)! The next one will be an in-depth look at Steve Blank’s new collaboration with Startup Weekend.

The site was B2C, so we decided to throw our survey up on Mechanical Turk.  We paid $0.10 for each respondent and we stopped after we got 50 (all told $5).

Mechanical Turk (mTurk) domain name survey HIT

mTurk naming survey HIT with bonus fraud detection

Fraud Detection: we had quality assurance questions built into the survey (e.g. what’s the 9th word in the second paragraph, what’s 16 + 32, etc.) to detect cheaters. We gave cheaters a different password than people who took the time to fill out the survey thoroughly.

Step 5: Evaluating the Results

Once the survey results were in, I threw them in a spreadsheet and scored each of the questions like so:

Hearability/Spellability: +2 points for a name each time it was spelled properly

Here are the Hear/Spellability results for our names:

Domain name hearability/spellability results

Domain name hearability/spellability results

Memorability: +2 points each time the name was properly recalled

Domain name memorability results

Domain name memorability results

Image associations: scored subjectively by me (actual responses below)

Positive (+1 point): “people protesting”, “trees, plants, soil, nature”, “tattooed girls and rock music”
Neutral (0 points): “people”, “money”, “change”, “coins”
Negative (-1 point):  “a steaming pile”, “a payday loan center”, “my cluttered basement”

Domain name image association results

Domain name image association results

Feeling associations:

Positive (+1 point): “curious”, “hopeful”, “interested to know what people are starting”
Neutral (0 points): “nothing”, “innovation, kind of a cheesy name”, “mildly inspired”
Negative (-1 point): “annoyed”, “confusion, boredom”, “lame”

Domain name feeling association results

Domain name feeling association results

Competitor associations:

Positive (+1 point): “none”, “no particular companies come to mind”
Neutral (0 points): “charity”, “Kickstarter”, “red cross”
Negative (-1 point): “republican websites”, “Obama campaign”, “payday loans”

Domain name existing websites association results

Domain name existing websites association results

Then we weighted the score of people who we thought were most like our targeted users (e.g. responses from Facebook and Kickstarter users got bumps) and added them all up:

Domain name final weighted results

Domain name final weighted results

View/download the complete spreadsheet here: http://sdrv.ms/Z9941S

Conclusion

We were wrong!  We were leaning towards altFunder.com because it was short & edgy (and thus we assumed more memorable).

Turns out, ThingsWeStart.com was not only more memorable, it was also easier to hear/spell and it had much more positive associations.

For a couple bucks and a couple hours, we saved ourselves thousands of $’s by not picking the crappy name and having to change it later, or trying to buy one from a squatter.

Takeaway:

Don’t waste time debating product names…test them. tweet-this-button

Tips & Tricks

- Before you post a survey about your domain names, be sure to purchase them all. Don’t want one of your clever respondents to snipe them from you.

- Most domain registrars offer a return policy. We bought all 3 domains we were testing, ran our test, and then got a refund for the 2 we didn’t use!

- Another great test would be to measure CTR on ads, or A/B test on a landing page. We didn’t have time for that, but I’d love to hear if you’ve run an experiment like that.

As always, comments and questions are appreciated!

If you liked this - consider sharing it so your friends don’t buy shitty domain names. You may also want to subscribe for new posts via Email or RSS. Our next article will be detailed explanation of Steve Blank & Startup Weekend’s Customer Development collaboration – SWNext – and how we think it fits into the greater startup ecosystem.

First time to Customer Development Labs?

Check out our most popular experiments:

  1. Interviewing 100 customers in 4 hours with MTurk
  2. Crowdtesting: How many $’s is your idea worth?
  3. How assumptions made an ass out of my startup

Crowdtesting: How many $’s is your idea worth?

Be your hairy best

Look your hairy best!

Imagine you come up with an app that eliminates the need for haircuts. It’s amazing, it’s innovative, it’s game-changing.

You want to milk this opportunity for everything it’s worth, but you’ve got some questions on how to do that:

Pricing. Since just about everybody can use it, will you make more money charging $0.99 for it, or since it’ll save a few people a lot of money, should you charge $99?

Target Segment. Is it best to target Supercuts customers who clearly hate getting haircuts so much they’re willing to walk around looking like shit?  Or should you target vain power brokers who would love to look perfect all the time?

Marketing. What’s the best marketing message:

Look your hairy best!

or

No more hair salon small talk.  You’re welcome.

Before Crowdtesting

If you wanted to know which combination of the above generated the most revenue, you had one choice:

  1. Build the product, and then a/b test it.

Sure, you could a/b test landing pages before you built the product to see which combination got you more email addresses, but you’re not going to be able to pay employees in email addresses.  And what if the landing page skews your results (e.g. what if Supercuts customers give out their email addresses more easily than CEOs)?

At the end of the day, we want to make our business decisions not counting visitors, Facebook likes, or email signups.  We want to base them on $.  Crowdtesting helps us do that.

What is Crowdtesting?

Crowdtesting is the combination of crowdfunding and a/b testing to validate business model hypotheses. But that’s a boring definition.  This is better:

Crowdtesting will tell you how much $ your idea is worth, before you build it.

Here’s the video version from the 2012 Lean Startup Conference:

Crowdtesting in action

Some screenshots from the “crowdtesting” experiment currently running for Bounce.

Crowdtesting Step-by-Step

Step #1 – Come up with an idea for a product

While the principles apply to any opportunity, this particular technique is probably best suited for B2C opportunities that want to make money.  If you’re doing an ad supported, or “why does everyone keep asking us how we’ll make money?” play, I’m sorry, I don’t have answers for you.

Step #2 – Figure out what you want to test

In my case, I wanted to know which of these would make more money:

  1. Selling Bounce directly to people who run late
  2. Offering Bounce as a gift people could give to others who run late

and then I wanted to know the optimal price point for the winner of the above.

Step #3 – Fork Selfstarter

The Lockitron folks were kind enough to open source the crowdfunding platform they built – Selfstarter.  That makes it easy for us to launch our own crowdfunding project, and incorporate a/b testing.

Step #4 – Setup Amazon Payments

It can take a couple days to get your Amazon Payments account approved, which you’ll use to accept people’s credit card info, so be sure to start this process well in advance. Instructions are in the Selfstarter FAQ.

Note: I got a “sorry you don’t meet the requirements of a business account”-type email when I first signed up but when I told them what I was doing, I eventually got approval.

Step #5 – Setup A/B Testing

The Experiments functionality within Google Analytics is terrific for this kind of project.  I actually tried to use them but I ran into trouble running two Selfstarters at once.  I’m sure it’s a solvable problem, I just didn’t have time to resolve it.

Instead I simply ran one variation at the time (e.g. $10 instead of $5), and added a “variation” column to my orders table so I could count the number of pre-orders for each variation. Since the vast majority of my traffic was driven by an email list I segmented (more below), I was able to control exactly how many people saw each variation, and could compare apples-to-apples.

Step #6 – Launch

Yeah, baby!

Step #7 – Send traffic

Now you need to send traffic to your campaign. The great part about this step is that it’s fantastic practice to see how easy it’ll be to generate interest if you actually build the product.

As for how much you’ll need, I’m no statistician, but my thinking is, “just enough to run your experiment.”  So far my experiments have been fairly conclusive, so I’ve gotten by with a couple thousand visitors for each variation.

As far as how I generated my traffic.  I had posted a landing page collecting email addresses for people interested in Bounce for iPhone, before.  The majority of my traffic came from Hacker News, press, and emails to folks who signed up on Bounce’s landing page.

Step #8 – Analyze the results

Once the orders come in, the fun starts!

conversion_rate_spreadsheet

The metric I found most useful was $/visitor.  For example:

  • With a $5 price tag, 1.4% of visitors pre-order Bounce. That’s $.07/visitor.
  • With a $10 price tag, 1.7% of visitors pre-order Bounce. That’s $.17/visitor.

Sample size: 11,108

The combination of messaging and price points that generates the biggest $/visitor is the combination you want.

Important Notes:

There are a few non-obvious things you want to keep in mind when launching a crowdtesting project:

Crowdtesting isn’t perfect.  I’m clearly a fan, but crowdtesting is by no means perfect. It’s probably best suited for B2C plays that can charge money. We also don’t have data yet on how accurately it predicts sales once the product launches. You need to be very aware of where your traffic comes from, because that can skew your results as well. Some mitigations for these issues are below. Overall, crowdtesting is no silver bullet, but it’s clearly better than email sign-ups on a landing page or building a product no one wants.

Your goal ISN’T to raise lots of money.  Avoid trying to bring in tons of money during this campaign, that’s not the goal.  The goal is to test a hypothesis in a scenario that resembles market conditions when your product actually launches.  If you make getting lots of money a priority, you’re likely to try tactics that won’t be applicable when your product launches.  In fact, I eliminated the total $ raised from my crowdtest altogether – Bounce’s published “goal” was in terms of backers (5,000).

Don’t sell your story.  A lot of consumers support crowdfunding projects not because they want the rewards, but because they want to support the person raising the money.  You don’t want that.  You want people pre-ordering your product because they want it so badly, they’re willing to put down money early, to ensure it gets made.  Avoid talking much about yourself during the campaign and instead focus on the value prop – that’s what you’re trying to test.

Do no harm.  Crowdtesting only works because consumers trust the crowdfunding process.  If crowdtesters start charging credit cards before they build their products, don’t deliver on the promises they make, or otherwise mislead consumers, our customers will lose faith in the process and the tool will lose its value.

Set your goal high.  Remember, your primary objective is to test a hypothesis.  If you set a goal that’s low and you hit it, you’re obligated to build the product (see “Do no harm”), even if it turns out your $’s/visitor is much less than you need for this venture to be profitable.  Instead, set your goal to a high, but not outrageous level so that if you happen to hit it, you’d be happy to build the product.

Prime the pump.  You know how baristas never put an empty tip jar out?  It’s because people feel weird being the first to do something (e.g. first person to tip, first person to back a crowdfunding project, etc.).  Since we want to mimic real-world market conditions as quickly as possible, we don’t want the first visitors to our site to feel like “oh, well no one else has supported this, so it must be dumb.”  To avoid that, I started my backer count at 114 (with a goal of trying to reach 5,000).  It wasn’t so high that I felt consumers would feel mislead about the popularity of the product, but it also wasn’t an empty tip jar.

Don’t charge credit cards until the product launches.  Kickstarter lets you have customer money as soon as you reach your goal, but unless you have to have the money to pay for upfront costs, I’m a much bigger fan of not charging customers until you’ve actually launched the product.  This will build more trust with your customers, it gives you an incentive to deliver, and if you happen to miss a deadline, at least you earn interest on your customer’s cash.

Only the beginning

This was a first crack at crowdtesting. As we learn more we’ll continue to post information. If you’re interested, you can subscribe for updates via Email or RSS.

Please, if you have any questions or comments send them our way. Also, if you run your own crowdtest, we’d love to hear about it (and have you post about it)!

First time to Customer Dev Labs?

If you found this experiment interesting, you may dig some of our others:

  1. Interviewing 100 customers in 4 hours with MTurk
  2. How assumptions made an ass out of my startup
  3. How we Found Customers to Start Developing

Join the experiment – follow along via Email or RSS for updates and future experiments.

Crowdtesting: what do you think?

I’m giving a talk next week on crowdtesting and I’d really love your thoughts before I do. I’ll be giving it at the Lean Startup Conference so if you’re reading this, you know the target audience. It’s you.

Here’s a run through (note: it’s < 5 minutes):

So…

  1. What questions do you have?
  2. What can I cut?  Aka, when did you tune out and flip to Facebook?
  3. Anything else?

You’re my customers so let me have it in the comments, or you can email me: Justin@needToBounce.com.

See it Live

If you’d like to hear the final version live (oh, and some other no-name speakers Eric Ries, Steve Blank, Ash Maurya, etc.) I’ve got two suggestions:

  1. If you’ve got $900 and can/will be in SF, come in person! Scratch that, I’ve got your back.  Take 15% off (coupon code: speaker15)
  2. Watch for almost free at a local simulcast.  John Sechrest is running one in Seattle, and there are also tons of others running throughout the world.

Thanks again for your feedback!

Join the experiment – subscribe via Email or RSS for tips on how to run your own crowdtesting experiment which I’ll be writing up soon-ish.

We Should Hang out Sometime

Sorry for the lack of articles lately.  Pick any of the following as an excuse:

  1. Been busy building stuff
  2. Been busy testing stuff
  3. Been fighting with my parole officer over the definition of “recreational” drug use

That said, I’ll be at the following events and if you will be too, coffee is on me.  It would be my pleasure to treat for caffeine and chat about lean, customer dev, and anything else that suits you, since I haven’t been of any help here in the last couple months.

Lean Startup Machine Seattle: Nov 2nd – 4th

Lean startup machine logoAs I’ve mentioned before, LSM is an incredible experience.  I highly recommend it to anyone serious about starting a company.  The lessons you’ll learn over a weekend will literally save you months of time.

If that endorsement isn’t enough, here are two more reasons you should go:

  1. LSM will pay for you to attend a Startup Weekend.  That means you’ll learn how to test your business model assumptions during LSM, and use that knowledge to build a product customers actually want at a Startup Weekend – all for the price of one LSM ticket.  Double the learning and double the experience for the price of one LSM ticket.
  2. There are a handful of 50% coupon codes left.  If this link still works, it’ll get you admission to Lean Startup Machine and Startup Weekend, for $149.  I previously said LSM alone was worth $500, so to get LSM + SW (plus food & drink) for $149…it’s really…amazing.

I’ll be there helping teams test their hypotheses.  Really, there’s no reason you shouldn’t be there.

NWEN Entrepreneur University Seattle: Nov 9th

NWEN Entrepreneur UniversityNWEN’s got some big names speaking at their Entrepreneur University.  If you’ve ever heard of Cranium, Molly Moon’s Ice Cream, or Jawfish Games, you’ll know the keynote speakers.

You may have also heard of some of the lesser known speakers.  Mark and I will be doing a breakout session on “Interviewing Customers the Easy Way.”  It will be a live version of our How to Interview 100 Customers in 4 Hours with Mechanical Turk post.

We’ll pick a couple startups out of the audience, learn a little bit about their target customers, and then see if we can interview a couple of them during the session so everyone can see easy and powerful interviewing customers is.

Founder’s Institute Seattle: Nov 27th

Founders InstituteIf you’re a member of the Founder’s Institute Seattle session, I’ll see you for a talk on Product Development.  More specifically, we’ll be narrowing down the definition of MVP (which runs the gamut depending who you talk to).

If you’re not a member of FI, I think they’ll record the talk so if possible, I’ll post a link here afterwards.

And certainly not least…

Lean Startup Conference San Francisco: Dec 3rd – 4th

Lean Startup ConferenceI am nothing less than humbled and honored to be speaking at this year’s Lean Startup Conference with the likes of Steve Blank, Eric Reis, Ash Maurya, Patrick Vlaskovits, and on, and on… (oh, how clumsy of me. I just dropped all those names…)

My talk, “Crowdtesting: Using Crowdfunding to Test an MVP” will be only 5 minutes of what’s bound to be a day of innovative ways to build successful companies faster.  It feels like a combination of TED + Ignite dedicated to Lean Startups.  Come if you can.

Huge Thanks

To Ramphis Castro & Marcos Polanco for getting the speaking ball rolling, and to Mark Horoszowski for not only his encouragement, but his enthusiasm, coaching (and videography) during the Lean Startup Conference application process.

I’m Serious

With this much yapping, I’m may not be posting much over the next couple months, so like I say, please hit me up if you’re attending any of the above.  I’d love help however I can.

Join the experiment – I’ve got killer data for our next post (testing your startup’s name). Subscribe via Email or RSS to make sure you get it.

Help Investors Believe you – show them Customer Quotes

When my mentor asked me for specific quotes from “potential customers” to demonstrate demand in an investor presentation the next day, I was up S*&% creek.

We don’t want investors “guessing” whether or not there’s demand for our products, but we don’t have time to walk through every single experiment and interview we’ve done during a pitch.  What we can do though, is share the excitement, demand, and desperation for our product via real customer quotes:

“This is Super needed in the marketplace. We will pay you for this” – Impact Investor*

“Our members will position themselves for better jobs if they have skills-based volunteering experience” – Career site for impact professionals*

“Existing programs are way too expensive, and my frustration in trying to find a legitimate opportunity could cause me to do nothing.” - Professional interested in skills-based travel volunteering*

“We can’t find the volunteers we need in order to grow our social enterprise.” - Potential hosting enterprise in Turkey*

*Names excluded for this blog post. Permission was specific to being used in presentation decks.
Make sure to respect your customers (and potential ones).

Unfortunately, through all my early customer interviews I had failed to collect quality quotes (until now).  I know I heard some great quotes, and probably scratched some down on paper, but I did a terrible job at documenting them, and I never asked for permission to use them.

I needed new customers to interview, FAST, and I needed enough quotes from them to ensure that I had a pool of HIGH QUALITY quotes. Sure, I could have gone back to people I interviewed, but I decided to use this as an opportunity to reach more people and do more validation.

And as my new venture, MovingWorlds, is working to effectively scale international, skills-based volunteering, I needed quotes about professionals who, in their own words, expressed needs that our solution addresses.

Using Quora to Find Potential Customers and Validate Our Problem and Solution Hypotheses

I was blown away how easy it was to find people with the EXACT challenges we were trying to solve. I used Quora in 2 ways

  1. Find people with relevant questions, and react to their posts
  2. Asked for specific quotes about my product

Finding people was easy. I started typing in “Skills-based volunteering” and before I could finish Quora was suggesting a myriad of topics that were addressing both parts of my marketplace – people looking for opportunities, and enterprises looking for volunteers (as well as companies looking to promote skills-based volunteering:

I was hoping for 2 things from finding people on Quora: first, to get feedback directly in Quora, and also to direct some traffic to a survey.

After I looked through Quora for a few minutes to better understand that types of questions and answers people were giving about this skills-based volunteering topic, I went to SurveyMonkey and setup a survey to capture the data I needed. I was then able to leave comments on Quora, and in exchange, ask for people to take few moments to give me feedback on Quora, and in my survey.

NOTE: Be respectful about searching for feedback here. People come to Quora for answers, not for spammy trawlers.

Here is an example of how I responded to a question so that I was still offering valuable feedback, but also asking for the opportunity to get a survey response, and maybe even a phone interview:

In addition to survey responses and comments on Quora, I also received personal messages as follow-up.

In a  relatively short period of time I was able get useful quotes that I needed to use as anecdotes and proof points in my pitch deck, find new customers to interview, better understand customer needs, and find matches for the next phase: concierge MVP (I’ll write more on concierge testing, later).

Using Mechanical Turk to Get Customer Validation Quotes

I didn’t have the time or resources to setup extensive customer interviews through Mechanical Turk like Justin wrote about on a previous post, so instead, I setup a quick survey to do the following:

  • Only capture professionals willing to volunteer their time
  • Make sure respondents were from our target demographic (from the US, had a professional skill, and attained a college degree)
  • Get quotes about challenges and opportunities for finding skills-based volunteering opportunities abroad

Step 1: Setup survey

I used SurveyMonkey and added qualifier questions to ensure respondents weren’t faking it. Surveyors got bounced with a “reject” code  if they didn’t have the right education level or live in the right place. “Trick” questions like ‘age’ and ‘location’ and ‘date of birth’ were included to cross-check data in case people were trying to spam the system (e.g. ask for age on one page, and then ask for date of birth on another, if they don’t match, its most likely worthless data)

Step 2: Setup HITs in Mechanical Turk.

I used the following project (screen shot below) to get people for the survey. I found that adding qualifiers in the ask produced better results. In this case, I had two main qualifiers:

  1. Must be a resident of the U.S.A. and have at least a Bachelor’s degree. I then confirmed these facts in the survey with specific questions (e.g. where do you live and what is your highest level of education achieved)
  2. Adding a line “Important: Be sure to read each question carefully. Randomly answering questions will be detected and will result in rejection of the HIT” also helped improve results

You can view the survey by following this link if you want to see the exact questions. Here is a screen shot of the project request on Mechanical Turk:

Here is the code for Mechanical Turk  project request that created the field above:

<h3>Answer a &lt;5 min survey about international travel and volunteering</h3>
<div class=”highlight-box”>

We are conducting a survey to better understand international travel and volunteering habits of people living in the US. We need to understand your opinion about professionals who want to travel and/or donate their expertise to make the world a better place. Select the link below to complete the survey.&nbsp; At the end of the survey, you will receive a code to paste into the box below to receive credit for taking our survey.
<p>In order to qualify for this survey, you <i><b>must </b></i>be from and/or living in the United States, <i><b>and </b></i>have achieved at least a Bachelor’s degree.</p>
</div>
<p>Survey link: <a href=”https://www.surveymonkey.com/s/VolunteeringAbroad&#8221; target=”_blank”>https://www.surveymonkey.com/s/VolunteeringAbroad</a></p&gt;

IMPORTANT: Be sure to read each question carefully. Randomly answering quesitons will be detected and will result in rejection of the HIT.
<p>Provide the survey code here:</p>
<p><input type=”text” size=”10″ id=”Q2age” name=”Q2age” /></p>
<p><style type=”text/css”>
<!–
.highlight-box { border:solid 0px #98BE10; background:#FCF9CE; color:#222222; padding:4px; text-align:left; font-size: smaller;}
–>
</style></p>
<p>&nbsp;Watch the video on our home page to learn more:&nbsp;<a href=”http://www.movingworlds.org&#8221; target=”_blank”>http://www.movingworlds.org</a></p&gt;

In the survey, in addition to asking for people to provide a quote if they were interested in skills-based volunteering, I also asked for email addresses if I could follow-up to schedule an interview. I got nearly a 50% response rate – within 1 day – from qualified leads to schedule an interview:

I have not yet had the time to follow-up with all my survey responders, but I have gotten some useful feedback and a great list of qualified people to interview and get quotes from.

Extra Credit Tip:

Don’t make the same mistake I did and not have powerful quotes and anecdotes ready to go when potential partners are asking for them :)

For all of you early on in the interview process, make sure to collect quality quotes (and get permission to share them) from the people you interview. To help organize this, my team and I setup a Google Form that made it easy to capture quotes. We could either submit ourselves, or invite others to leave us a quote through a Google Form. Key fields: Quote, Date, First Name, Last Name, Permission to Publish (Yes, No, maybe – get confirmation for each use), Email, OK to follow-up for more quotes (Yes, No). Looks like this:

Join the experiment – want more ideas on validating your business model and sharing that validation with others? Subscribe via Email or RSS for our next update: How to Interview Complete Strangers.

Prioritizing customer segments with Excel

We all know, “everyone” isn’t a customer segment.  But what do you we do when we’ve got a bunch of potential customer segments?

It’s an issue we all face at one point or another, in fact, as we previously noted, Bounce has a pile of potential customers:

For Real Estate agents, sales people, event planners, founders, general contractors, etc…
We could offer appointment reminders plus Salesforce integration!
For florists, caterers, utility installers/repairers, plumbers, electricians, etc…
We could provide on-time metrics for each employee! 
For long distance commuters, van pools, people who commute over a bridge, etc…
We could make an alarm that lets you sleep in or wakes you up early based on how bad traffic is!
For single people who are dating, busy parents, the “chronically late”….
We could gamify punctuality so they’re late less often!
For fusiness travelers, people who have just moved to an area.
We could offer additional information about sporting events, concerts, etc. that could be causing worse than normal traffic congestion.

If we target all of these customers, not only will it take forever to build this product, it’ll be a complete mess. So which group of customers are our ideal targets?

We could interview 20 of each group, but I think there’s a more efficient way.

Gut Check Prioritization

Lowest priority customers

Lowest priority customers

Let’s just gut check prioritize the segments we know about:

  1. First, add each customer group as a column in Excel
  2. Assign each group a value of 1-3 (1 = low, 3 = high) for the following criteria:
    1. Market size – how “many” of this type of customer exist
    2. Accessibility – how easy is it to find, contact and sell customers in this segment
    3. Pay for value – how much we think they’ll pay for this value
  3. Multiply the scores together and sort them from highest score to lowest
  4. Start testing hypotheses with the highest scoring segment, and work your way down until you’ve got something people are dying to pay you for
Highest priority customer groups

Highest priority customer groups

Why I love this process

  1. It’s fast. Takes < 10 minutes.
  2. It’s okay that we don’t know the real values, we’re going to test our guesses anyway. This is just a way to prioritize our customers so we don’t get stuck in Lean Startup Paralysis, or start building a product that does everything for everyone.
  3. We don’t need to interview everybody. If our first couple rounds of  testing reveal a lot of interest from our top “gut check” customer segments, but in reality “VP of Sales” were the ones most willing to pay for a product, I hypothesize they’ll make themselves known once the product is launched.  No need to stress out testing all of our potential revenue sources – find one that meets our minimum success criteria and go with it.
  4. I was wrong. Before I did this process I had thought initially thought “VPs of Sales”, the “Chronically Late” and “Regular Commuters” would all be ideal customer segments.  This quick process hinted otherwise.

How have you solved this problem?

I just made this stuff up because I was feeling overwhelmed.  When you’ve got a problem & solution that is applicable to a range of customers, what have you done to figure out who to approach first?

Join the experiment – follow along via Email or RSS for updates on our next post: Help Investors Believe you – show them Customer Quotes.

Using mTurk to interview 100 customers (in 4 hours)

This has to be one of my favorite customer development tips: using Mechanical Turk to do customer interviews.

Nick Soman, Founder of LikeBright, and I discuss how he used Mechanical Turk to interview 100 customers in 4 hours, and how that got him into TechStars Seattle.

If you haven’t heard of Mechanical Turk (mTurk), here’s what you need to know:

  • it’s a website were lots of people, do small amounts of work, for small amounts of money
  • Who does work on mTurk? Although a bit stale, these stats indicate a fairly representative cross-section of the population.
  • Why would they do work for such little $? Something to do during free time. It’s a replacement for solitaire, not a replacement for a job.

Since Nick and I talked, I’ve done a couple dozen interviews this way and the results have been fantastic. Nothing like “getting out of the building” at home, at midnight, with an ice cream sandwich in hand.

Sample interview and instructions:

1. Write the interview script

To make sure I accurately test my hypothesis, and so I don’t forget any questions, I always write up my interview script ahead of time.  Coming up with good interview questions is another post on its own, but here’s the intro I use:

Hi there, my name is Justin.  Can I get your first name please? *

Great, thanks ______.  Like I said, my name is Justin and I’m here in Seattle doing a little research.  I’m happy to tell you about the project I’m working on, but so that I don’t accidently bias any of your answers, I’ll wait until we finish the survey.  Is that alright?

Great.  So that I don’t have to slow us down to take notes while we’re chatting, is it okay if I record this call?  Great, here we go.

*Note: about half the time, people hang up after my first line. My assumption is that the hangups were expecting an automated phone survey, as opposed to a personal interview.

2. Get a (new?) Google Voice number

You’re going to post this number on mTurk, and strangers are going to call it so if you’re not down with that, create a new Google Voice account and grab a new number.  I’ve been using my actual Google Voice number and haven’t had any negative affects so far, but do what makes sense for you.

Outside the US/Canada: Google Voice is restricted to US/Canada so if you’re not there, here’s a great blog post on how to borrow a Google Voice number. (Note: may be against Google’s TOS. Use your judgement.)

3. (Optional) Forward Incoming Google Voice calls to your Gmail

I like answering my interview calls via GTalk within Gmail for a couple reasons:

  1. The gmail interface makes it easy to record the call
  2. It’s easier for me to talk hands-free with my computer

Instructions on forwarding Google Voice calls to your gmail are here: http://support.google.com/chat/bin/answer.py?hl=en&answer=187936.

4. Create a HIT on mTurk

Create a new HIT on http://requester.mturk.com

Select "other" template

Select “other” template

Describe the HIT

Describe the HIT

Describe the HIT

Pick a Price

Pick a price

Pick a price

Write up the HIT

Write up the HIT

Write up the HIT

Feel free to copy and paste this HTML:

<p><span style=”font-family: Arial;”>If you are a parent who picks your kids at day care at least once/week, please call us for a 5-10 minute phone survey.</span></p>
<p><span style=”font-family: Arial;”>Please dial the following number:</span></p>
<ul>
<li><span style=”font-family: Arial;”>*67&nbsp; [your google voice number]</span></li>
</ul>
<p><span style=”font-family: Arial;”>Note: dialing *67 before the actual phone number will protect the privacy of your phone number. &nbsp;</span><span style=”font-family: Arial;”>If you reach voicemail again, please wait 10 minutes.</span></p>
<p><span style=”font-family: Arial;”>&nbsp;<b>Required after Calling</b>&nbsp;- after we finish the survey, we will give you a password to confirm you successfully completed it. Please enter it below:</span></p>
<p><span style=”font-family: Arial;”><b>Password:</b>&nbsp;<textarea rows=”1″ cols=”80″ name=”answer”></textarea></span></p>
<p><span style=”font-family: Arial;”>Thank you very much! &nbsp;We really appreciate your help! &nbsp;</span></p>

*Note: the “password” is a word you tell your interviewee to type in once the interview is complete.  You’ll see what they type in before you approve the HIT (i.e. pay them) so you can ensure only the people who successfully completed the interview get paid.

Outside the US – mTurk needs a US address to sign up.  I’ve heard of people using any US address to create their accounts if they’re outside the US. Again, may be against TOS. Use your judgement.

5. Publish the HITs

Create a new batch

HIT it!

6. Get an ice cream sandwich

Now it’s time to wait for your first call. When it comes, answer it with your intro and once you have permission, record the call so you can share it with your team or review it later.

Incoming call

Incoming call

Record call

Record call

When you’re done, you can review the calls on Google Voice

Review recording

Review recording

Video Walkthrough

Using mTurk for the first time can be a bit confusing, so here’s a video walkthrough of the mTurk steps above:

Conclusion

If your target customers are on mTurk, which is likely if you’re building a B2C company, this is an awesome way to get lots of interviews in a short amount of time.

Thoughts, questions, or other suggestions for quick customer interviews? We’d love to hear ‘em.

Big thanks again to Nick Soman from LikeBright for sharing this tip, Andy Sack for making him do it, and Dan Shapiro for teaching all of us the power of mTurk for customer development.

Join the experiment – follow along via Email or RSS for updates on our next post: Prioritizing Customer Segments with Excel.

Threatening Puppies (kinda) Works

The results from our email interview experiment are in and, thank goodness, our customers saved the puppies.

* I didn’t get a chance to send follow-up emails for either of these emails.

Note these results include two new emails that weren’t detailed in the original blog post.  The “French Vanilla” email was the result of a friend telling me I had done everything wrong and that he could do a much better job:

French Vanilla

The Why? Email was inspired by my friend Adam Loving at Linksy who sent out an email to his customers asking for 2-3 words about their online marketing challenges:

Why?

Interesting stuff:

  1. Follow-up emails produced the most interviews.  Each person who didn’t reply to my first email got another email a couple days later repeating my request for their time.  Doing that more than doubled the number of interviews I got to do. Highly recommend FollowUpThen.
  2. Overall, the interview conversion rate was low.  7% on average, with the “winning” email converting at 12%.
  3. Sample size was probably too small to claim anything scientific, but “Rocky Road” generated “3x” the interviews “Vanilla” did.
  4. The “Why?” email performed well, but didn’t really let me dive as deep as I would have liked into my customer’s pain.
  5. Our readers are smarter than I am.  I thought “Chocolate” would win, but ya’ll were right:

”Results

Looks like this experiment provides two takeaways:

  1. Be sure to follow-up on requests for interviews.
  2. Being humorous/unique/edgy helps. Not a lot, but it helps.

Join the experiment – follow along via Email or RSS for updates on our next post: Customer Interviews using Mechanical Turk.

Not Going to Lean Startup Machine is a Waste of Time

Lean startup machine logoI went to Lean Startup Machine Seattle and it was, without a doubt, worth it. When LSM comes to your city, I highly recommend going.

There are a couple reasons I typically don’t participate in workshops/conferences:

  1. I don’t trust that the people talking know what they’re talking about
  2. I can learn what they’re teaching somewhere else (e.g. book, blog, etc.)
  3. I’m cheap and I don’t think it’ll be worth the money

Here’s how LSM mitigates those risks.

For Founders, by Founders

The guys running LSM are founders. They know what it’s like to start a company, and they know what it’s like to see it fail.

They’ve used those failures to figure out why they failed. They’ve consumed Blank, Reis, and Maurya’s ideas, distilled them, and produced a process that teaches the most fundamental principle of lean – how to identify our riskiest assumption and test it with an experiment.

This isn’t junk, it’s not bs, it’s not some guy talking to hear his own voice.  It’s stuff I think every founder should learn, provided by other founders.

LSM is an Experience – you can’t read it

The goal of the weekend is to change the way we think.  Instead of being execution focused, lean startups need to be testing & learning focused – but after years of hearing “first to market advantage”, and “those that ship win”, it takes more than reading books and blogs to rewire our brains.  We have to learn by doing.

LSM is learn by doing.

We probably spent less than an hour being “taught” how to walk through the LSM Canvas (different than the business model canvas).  The rest of the time, we were identifying our assumptions, prioritizing them, and testing them…over, and over, and over again, until our brains started to rewire themselves.

You can’t “read” an experience, you have to do it.

Not Going is a Waste of Time

If we agree that doing is a more effective way to learn than simply reading, we’ve got two choices re “doing:”

  1. Applying the lean startup techniques we read about to our own startups and over the course of months/years, we’ll have this stuff down cold
  2. Learn it in a weekend

It’s really that simple.  LSM is specifically designed to teach the essence of Lean in 54 hours.  Our startups on the other hand are designed to be businesses, not teaching tools.  Using them as such is possible, it’s just not an efficient use of time.

In terms of the amount of time and energy the weekend saved me, I’d say I’d pay $500 for it.  It doesn’t cost that much, but they could charge it and I’d still recommend it to people.

Conclusion

If you’re reading this, I highly recommend you go to an LSM.

Justin

PS – Suggestions for the Future

While the weekend was absolutely worth it, there were a couple things that could be improved:

  1. The “mentors” didn’t know lean.  While a couple did, there were certainly a few people walking around and giving advice that really didn’t know process we were trying to learn.  It would be great to require the mentors take a crash course in the techniques beforehand.

  2. Cut the speakers.  There were a handful of speeches throughout the weekend, but I wasn’t really sure why.  They killed our momentum and they didn’t have anything to do with Lean – they were just startup oriented talks.  Imo, we can get that stuff elsewhere, make the speeches lean-oriented or cut them.

  3. The “pitch” competition at the end wasn’t a good use of time.  Not only did it emphasize the wrong thing (results, as opposed to learning), the judges didn’t have a clue what it really meant to be Lean.

    I’d suggest doing something like a “Lean Tournament” as a finale instead.  Two teams at a time are pitted against one another and given a customer hypothesis, a problem hypothesis, and a solution hypothesis.  The two teams have 1 minute to identify the riskiest assumption and design a test for it.  Lean-educated judges pick the winner and it continues bracket-style until there’s a winner.  A high energy way to end the weekend that emphasizes learning, not execution.

Btw, don’t let these suggestions dissuade you, they comprise maybe 5% of the weekend.  I’m including them for completeness sake.

Join the Experiment – Want to save more time learning lean? Follow along with us via Email or RSS.