This has to be one of my favorite customer development tips: using Mechanical Turk to do customer interviews.

Nick Soman, Founder of LikeBright, and I discuss how he used Mechanical Turk to interview 100 customers in 4 hours, and how that got him into TechStars Seattle.

If you haven’t heard of Mechanical Turk (mTurk), here’s what you need to know:

• it’s a website were lots of people, do small amounts of work, for small amounts of money
• Who does work on mTurk? Although a bit stale, these stats indicate a fairly representative cross-section of the population.
• Why would they do work for such little $? Something to do during free time. It’s a replacement for solitaire, not a replacement for a job.

Since Nick and I talked, I’ve done a couple dozen interviews this way and the results have been fantastic. Nothing like “getting out of the building” at home, at midnight, with an ice cream sandwich in hand.

[Update – 5/29/2015]

There’s an easier and faster way to do everything listed here. Our friends at Customer Discovery Ninja have automated this entire workflow below and made it easy for folks outside the US to access Turkers.

Feel free to familiarize yourself with the steps below if you like, but the fastest way to talk to customers is going to be Customer Discovery Ninja.

Sample interview and instructions:

1. Write the interview script

To make sure I accurately test my hypothesis, and so I don’t forget any questions, I always write up my interview script ahead of time.  Coming up with good interview questions is another post on its own, but here’s the intro I use:

Hi there, my name is Justin.  Can I get your first name please? *

Great, thanks ______.  Like I said, my name is Justin and I’m here in Seattle doing a little research.  I’m happy to tell you about the project I’m working on, but so that I don’t accidently bias any of your answers, I’ll wait until we finish the survey.  Is that alright?

Great.  So that I don’t have to slow us down to take notes while we’re chatting, is it okay if I record this call?  Great, here we go.

*Note: about half the time, people hang up after my first line. My assumption is that the hangups were expecting an automated phone survey, as opposed to a personal interview.

2. Get a (new?) Google Voice number

You’re going to post this number on mTurk, and strangers are going to call it so if you’re not down with that, create a new Google Voice account and grab a new number.  I’ve been using my actual Google Voice number and haven’t had any negative affects so far, but do what makes sense for you.

Outside the US/Canada: Google Voice is restricted to US/Canada so if you’re not there, here’s a great blog post on how to borrow a Google Voice number. (Note: may be against Google’s TOS. Use your judgement.)

3. (Optional) Forward Incoming Google Voice calls to your Gmail

I like answering my interview calls via GTalk within Gmail for a couple reasons:

1. The gmail interface makes it easy to record the call
2. It’s easier for me to talk hands-free with my computer

Instructions on forwarding Google Voice calls to your gmail are here: http://support.google.com/chat/bin/answer.py?hl=en&answer=187936.

4. Create a HIT on mTurk

Create a new HIT on http://requester.mturk.com

Describe the HIT

Super Important:

You must turn off “Master Turkers.” Master Turkers are a pre-screened, and very small, subset of the MTurk population. We want any folks on MTurk to be able to contact us, as long as they meet our qualifications. Here’s how to do that:

Pick a Price

Write up the HIT

Feel free to copy and paste this HTML:

<p><span style=”font-family: Arial;”>If you are a parent who picks your kids at day care at least once/week, please call us for a 5-10 minute phone survey.</span></p>
<p><span style=”font-family: Arial;”>Please dial the following number:</span></p>
<ul>
<li><span style=”font-family: Arial;”>*67&nbsp; [your google voice number]</span></li>
</ul>
<p><span style=”font-family: Arial;”>Note: dialing *67 before the actual phone number will protect the privacy of your phone number. &nbsp;</span><span style=”font-family: Arial;”>If you reach voicemail again, please wait 10 minutes.</span></p>
<p><span style=”font-family: Arial;”>&nbsp;<b>Required after Calling</b>&nbsp;- after we finish the survey, we will give you a password to confirm you successfully completed it. Please enter it below:</span></p>
<p><span style=”font-family: Arial;”><b>Password:</b>&nbsp;<textarea rows=”1″ cols=”80″ name=”answer”></textarea></span></p>
<p><span style=”font-family: Arial;”>Thank you very much! &nbsp;We really appreciate your help! &nbsp;</span></p>

*Note: the “password” is a word you tell your interviewee to type in once the interview is complete.  You’ll see what they type in before you approve the HIT (i.e. pay them) so you can ensure only the people who successfully completed the interview get paid.

Outside the US – mTurk needs a US address to sign up.  I’ve heard of people using any US address to create their accounts if they’re outside the US. Again, may be against TOS. Use your judgement.

5. Publish the HITs

6. Get an ice cream sandwich

Now it’s time to wait for your first call. When it comes, answer it with your intro and once you have permission, record the call so you can share it with your team or review it later.

When you’re done, you can review the calls on Google Voice

Video Walkthrough

Using mTurk for the first time can be a bit confusing, so here’s a video walkthrough of the mTurk steps above:

Conclusion

If your target customers are on mTurk, which is likely if you’re building a B2C company, this is an awesome way to get lots of interviews in a short amount of time.

Thoughts, questions, or other suggestions for quick customer interviews? We’d love to hear ’em.

Big thanks again to Nick Soman from LikeBright for sharing this tip, Andy Sack for making him do it, and Dan Shapiro for teaching all of us the power of mTurk for customer development.

Outside the US? Or Don’t want to Mess with Mechanical Turk?

There’s an easier and faster way to do everything listed above. The guys at Customer Discovery Ninja have automated this entire workflow, which also makes it easy for folks outside the US to access US Turkers.

What’s Next?

Join the experiment – follow along via Email or RSS for updates on our next post: Prioritizing Customer Segments with Excel.

Customer Development Made Easy…

What to test
+
How to test =

Now offering the...

FOCUS Framework

The results from our email interview experiment are in and, thank goodness, our customers saved the puppies.

* I didn’t get a chance to send follow-up emails for either of these emails.

Note these results include two new emails that weren’t detailed in the original blog post.  The “French Vanilla” email was the result of a friend telling me I had done everything wrong and that he could do a much better job:

French Vanilla

The Why? Email was inspired by my friend Adam Loving at Linksy who sent out an email to his customers asking for 2-3 words about their online marketing challenges:

Why?

Interesting stuff:

1. Follow-up emails produced the most interviews.  Each person who didn’t reply to my first email got another email a couple days later repeating my request for their time.  Doing that more than doubled the number of interviews I got to do. Highly recommend FollowUpThen.
2. Overall, the interview conversion rate was low.  7% on average, with the “winning” email converting at 12%.
3. Sample size was probably too small to claim anything scientific, but “Rocky Road” generated “3x” the interviews “Vanilla” did.
4. The “Why?” email performed well, but didn’t really let me dive as deep as I would have liked into my customer’s pain.
5. Our readers are smarter than I am.  I thought “Chocolate” would win, but ya’ll were right:

Looks like this experiment provides two takeaways:

1. Be sure to follow-up on requests for interviews.
2. Being humorous/unique/edgy helps. Not a lot, but it helps.

Join the experiment – follow along via Email or RSS for updates on our next post: Customer Interviews using Mechanical Turk.

Customer Development Made Easy…

What to test
+
How to test =

Now offering the...

FOCUS Framework

I went to Lean Startup Machine Seattle and it was, without a doubt, worth it. When LSM comes to your city, I highly recommend going.

There are a couple reasons I typically don’t participate in workshops/conferences:

1. I don’t trust that the people talking know what they’re talking about
2. I can learn what they’re teaching somewhere else (e.g. book, blog, etc.)
3. I’m cheap and I don’t think it’ll be worth the money

Here’s how LSM mitigates those risks.

For Founders, by Founders

The guys running LSM are founders. They know what it’s like to start a company, and they know what it’s like to see it fail.

They’ve used those failures to figure out why they failed. They’ve consumed Blank, Reis, and Maurya’s ideas, distilled them, and produced a process that teaches the most fundamental principle of lean – how to identify our riskiest assumption and test it with an experiment.

This isn’t junk, it’s not bs, it’s not some guy talking to hear his own voice.  It’s stuff I think every founder should learn, provided by other founders.

LSM is an Experience – you can’t read it

The goal of the weekend is to change the way we think.  Instead of being execution focused, lean startups need to be testing & learning focused – but after years of hearing “first to market advantage”, and “those that ship win”, it takes more than reading books and blogs to rewire our brains.  We have to learn by doing.

LSM is learn by doing.

We probably spent less than an hour being “taught” how to walk through the LSM Canvas (different than the business model canvas).  The rest of the time, we were identifying our assumptions, prioritizing them, and testing them…over, and over, and over again, until our brains started to rewire themselves.

You can’t “read” an experience, you have to do it.

Not Going is a Waste of Time

If we agree that doing is a more effective way to learn than simply reading, we’ve got two choices re “doing:”

1. Applying the lean startup techniques we read about to our own startups and over the course of months/years, we’ll have this stuff down cold
2. Learn it in a weekend

It’s really that simple.  LSM is specifically designed to teach the essence of Lean in 54 hours.  Our startups on the other hand are designed to be businesses, not teaching tools.  Using them as such is possible, it’s just not an efficient use of time.

In terms of the amount of time and energy the weekend saved me, I’d say I’d pay $500 for it.  It doesn’t cost that much, but they could charge it and I’d still recommend it to people.

Conclusion

If you’re reading this, I highly recommend you go to an LSM.

Justin

PS – Suggestions for the Future

While the weekend was absolutely worth it, there were a couple things that could be improved:

1. The “mentors” didn’t know lean.  While a couple did, there were certainly a few people walking around and giving advice that really didn’t know process we were trying to learn.  It would be great to require the mentors take a crash course in the techniques beforehand.

2. Cut the speakers.  There were a handful of speeches throughout the weekend, but I wasn’t really sure why.  They killed our momentum and they didn’t have anything to do with Lean – they were just startup oriented talks.  Imo, we can get that stuff elsewhere, make the speeches lean-oriented or cut them.

3. The “pitch” competition at the end wasn’t a good use of time.  Not only did it emphasize the wrong thing (results, as opposed to learning), the judges didn’t have a clue what it really meant to be Lean.

   I’d suggest doing something like a “Lean Tournament” as a finale instead.  Two teams at a time are pitted against one another and given a customer hypothesis, a problem hypothesis, and a solution hypothesis.  The two teams have 1 minute to identify the riskiest assumption and design a test for it.  Lean-educated judges pick the winner and it continues bracket-style until there’s a winner.  A high energy way to end the weekend that emphasizes learning, not execution.

Btw, don’t let these suggestions dissuade you, they comprise maybe 5% of the weekend.  I’m including them for completeness sake.

Join the Experiment – Want to save more time learning lean? Follow along with us via Email or RSS.