This has to be one of my favorite customer development tips: using Mechanical Turk to do customer interviews.

Nick Soman, Founder of LikeBright, and I discuss how he used Mechanical Turk to interview 100 customers in 4 hours, and how that got him into TechStars Seattle.

If you haven’t heard of Mechanical Turk (mTurk), here’s what you need to know:

• it’s a website were lots of people, do small amounts of work, for small amounts of money
• Who does work on mTurk? Although a bit stale, these stats indicate a fairly representative cross-section of the population.
• Why would they do work for such little $? Something to do during free time. It’s a replacement for solitaire, not a replacement for a job.

Since Nick and I talked, I’ve done a couple dozen interviews this way and the results have been fantastic. Nothing like “getting out of the building” at home, at midnight, with an ice cream sandwich in hand.

[Update – 5/29/2015]

There’s an easier and faster way to do everything listed here. Our friends at Customer Discovery Ninja have automated this entire workflow below and made it easy for folks outside the US to access Turkers.

Feel free to familiarize yourself with the steps below if you like, but the fastest way to talk to customers is going to be Customer Discovery Ninja.

Sample interview and instructions:

1. Write the interview script

To make sure I accurately test my hypothesis, and so I don’t forget any questions, I always write up my interview script ahead of time.  Coming up with good interview questions is another post on its own, but here’s the intro I use:

Hi there, my name is Justin.  Can I get your first name please? *

Great, thanks ______.  Like I said, my name is Justin and I’m here in Seattle doing a little research.  I’m happy to tell you about the project I’m working on, but so that I don’t accidently bias any of your answers, I’ll wait until we finish the survey.  Is that alright?

Great.  So that I don’t have to slow us down to take notes while we’re chatting, is it okay if I record this call?  Great, here we go.

*Note: about half the time, people hang up after my first line. My assumption is that the hangups were expecting an automated phone survey, as opposed to a personal interview.

2. Get a (new?) Google Voice number

You’re going to post this number on mTurk, and strangers are going to call it so if you’re not down with that, create a new Google Voice account and grab a new number.  I’ve been using my actual Google Voice number and haven’t had any negative affects so far, but do what makes sense for you.

Outside the US/Canada: Google Voice is restricted to US/Canada so if you’re not there, here’s a great blog post on how to borrow a Google Voice number. (Note: may be against Google’s TOS. Use your judgement.)

3. (Optional) Forward Incoming Google Voice calls to your Gmail

I like answering my interview calls via GTalk within Gmail for a couple reasons:

1. The gmail interface makes it easy to record the call
2. It’s easier for me to talk hands-free with my computer

Instructions on forwarding Google Voice calls to your gmail are here: http://support.google.com/chat/bin/answer.py?hl=en&answer=187936.

4. Create a HIT on mTurk

Create a new HIT on http://requester.mturk.com

Describe the HIT

Super Important:

You must turn off “Master Turkers.” Master Turkers are a pre-screened, and very small, subset of the MTurk population. We want any folks on MTurk to be able to contact us, as long as they meet our qualifications. Here’s how to do that:

Pick a Price

Write up the HIT

Feel free to copy and paste this HTML:

<p><span style=”font-family: Arial;”>If you are a parent who picks your kids at day care at least once/week, please call us for a 5-10 minute phone survey.</span></p>
<p><span style=”font-family: Arial;”>Please dial the following number:</span></p>
<ul>
<li><span style=”font-family: Arial;”>*67&nbsp; [your google voice number]</span></li>
</ul>
<p><span style=”font-family: Arial;”>Note: dialing *67 before the actual phone number will protect the privacy of your phone number. &nbsp;</span><span style=”font-family: Arial;”>If you reach voicemail again, please wait 10 minutes.</span></p>
<p><span style=”font-family: Arial;”>&nbsp;<b>Required after Calling</b>&nbsp;- after we finish the survey, we will give you a password to confirm you successfully completed it. Please enter it below:</span></p>
<p><span style=”font-family: Arial;”><b>Password:</b>&nbsp;<textarea rows=”1″ cols=”80″ name=”answer”></textarea></span></p>
<p><span style=”font-family: Arial;”>Thank you very much! &nbsp;We really appreciate your help! &nbsp;</span></p>

*Note: the “password” is a word you tell your interviewee to type in once the interview is complete.  You’ll see what they type in before you approve the HIT (i.e. pay them) so you can ensure only the people who successfully completed the interview get paid.

Outside the US – mTurk needs a US address to sign up.  I’ve heard of people using any US address to create their accounts if they’re outside the US. Again, may be against TOS. Use your judgement.

5. Publish the HITs

6. Get an ice cream sandwich

Now it’s time to wait for your first call. When it comes, answer it with your intro and once you have permission, record the call so you can share it with your team or review it later.

When you’re done, you can review the calls on Google Voice

Video Walkthrough

Using mTurk for the first time can be a bit confusing, so here’s a video walkthrough of the mTurk steps above:

Conclusion

If your target customers are on mTurk, which is likely if you’re building a B2C company, this is an awesome way to get lots of interviews in a short amount of time.

Thoughts, questions, or other suggestions for quick customer interviews? We’d love to hear ’em.

Big thanks again to Nick Soman from LikeBright for sharing this tip, Andy Sack for making him do it, and Dan Shapiro for teaching all of us the power of mTurk for customer development.

Outside the US? Or Don’t want to Mess with Mechanical Turk?

There’s an easier and faster way to do everything listed above. The guys at Customer Discovery Ninja have automated this entire workflow, which also makes it easy for folks outside the US to access US Turkers.

What’s Next?

Join the experiment – follow along via Email or RSS for updates on our next post: Prioritizing Customer Segments with Excel.

Customer Development Made Easy…

The results from our email interview experiment are in and, thank goodness, our customers saved the puppies.

* I didn’t get a chance to send follow-up emails for either of these emails.

Note these results include two new emails that weren’t detailed in the original blog post.  The “French Vanilla” email was the result of a friend telling me I had done everything wrong and that he could do a much better job:

French Vanilla

The Why? Email was inspired by my friend Adam Loving at Linksy who sent out an email to his customers asking for 2-3 words about their online marketing challenges:

Why?

Interesting stuff:

1. Follow-up emails produced the most interviews.  Each person who didn’t reply to my first email got another email a couple days later repeating my request for their time.  Doing that more than doubled the number of interviews I got to do. Highly recommend FollowUpThen.
2. Overall, the interview conversion rate was low.  7% on average, with the “winning” email converting at 12%.
3. Sample size was probably too small to claim anything scientific, but “Rocky Road” generated “3x” the interviews “Vanilla” did.
4. The “Why?” email performed well, but didn’t really let me dive as deep as I would have liked into my customer’s pain.
5. Our readers are smarter than I am.  I thought “Chocolate” would win, but ya’ll were right:

Looks like this experiment provides two takeaways:

1. Be sure to follow-up on requests for interviews.
2. Being humorous/unique/edgy helps. Not a lot, but it helps.

Join the experiment – follow along via Email or RSS for updates on our next post: Customer Interviews using Mechanical Turk.

Customer Development Made Easy…

I went to Lean Startup Machine Seattle and it was, without a doubt, worth it. When LSM comes to your city, I highly recommend going.

There are a couple reasons I typically don’t participate in workshops/conferences:

1. I don’t trust that the people talking know what they’re talking about
2. I can learn what they’re teaching somewhere else (e.g. book, blog, etc.)
3. I’m cheap and I don’t think it’ll be worth the money

Here’s how LSM mitigates those risks.

For Founders, by Founders

The guys running LSM are founders. They know what it’s like to start a company, and they know what it’s like to see it fail.

They’ve used those failures to figure out why they failed. They’ve consumed Blank, Reis, and Maurya’s ideas, distilled them, and produced a process that teaches the most fundamental principle of lean – how to identify our riskiest assumption and test it with an experiment.

This isn’t junk, it’s not bs, it’s not some guy talking to hear his own voice.  It’s stuff I think every founder should learn, provided by other founders.

LSM is an Experience – you can’t read it

The goal of the weekend is to change the way we think.  Instead of being execution focused, lean startups need to be testing & learning focused – but after years of hearing “first to market advantage”, and “those that ship win”, it takes more than reading books and blogs to rewire our brains.  We have to learn by doing.

LSM is learn by doing.

We probably spent less than an hour being “taught” how to walk through the LSM Canvas (different than the business model canvas).  The rest of the time, we were identifying our assumptions, prioritizing them, and testing them…over, and over, and over again, until our brains started to rewire themselves.

You can’t “read” an experience, you have to do it.

Not Going is a Waste of Time

If we agree that doing is a more effective way to learn than simply reading, we’ve got two choices re “doing:”

1. Applying the lean startup techniques we read about to our own startups and over the course of months/years, we’ll have this stuff down cold
2. Learn it in a weekend

It’s really that simple.  LSM is specifically designed to teach the essence of Lean in 54 hours.  Our startups on the other hand are designed to be businesses, not teaching tools.  Using them as such is possible, it’s just not an efficient use of time.

In terms of the amount of time and energy the weekend saved me, I’d say I’d pay $500 for it.  It doesn’t cost that much, but they could charge it and I’d still recommend it to people.

Conclusion

If you’re reading this, I highly recommend you go to an LSM.

Justin

PS – Suggestions for the Future

While the weekend was absolutely worth it, there were a couple things that could be improved:

1. The “mentors” didn’t know lean.  While a couple did, there were certainly a few people walking around and giving advice that really didn’t know process we were trying to learn.  It would be great to require the mentors take a crash course in the techniques beforehand.

2. Cut the speakers.  There were a handful of speeches throughout the weekend, but I wasn’t really sure why.  They killed our momentum and they didn’t have anything to do with Lean – they were just startup oriented talks.  Imo, we can get that stuff elsewhere, make the speeches lean-oriented or cut them.

3. The “pitch” competition at the end wasn’t a good use of time.  Not only did it emphasize the wrong thing (results, as opposed to learning), the judges didn’t have a clue what it really meant to be Lean.

   I’d suggest doing something like a “Lean Tournament” as a finale instead.  Two teams at a time are pitted against one another and given a customer hypothesis, a problem hypothesis, and a solution hypothesis.  The two teams have 1 minute to identify the riskiest assumption and design a test for it.  Lean-educated judges pick the winner and it continues bracket-style until there’s a winner.  A high energy way to end the weekend that emphasizes learning, not execution.

Btw, don’t let these suggestions dissuade you, they comprise maybe 5% of the weekend.  I’m including them for completeness sake.

Join the Experiment – Want to save more time learning lean? Follow along with us via Email or RSS.

Lean Startup Machine is like a Startup Weekend, except exclusively focused on business model validations as opposed to product building.  It’s coming to Seattle next weekend and I’ve decided to go.

Here’s why I’m going (and why you might think about doing the same):

My name is Justin, and I’m Addicted to Writing Code

I love building things. It’s why I became a developer. It’s a rush, it’s a high, it’s nerd cocaine, and I’m addicted.

Combine that with years of mantras like, “Those that ship, win!” and “code, code, code” and as developers, we’re taught to code first, and worry about who will buy this thing, never.  “Developers code, let the business guys figure out how to sell it.”

Of course, now that I do my own startups, I think I’m that business guy, but subconsciously my inner addict is likely running more of the show than I’d care to admit. To be truly lean though, and to increase the likelihood of my startup’s success, means I need to focus not on coding, but on learning:

Deadly Startup Sin #4: Emphasizing execution instead of testing, learning and iteration – Steve Blank, 9 Deadliest Startup Sins

The fastest way to lessen the impact of my addiction, is to immerse myself in learning Lean processes.  Lean Startup Machine feels like rehab for those of us who are addicted to coding.

The First Time, is the Worst Time

Whether it’s the first pancake off the griddle, the first version of a product, or the first time following a process, my first time is always my worst time.

With that in mind, why should I subject Bounce, a business I’d really like to succeed, to my first time practicing Lean Startup processes end-to-end?  At Lean Startup Machine, I can practice Lean techniques on other people’s business models, and take what I learn and apply all the good stuff to my own.

I got a Discount

I’m cheap. Truth is, I feel genetically predisposed to not spending money, and while it’s handy to be a frugal founder, I worry sometimes that I miss out on opportunities like Lean Startup Machine because they cost $.

Luckily, I didn’t have to pay the full $299 price…and if you’d like to go, you won’t have to either.  You’ve got two choices re discounts:

1. 30% off courtesy of John Sechrest – http://bit.ly/LH62we
2. Lean Startup Machine + Startup Weekend Bundle – http://bit.ly/LH6b2s

While financially, they’re essentially the same deal, that second one is particularly interesting if you’re wondering about how Lean Startup Machine compares to Startup Weekend – the Lean Startup Machine team will pay for your next Seattle Startup Weekend.

That means you can validate your business model at Lean Startup Machine, and then build it at a Startup Weekend.

Conclusion

I’ll be at the Lean Startup Machine next weekend, breaking my coding addiction by learning Lean processes on someone else’s business model.  You should come too.

Join the Experiment – Can’t make to Lean Startup Machine but want to hear how it went? Get updates via Email or RSS.

[8/14 Update: results from this experiment are in]

Last year I was hanging out on OkCupid and realized online dating is another startup, replete with landing pages, copy optimization and “filling the funnel.” The only difference is that we’re selling ourselves instead of software.

Hoping to up my conversion rate, I wanted to figure out what style of messages OkCupid members respond to the most, so I did a little a/b testing with four types of messages:

1. “Hi” – Fairly lame, but simple and says, “look at my profile”
2. Generic, short message – Same as above, but not quite as lame as the word “hi.” Something like (literally), “look at my profile.”
3. Personalized positive message – Prove that I’ve read her profile, “Canceling Arrested Development was a travesty, but I take solace in watching It’s Always Sunny while huffing paint.”
4. Personalized Negative Message  – Nice guys finish last…“You’re wearing the same shirt in 3 of your photos.  Don’t get out much?”

Care to guess which performed best?

From one of the respondents:

It’s interesting that I replied to this right away even though you basically insulted me twice.

So how can we apply this to  customer development?  What if we tried the same experimentation when contacting our customers?

Requesting Customer Interviews

First off, I won’t be (intentionally) insulting any of my customers.  That said, I have the email addresses of a couple thousand people who have requested the iPhone version of Bounce, so we’ve got an ample size to run some a/b tests.

For this experiment, I’m looking to do some customer interviews to answer questions like:

• Why did they give me their email address?
• What do they expect Bounce to do?
• How much would they pay for it, if it did what they wanted?

Now let’s see which of the following emails results in the most interviews:

Vanilla

Simple (and boring) but to the point

Chocolate

Trying to add a bit more flavor

Rocky Road

Like the Chocolate version, but a little extra puppy killing

What’s your Hypothesis?

I sent 75 emails out this morning, time to place your bets.  Which one do you think will perform best?

Will the puppies live?

Join the experiment – get notified when the results are in via Email or RSS.

Customer Development Made Easy…