Attn Customers: answer my questions or I’ll kill these puppies

Awww…
[8/14 Update: results from this experiment are in]

Last year I was hanging out on OkCupid and realized online dating is another startup, replete with landing pages, copy optimization and “filling the funnel.” The only difference is that we’re selling ourselves instead of software.

Hoping to up my conversion rate, I wanted to figure out what style of messages OkCupid members respond to the most, so I did a little a/b testing with four types of messages:

  1. “Hi” – Fairly lame, but simple and says, “look at my profile”
  2. Generic, short message – Same as above, but not quite as lame as the word “hi.” Something like (literally), “look at my profile.”
  3. Personalized positive message – Prove that I’ve read her profile, “Canceling Arrested Development was a travesty, but I take solace in watching It’s Always Sunny while huffing paint.”
  4. Personalized Negative Message  – Nice guys finish last…“You’re wearing the same shirt in 3 of your photos.  Don’t get out much?”

Care to guess which performed best?

From one of the respondents:

It’s interesting that I replied to this right away even though you basically insulted me twice.

So how can we apply this to  customer development?  What if we tried the same experimentation when contacting our customers?

Requesting Customer Interviews

First off, I won’t be (intentionally) insulting any of my customers.  That said, I have the email addresses of a couple thousand people who have requested the iPhone version of Bounce, so we’ve got an ample size to run some a/b tests.

For this experiment, I’m looking to do some customer interviews to answer questions like:

  • Why did they give me their email address?
  • What do they expect Bounce to do?
  • How much would they pay for it, if it did what they wanted?

Now let’s see which of the following emails results in the most interviews:

Vanilla

Simple (and boring) but to the point

I’m interviewing a few folks to understand what features they want most, and why.  Do you have 12-15 minutes for a phone/skype call?  I’m free any of these times (Pacific time):

Chocolate

Trying to add a bit more flavor

Can I borrow your car?<br/><br/>Or, can I get 15 minutes to chat about Bounce?  Your choice, whatever’s easy.<br/><br/>Here’s the deal, I’m a programmer. If I build Bounce <i>for me</i>, it’ll not only be ugly, it’ll be complicated and probably won’t do what you need it to. If however I build it <i>for you</i>, there’s a good chance it’ll get both of us where we need to be on time.<br/><br/>You free for a 12-15 minute phone/skype call at any of these times (Pacific time)?

Rocky Road

Like the Chocolate version, but a little extra puppy killing

Can I borrow your car? &nbsp;Or, can I get 15 minutes to chat about Bounce? &nbsp;Your choice, whatever’s easy.<br/><br/>Here’s the deal, if I build Bounce <i>for me</i>, it’ll not only be ugly as sin, it probably won’t do what you need it to. &nbsp;That means you’ll give it an awful review, which means no one else will buy it, which means I'll have to learn how to cook methamphetamines in my bathtub for rent money, which means I’ll blow up my entire apartment building, and along with it, the adorable litter of husky puppies that’s living upstairs.<br/><br/>I don’t want to kill puppies. &nbsp;You don’t want to kill puppies.<br/><br/>How about we both save ourselves some trouble and make sure Bounce works exactly like you need it to?  Got 12-15 minutes to chat at any of these times (Pacific time)?

What’s your Hypothesis?

I sent 75 emails out this morning, time to place your bets.  Which one do you think will perform best?

Will the puppies live?

Join the experiment – get notified when the results are in via Email or RSS.

Customer Development Made Easy…

9 comments
  1. Brilliant! This follows many principals of great Word of Mouth marketing and copywriting that gets responses (e.g. humor, speaking with your own voice, being out of the ordinary, etc.). You’ve inspired me to change the emails I write to potential customers. I might wait to see the result first though… ;)

    Are you using metrics to measure responses, etc.? If so, I’d be interested in hearing what tools you’re using. Something you might consider adding in future emails is something along the lines of “if you know someone else who shares your love of puppies and can spend a few minutes with me, please forward”.

    1. Thanks Hakon! I considered using bit.ly links to measure the number of links clicked, as an indication of interest, but to your earlier point about only having a single metric, I decided to only track one thing – # of interviews.

      Also good call on the sharing. Maybe I’ll incorporate that next time.

  2. Great article and a lot more fun to read than the standard email marketing tips. Have you thought about playing with different subject lines though? Most emails never get opened and one of the biggest determining factors is the subject line itself..

    It’ll be interesting to see which approach gets the most opens. And for the record, I voted Chocolate..

    1. Thanks Emil! I thought about tweaking the subject lines but because I was playing with the content so much, I wanted to leave as much of the rest of the email the same. Hopefully the mails will get the same open rates since they were sent at the same time and have the same subject line, and we’ll be able to tell if the content itself drives behavior.

  3. I’m really curious to not only see quantity, but quality. Harder to measure, but very interesting.

    1. So am I!
      But the metric we are waiting for is a bit more qualitative (e.g. they are unlikely to complain about the mail, and at least give some feedback) than response messages on a dating portal (which are less likely to (directly or indirectly) lead to the desired outcome, e.g. a date)

      1. Well put; fun experiment, and I wonder how many of the respondents fit your early adopters demographics (hypothesis).

        In a way, I feel that Redditors will flock to an interview, but I wonder if those are your customers?

  4. Great experiment and thanks for sharing it. Good to get some help on getting emails right!

    The real, ultimate goal is to make Bounce better so more people use it. It will be interesting to see if after implementing changes after the interviews it improves usage and take up.

  5. Very interesting idea. I’m curious what the results will be. I’d guess “chocolate” would prevail because it’s hella funny but not too long.

Leave a Reply