Last year I was hanging out on OkCupid and realized online dating is another startup, replete with landing pages, copy optimization and “filling the funnel.” The only difference is that we’re selling ourselves instead of software.
Hoping to up my conversion rate, I wanted to figure out what style of messages OkCupid members respond to the most, so I did a little a/b testing with four types of messages:
- “Hi” – Fairly lame, but simple and says, “look at my profile”
- Generic, short message – Same as above, but not quite as lame as the word “hi.” Something like (literally), “look at my profile.”
- Personalized positive message – Prove that I’ve read her profile, “Canceling Arrested Development was a travesty, but I take solace in watching It’s Always Sunny while huffing paint.”
- Personalized Negative Message – Nice guys finish last…“You’re wearing the same shirt in 3 of your photos. Don’t get out much?”
Care to guess which performed best?
From one of the respondents:
It’s interesting that I replied to this right away even though you basically insulted me twice.
So how can we apply this to customer development? What if we tried the same experimentation when contacting our customers?
Requesting Customer Interviews
First off, I won’t be (intentionally) insulting any of my customers. That said, I have the email addresses of a couple thousand people who have requested the iPhone version of Bounce, so we’ve got an ample size to run some a/b tests.
For this experiment, I’m looking to do some customer interviews to answer questions like:
- Why did they give me their email address?
- What do they expect Bounce to do?
- How much would they pay for it, if it did what they wanted?
Now let’s see which of the following emails results in the most interviews:
Vanilla
Simple (and boring) but to the point
Chocolate
Trying to add a bit more flavor
Rocky Road
Like the Chocolate version, but a little extra puppy killing
What’s your Hypothesis?
I sent 75 emails out this morning, time to place your bets. Which one do you think will perform best?
Will the puppies live?
Join the experiment – get notified when the results are in via Email or RSS.
Very interesting idea. I’m curious what the results will be. I’d guess “chocolate” would prevail because it’s hella funny but not too long.
Great experiment and thanks for sharing it. Good to get some help on getting emails right!
The real, ultimate goal is to make Bounce better so more people use it. It will be interesting to see if after implementing changes after the interviews it improves usage and take up.
I’m really curious to not only see quantity, but quality. Harder to measure, but very interesting.
So am I!
But the metric we are waiting for is a bit more qualitative (e.g. they are unlikely to complain about the mail, and at least give some feedback) than response messages on a dating portal (which are less likely to (directly or indirectly) lead to the desired outcome, e.g. a date)
Well put; fun experiment, and I wonder how many of the respondents fit your early adopters demographics (hypothesis).
In a way, I feel that Redditors will flock to an interview, but I wonder if those are your customers?
Great article and a lot more fun to read than the standard email marketing tips. Have you thought about playing with different subject lines though? Most emails never get opened and one of the biggest determining factors is the subject line itself..
It’ll be interesting to see which approach gets the most opens. And for the record, I voted Chocolate..
Thanks Emil! I thought about tweaking the subject lines but because I was playing with the content so much, I wanted to leave as much of the rest of the email the same. Hopefully the mails will get the same open rates since they were sent at the same time and have the same subject line, and we’ll be able to tell if the content itself drives behavior.
Brilliant! This follows many principals of great Word of Mouth marketing and copywriting that gets responses (e.g. humor, speaking with your own voice, being out of the ordinary, etc.). You’ve inspired me to change the emails I write to potential customers. I might wait to see the result first though… ;)
Are you using metrics to measure responses, etc.? If so, I’d be interested in hearing what tools you’re using. Something you might consider adding in future emails is something along the lines of “if you know someone else who shares your love of puppies and can spend a few minutes with me, please forward”.
Thanks Hakon! I considered using bit.ly links to measure the number of links clicked, as an indication of interest, but to your earlier point about only having a single metric, I decided to only track one thing – # of interviews.
Also good call on the sharing. Maybe I’ll incorporate that next time.