Testing your Domain Name

tl;dr

Using clever survey questions to quickly test the memorability, spellability and emotional response of potential domain names. Glad we did – almost picked a lousy one.

Problem: Domain Squatters

I’ve found what should be the joyous process of naming a company, quickly devolves into:

Founder A: Are you kidding with me?! How is every good domain taken!?
Founder B: Let’s make up a word. “ooFooVu.com” is availble.
Founder A: Maybe “KillerKittenKites.co?”
Founder B: TwoGirlsOneCup…dot ly is available.

Most of the time we’re left with abysmal choices and forced to argue over which one sucks the least.

Some friends and I who were building a crowdfunding aggregator wanted to avoid all that, so we came up with a way to test our company/domain names with customer data. Here’s how…

Step 1: Crowdsource Ideas

SquadHelp is 99Designs for domain names. tweet-this-button That means you pay a couple bucks and SquadHelp users will find you some 500+ domain names, all of which are currently available. And if you don’t like any of them, you don’t pay anything.

Note: 99% of the suggestions are going to suck, but I always find that are good ones are worth the money, and they provide inspiration to come up with my own names.

Looking over the list of possible names for our crowdfunding aggregator, we chose our top 3:

  1. altFunder.com
  2. ChangeFunding.com
  3. ThingsWeStart.com

altFunder.com was our favorite name going in to the experiment, the next step of which was deciding…

Step 2: What to Measure?

After reading up on what makes for a good company name, we decided to measure the following characteristics of each name:

  1. Memorability – If users can’t remember your name, they can’t tell their friends about it. tweet-this-button
  2. Spellability & Hearability – If users can’t spell your domain, they could become someone else’s. tweet-this-button
  3. Associations
    1. Emotional associations – what feelings do these names evoke? For more on why this is important see Selling the Why
    2. Image associations – Names people automatically associate images with are more memorable.
    3. Competitor associations – The internet told us you want to avoid a name that gives your competition more clout. Seemed like reasonable advice.

Next up…

Step 3: Design the Experiment

Here’s a video tour of the survey we designed (i.e. why you’re reading this post):

If you don’t want to watch the video, here are the key questions we asked:

To test hear/spellability (try it):

1. Play this audio clip only once and type in the name of the website you hear:

What website did you hear? [input box goes here]

To test memorability:

After speaking & showing the domain names, we’d ask them a bunch of “busy work” questions (e.g. demographic questions, trivial math questions, etc.) to help them forgot the less memorable names:

12. What is the 9th word of the 1st paragraph on this page?

  • eat
  • faster
  • rubber
  • printing

With their minds distracted for a bit, we’d then ask:

Testing domain name memorability

Testing domain name memorability

To test Associations:

For each of the associations (e.g. emotional, image, and competitor) we asked them simple questions like:

  • What images come to mind when you hear this time?
  • How do you feel when you hear this name?
  • What organizations come to mind when you hear this name?

Btw, we used nsurvey for this – an open source survey tool that gave us the power & flexibility we needed. With the survey designed, it was time to…

Step 4: Run the Experiment

Pssst: If you like this blog post so far, you should subscribe (email or RSS)! The next one will be an in-depth look at Steve Blank’s new collaboration with Startup Weekend.

The site was B2C, so we decided to throw our survey up on Mechanical Turk.  We paid $0.10 for each respondent and we stopped after we got 50 (all told $5).

Mechanical Turk (mTurk) domain name survey HIT

mTurk naming survey HIT with bonus fraud detection

Fraud Detection: we had quality assurance questions built into the survey (e.g. what’s the 9th word in the second paragraph, what’s 16 + 32, etc.) to detect cheaters. We gave cheaters a different password than people who took the time to fill out the survey thoroughly.

Step 5: Evaluating the Results

Once the survey results were in, I threw them in a spreadsheet and scored each of the questions like so:

Hearability/Spellability: +2 points for a name each time it was spelled properly

Here are the Hear/Spellability results for our names:

Domain name hearability/spellability results

Domain name hearability/spellability results

Memorability: +2 points each time the name was properly recalled

Domain name memorability results

Domain name memorability results

Image associations: scored subjectively by me (actual responses below)

Positive (+1 point): “people protesting”, “trees, plants, soil, nature”, “tattooed girls and rock music”
Neutral (0 points): “people”, “money”, “change”, “coins”
Negative (-1 point):  “a steaming pile”, “a payday loan center”, “my cluttered basement”

Domain name image association results

Domain name image association results

Feeling associations:

Positive (+1 point): “curious”, “hopeful”, “interested to know what people are starting”
Neutral (0 points): “nothing”, “innovation, kind of a cheesy name”, “mildly inspired”
Negative (-1 point): “annoyed”, “confusion, boredom”, “lame”

Domain name feeling association results

Domain name feeling association results

Competitor associations:

Positive (+1 point): “none”, “no particular companies come to mind”
Neutral (0 points): “charity”, “Kickstarter”, “red cross”
Negative (-1 point): “republican websites”, “Obama campaign”, “payday loans”

Domain name existing websites association results

Domain name existing websites association results

Then we weighted the score of people who we thought were most like our targeted users (e.g. responses from Facebook and Kickstarter users got bumps) and added them all up:

Domain name final weighted results

Domain name final weighted results

View/download the complete spreadsheet here: http://sdrv.ms/Z9941S

Conclusion

We were wrong!  We were leaning towards altFunder.com because it was short & edgy (and thus we assumed more memorable).

Turns out, ThingsWeStart.com was not only more memorable, it was also easier to hear/spell and it had much more positive associations.

For a couple bucks and a couple hours, we saved ourselves thousands of $’s by not picking the crappy name and having to change it later, or trying to buy one from a squatter.

Takeaway:

Don’t waste time debating product names…test them. tweet-this-button

Tips & Tricks

- Before you post a survey about your domain names, be sure to purchase them all. Don’t want one of your clever respondents to snipe them from you.

- Most domain registrars offer a return policy. We bought all 3 domains we were testing, ran our test, and then got a refund for the 2 we didn’t use!

- Another great test would be to measure CTR on ads, or A/B test on a landing page. We didn’t have time for that, but I’d love to hear if you’ve run an experiment like that.

As always, comments and questions are appreciated!

If you liked this - consider sharing it so your friends don’t buy shitty domain names. You may also want to subscribe for new posts via Email or RSS. Our next article will be detailed explanation of Steve Blank & Startup Weekend’s Customer Development collaboration – SWNext – and how we think it fits into the greater startup ecosystem.

What’s Next?

Want help picking a domain? Schedule a 1-on-1 mentoring call.

First time to Customer Development Labs?

Check out our most popular experiments:

  1. Interviewing 100 customers in 4 hours with MTurk
  2. Crowdtesting: How many $’s is your idea worth?
  3. How assumptions made an ass out of my startup
About these ads

39 comments

  1. Meaghan Nolan

    Wow, this is a great approach to naming your website, and laying it out step by step is super helpful. I think the use of Mechanical Turk is a great idea, as it provides you with essentially on-demand survey results at such a low cost.

    How long did you have to wait to get your 50 respondents?

    • Justin Wilcox

      If I recall it only took one evening. Whatever it was, it wasn’t very long and if it’s taking longer than you like, you can up the payment amount. (and thanks for the question!)

  2. Orion Bukantis

    This is awesome! I wouldn’t have thought to check associations.
    I’ve been wondering whether there are better names than Logical Increments for my thing. Now I know how to find out.
    Thanks for sharing, Justin!

  3. Alina Morris

    This is great! Please also consider looping in a trademark lawyer before the final decision, to help you evaluate whether the name you’ve selected is a good one legally as well. They can help you confirm whether it’s unlikely to conflict with someone already out there, and can function as a “strong” trademark. This is truly an ounce of prevention (a lawyer’s look-see) vs. a pound of cure (letters, lawsuit, and total re-brand) consideration!

    • Justin Wilcox

      Fair point, thanks for bringing it up Alina. That said, we can probably save ourselves a couple (hundred) bucks by using the online trademark search tool: http://1.usa.gov/YMHsyJ. If the name that users choose is one that conflicts w/ an existing trademark, perhaps then it’s time to get an official legal opinion.

      Thoughts?

  4. Pedro Vieira

    Justin, awesome “hack”!

    2 other things we’re also using:
    - “Pronounceability”: give Turkers a list of the potential names and have them call a Google Voice Mail and read the names. You can then go through all the recordings to see how people were pronouncing the names. In our case, some names were “made up” words and we expected to get different pronunciations, but we used that to see if some of these different pronunciations could results in spellability problems later down the road. So we took the Voice messages from Turkers (one male and one female) and used those for the spellability tests (which we ran in the same you you did).

    - Mission Alignment: We describe the company mission and ask them which names better reflect that mission

    • Justin Wilcox

      Those are awesome ideas Pedro! Love the “pronounceability” idea, especially if you’re testing non-native language words and then reusing them as your spellability test. Great stuff!

      Btw, what’s your new name?

      • Pedro Vieira

        ah ah! That’s an even better question! We’re running final tests this weekend. Will shoot you an email soon with news. I can also share some data if you find it useful.

    • Abhilash

      +1 for Pedro. Once it becomes a branding exercise, there’s all kinds of of work to do. But those are great. Another question we sometimes ask is “Scanability” of a domain – basically whether your eyes can scan a name and actually comprehend the words that are being squished together.
      Ex:
      ThingsWeStart scans well
      Thingsstevecandoonsundays does not (repetitive letters, etc.)

      Case-sensitivity aside, it’s nice when a domain name seems to jump off a page when written in all lower caps.

  5. Abhilash

    Well done! It’s amazing how many people assume their own instincts on a name are always right. Yet, testing can be a daunting challenge. You made the test strong by measuring meaningful qualities, but you also made the test approachable/accessible. Doubt it would only take a couple hours (!) but great work. Wish more people did this in advance if picking their shitty company names.

  6. Athula

    Great test… but couldn’t help wondering how Google, Yahoo or eBay would have faired on this test if they ran it prior to launch.

    • Justin Wilcox

      Fair question. That said, if you had to choose between having a more memorable name, and a less memorable one…the choice would be obvious.

      I suppose what your question raises though is, does a name actually matter at all? I don’t know. My very un-data-driven answer to that question is, “Probably a little.”

    • Justin Wilcox

      Concept would be similar, you want to test the hearability, memorability and associations with a B2B brand.

      One way would be to go to a conference with your customers and try telling people different domain names. Make a note of how many times you need to repeat each domain name you say out loud.

      You could also make up a small set of business cards for each domain name and hand out an equal number of them. See if any of the domains generates more traffic than the others.

      Similar concepts – get your customers to demonstrate which name you should choose – just may need slightly tweaking for each customer base.

      Best of luck!
      Justin

    • Beth Morgan

      I loved this method so much I just tried it out for a B2B company I’m working with. Changes I made:
      1) I didn’t put it on Mechanical Turk but rather posted it on LinkedIn and emailed my contacts who fit the proper profile
      2) Instead of busy work like adding up numbers, etc, I asked a couple screens of demo-type questions. I thought the random busy work would just confuse and irritate my respondents, and since I wasn’t paying them I needed them to respond out of goodwill.

  7. StartupPanda

    Hey Justin. Absolutely awesome idea! I’m coming to your blog quite often lately ;)
    Can you tell me what tool you used to build the questionnaire? I tried surveymonkey, but couldn’t implement soundcloud-files (with the basic account).
    What’s the best way to go here?

  8. Tanner Jones

    Justin, thank you so much for this – used this process to pick our startup name about 2 months and it has worked out perfectly. Keep this stuff coming, I love it!

  9. Paul

    great article. did you use squadhelp for the measurement? I could find the naming option, but not the testing. Wasn’t sure if you used a different website for the other portions…. thanks!

  10. Ash

    Hi Justin,

    I’m not too sure what happened but it looks as though my post from last night is no longer there! Anyways, great article, it really has been a useful tool for me. Just one question, how did you create the last question in nsurvey which gave users who answered the CAPTCHA questions correctly, the correct code for MTurk?

    Thanks for your help on this!

    • Justin Wilcox

      Great question Ash! I actually had two different “Thank You” pages for the survey. One page with the correct code, the other an incorrect one. I then added some simple conditional logic to the survey that said, “If they answered correctly to the CAPTCHAs, send them to the good page. If they answered incorrectly, send them to the bad page.”

      That make sense?

      Good luck!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s