Back
How to (Scientifically) Pick the Best Domain Name
March 5, 2013

tl;dr

Using clever survey questions to quickly test the memorability, spellability and emotional response of potential domain names. Glad we did – almost picked a lousy one.

Problem: Domain Squatters

I’ve found what should be the joyous process of naming a company, quickly devolves into in argument over which of the abysmal choices of available domain names suck the least.

Some friends and I who were building a crowdfunding aggregator wanted to avoid all that, so we came up with a way to test our company/domain names with customer data. Here’s how…

Step 1: Crowdsource Ideas

SquadHelp is 99Designs for domain names. tweet-this-button That means you pay a couple bucks and SquadHelp users will find you some 500+ domain names, all of which are currently available. And if you don’t like any of them, you don’t pay anything.

Note: 99% of the suggestions are going to bad, but I always find that are good ones are worth the money, and they provide inspiration to come up with my own names.

Looking over the list of possible names for our crowdfunding aggregator, we chose our top 3:

  1. altFunder.com
  2. ChangeFunding.com
  3. ThingsWeStart.com

altFunder.com was our favorite name going in to the experiment, the next step of which was deciding…

Step 2: What to Measure?

After reading up on what makes for a good company name, we decided to measure the following characteristics of each name:

  1. Memorability – If users can’t remember your name, they can’t tell their friends about it. tweet-this-button
  2. Spellability & Hearability – If users can’t spell your domain, they could become someone else’s. tweet-this-button
  3. Associations
    1. Emotional associations – what feelings do these names evoke? For more on why this is important see Selling the Why
    2. Image associations – Names people automatically associate images with are more memorable.
    3. Competitor associations – The internet told us you want to avoid a name that gives your competition more clout. Seemed like reasonable advice.

Next up…

Step 3: Design the Experiment

Here’s a video tour of the survey we designed (i.e. why you’re reading this post):

If you don’t want to watch the video, here are the key questions we asked:

To test hear/spellability (try it):

1. Play this audio clip only once and type in the name of the website you hear:

What website did you hear? [input box goes here]

To test memorability:

After speaking & showing the domain names, we’d ask them a bunch of “busy work” questions (e.g. demographic questions, trivial math questions, etc.) to help them forgot the less memorable names:

12. What is the 9th word of the 1st paragraph on this page?

  • eat
  • faster
  • rubber
  • printing

With their minds distracted for a bit, we’d then ask:

Testing domain name memorability
Testing domain name memorability

To test Associations:

For each of the associations (e.g. emotional, image, and competitor) we asked them simple questions like:

  • What images come to mind when you hear this time?
  • How do you feel when you hear this name?
  • What organizations come to mind when you hear this name?

Btw, we used nsurvey for this – an open source survey tool that gave us the power & flexibility we needed. With the survey designed, it was time to…

Step 4: Run the Experiment

Pssst: If you like this blog post so far, you should subscribe (email or RSS)! The next one will be an in-depth look at Steve Blank’s new collaboration with Startup Weekend.

The site was B2C, so we decided to throw our survey up on Mechanical Turk.  We paid $0.10 for each respondent and we stopped after we got 50 (all told $5).

Mechanical Turk (mTurk) domain name survey HIT
mTurk naming survey HIT with bonus fraud detection

Fraud Detection: we had quality assurance questions built into the survey (e.g. what’s the 9th word in the second paragraph, what’s 16 + 32, etc.) to detect cheaters. We gave cheaters a different password than people who took the time to fill out the survey thoroughly.

Step 5: Evaluating the Results

Once the survey results were in, I threw them in a spreadsheet and scored each of the questions like so:

Hearability/Spellability: +2 points for a name each time it was spelled properly

Here are the Hear/Spellability results for our names:

Domain name hearability/spellability results
Domain name hearability/spellability results

Memorability: +2 points each time the name was properly recalled

Domain name memorability results
Domain name memorability results

Image associations: scored subjectively by me (actual responses below)

Positive (+1 point): “people protesting”, “trees, plants, soil, nature”, “tattooed girls and rock music”
Neutral (0 points): “people”, “money”, “change”, “coins”
Negative (-1 point):  “a steaming pile”, “a payday loan center”, “my cluttered basement”

Domain name image association results
Domain name image association results

Feeling associations:

Positive (+1 point): “curious”, “hopeful”, “interested to know what people are starting”
Neutral (0 points): “nothing”, “innovation, kind of a cheesy name”, “mildly inspired”
Negative (-1 point): “annoyed”, “confusion, boredom”, “lame”

Domain name feeling association results
Domain name feeling association results

Competitor associations:

Positive (+1 point): “none”, “no particular companies come to mind”
Neutral (0 points): “charity”, “Kickstarter”, “red cross”
Negative (-1 point): “republican websites”, “Obama campaign”, “payday loans”

Domain name existing websites association results
Domain name existing websites association results

Then we weighted the score of people who we thought were most like our targeted users (e.g. responses from Facebook and Kickstarter users got bumps) and added them all up:

Domain name final weighted results
Domain name final weighted results

View/download the complete spreadsheet here: http://sdrv.ms/Z9941S

Conclusion

We were wrong!  We were leaning towards altFunder.com because it was short & edgy (and thus we assumed more memorable).

Turns out, ThingsWeStart.com was not only more memorable, it was also easier to hear/spell and it had much more positive associations.

For a couple bucks and a couple hours, we saved ourselves thousands of $’s by not picking the crappy name and having to change it later, or trying to buy one from a squatter.

Takeaway:

Don’t waste time debating product names…test them. tweet-this-button

Tips & Tricks

Before you post a survey about your domain names, be sure to purchase them all. Don’t want one of your clever respondents to snipe them from you.

Most domain registrars offer a return policy. We bought all 3 domains we were testing, ran our test, and then got a refund for the 2 we didn’t use!

Another great test would be to measure CTR on ads, or A/B test on a landing page. We didn’t have time for that, but I’d love to hear if you’ve run an experiment like that.

As always, comments and questions are appreciated!

If you liked this – consider sharing it so your friends don’t buy shitty domain names. You may also want to subscribe for new posts via Email or RSS. Our next article will be detailed explanation of Steve Blank & Startup Weekend’s Customer Development collaboration – SWNext – and how we think it fits into the greater startup ecosystem.

Customer Development Made Easy…

guest

46 Comments
Sort by
Newest
Oldest Most Voted

Inline Feedbacks
View all comments
Drew

Hey Justin,

How did you add the audio recordings to the questions in nsurvey, or what program did you use record then upload?? I am trying to replicate this process but you made it look so easy :D

Reply
handground

Thanks for sharing this Justin! If we hadn’t done this test, we would of chose the name “Kaizen” which it turns out is very difficult for people to spell and is associated with negative images.

For anyone who is trying to replicate this process and hasn’t used Amazon Mechanical Turk before, we put together a step by step guide and built a business name generator template in Google Sheets that is free to use. http://handground.com/blog/business-name-generator-5-hacks-to-find-and-test-the-perfect-business-name/

Here is a screenshot of the results:
http://i2.wp.com/handground.com/blog/wp-content/uploads/2014/10/Screen-Shot-2014-10-13-at-1.11.41-PM.png?zoom=2&resize=474%2C214

Reply
Chi

Justin the information is invaluable, thank you for the great post! I am embarking on a similar exercise based on this. I had a quick question, on mTurk did you use Project type “Survey” or “Other” to create the HIT? Also how many respondents did you get to consider it a relevant sample for statistical significance. Thanks in advance.

Reply
Nicole Schwegman

Hi Justin, where do you purchase your domain names? I can’t seem to find any place that will allow me to return the names.

Reply
brian

“couple” means two

Reply
Ash

Hi Justin,

I’m not too sure what happened but it looks as though my post from last night is no longer there! Anyways, great article, it really has been a useful tool for me. Just one question, how did you create the last question in nsurvey which gave users who answered the CAPTCHA questions correctly, the correct code for MTurk?

Thanks for your help on this!

Reply
Justin Wilcox

Great question Ash! I actually had two different “Thank You” pages for the survey. One page with the correct code, the other an incorrect one. I then added some simple conditional logic to the survey that said, “If they answered correctly to the CAPTCHAs, send them to the good page. If they answered incorrectly, send them to the bad page.”

That make sense?

Good luck!

Reply
Paul

great article. did you use squadhelp for the measurement? I could find the naming option, but not the testing. Wasn’t sure if you used a different website for the other portions…. thanks!

Reply
Justin Wilcox

Squadhelp only helps with the naming. You’ll need to do the measurement yourself.

Best of luck!
Justin

Reply
Iris van de Kieft (@iris__k)

Which host did you use that allows you to return the domain names you didn’t use? My host doesn’t do it..

Reply
Justin Wilcox

Any of the GoDaddy affiliates should work – or at least did when I ran this experiment. Best of luck!

Reply
Iris van de Kieft (@iris__k)

Thanks!

Reply
Cristina

I found this to be a very helpful guide for creating a memorability survey. Thanks for sharing!

Reply
Justin Wilcox

Awesome :)

Reply
Tanner Jones

Justin, thank you so much for this – used this process to pick our startup name about 2 months and it has worked out perfectly. Keep this stuff coming, I love it!

Reply
StartupPanda

Hey Justin. Absolutely awesome idea! I’m coming to your blog quite often lately ;)
Can you tell me what tool you used to build the questionnaire? I tried surveymonkey, but couldn’t implement soundcloud-files (with the basic account).
What’s the best way to go here?

Reply
Mark Varnas

What would change if test was for B2B site (and not B2C as in this case)?

Reply
Beth Morgan

I loved this method so much I just tried it out for a B2B company I’m working with. Changes I made:
1) I didn’t put it on Mechanical Turk but rather posted it on LinkedIn and emailed my contacts who fit the proper profile
2) Instead of busy work like adding up numbers, etc, I asked a couple screens of demo-type questions. I thought the random busy work would just confuse and irritate my respondents, and since I wasn’t paying them I needed them to respond out of goodwill.

Reply
Justin Wilcox

Thanks for the suggestions Beth! How did the experiment turn out? Any valuable new information?

Reply
Beth Morgan

Not as decisive as I hoped, actually. I kind of wanted one of the names to be either a total loser or a clear winner. They were fairly even.

Reply
Mark V

The test was for B2C site. How would it be different if done for B2B site?

Reply
Justin Wilcox

Concept would be similar, you want to test the hearability, memorability and associations with a B2B brand.

One way would be to go to a conference with your customers and try telling people different domain names. Make a note of how many times you need to repeat each domain name you say out loud.

You could also make up a small set of business cards for each domain name and hand out an equal number of them. See if any of the domains generates more traffic than the others.

Similar concepts – get your customers to demonstrate which name you should choose – just may need slightly tweaking for each customer base.

Best of luck!
Justin

Reply
Beth Morgan

Have you ever spoken at the Lean Startup Conference? This would make a great presentation for them. And they are accepting applications until 5/16. =) http://www.startuplessonslearned.com/2013/05/how-to-get-free-speaker-training-with.html

Also, what survey software did you use? Is that just basic Survey Monkey?

Reply
Justin Wilcox

Thanks Beth! We used NSurvey.

Re the Lean Startup Conference, I have spoken there!

Reply
Beth Morgan

Oh yeah! I remember that one. (I spoke last year as well.)
Also putting in a vote for “This would be a good product”! Think what people pay naming firms, how much time they spend checking for domains, etc. One service that knit all this together and helped test it… people would pay for that.

Reply
Athula

Great test… but couldn’t help wondering how Google, Yahoo or eBay would have faired on this test if they ran it prior to launch.

Reply
Justin Wilcox

Fair question. That said, if you had to choose between having a more memorable name, and a less memorable one…the choice would be obvious.

I suppose what your question raises though is, does a name actually matter at all? I don’t know. My very un-data-driven answer to that question is, “Probably a little.”

Reply
Abhilash

Well done! It’s amazing how many people assume their own instincts on a name are always right. Yet, testing can be a daunting challenge. You made the test strong by measuring meaningful qualities, but you also made the test approachable/accessible. Doubt it would only take a couple hours (!) but great work. Wish more people did this in advance if picking their shitty company names.

Reply
Pedro Vieira

Justin, awesome “hack”!

2 other things we’re also using:
– “Pronounceability”: give Turkers a list of the potential names and have them call a Google Voice Mail and read the names. You can then go through all the recordings to see how people were pronouncing the names. In our case, some names were “made up” words and we expected to get different pronunciations, but we used that to see if some of these different pronunciations could results in spellability problems later down the road. So we took the Voice messages from Turkers (one male and one female) and used those for the spellability tests (which we ran in the same you you did).

– Mission Alignment: We describe the company mission and ask them which names better reflect that mission

Reply
Justin Wilcox

Those are awesome ideas Pedro! Love the “pronounceability” idea, especially if you’re testing non-native language words and then reusing them as your spellability test. Great stuff!

Btw, what’s your new name?

Reply
Pedro Vieira

ah ah! That’s an even better question! We’re running final tests this weekend. Will shoot you an email soon with news. I can also share some data if you find it useful.

Reply
Abhilash

+1 for Pedro. Once it becomes a branding exercise, there’s all kinds of of work to do. But those are great. Another question we sometimes ask is “Scanability” of a domain – basically whether your eyes can scan a name and actually comprehend the words that are being squished together.
Ex:
ThingsWeStart scans well
Thingsstevecandoonsundays does not (repetitive letters, etc.)

Case-sensitivity aside, it’s nice when a domain name seems to jump off a page when written in all lower caps.

Reply
Antonio Moreda (@amoreda)

Awesome post! Another resource to check possible trademark infringement: http://www.trademarkia.com/. I remember when we discussed whether Q… was a good name for the startup I was developing and you were right. It turns out it did horrible in the three categories of hear/spellability, memorability and associations.

Reply
Justin Wilcox

Great to hear from you Antonio! Stoked to hear about the problem Skyin solves.

Reply
hortovanyi

Reblogged this on Nick Hortovanyi's blog.

Reply
Alina Morris

This is great! Please also consider looping in a trademark lawyer before the final decision, to help you evaluate whether the name you’ve selected is a good one legally as well. They can help you confirm whether it’s unlikely to conflict with someone already out there, and can function as a “strong” trademark. This is truly an ounce of prevention (a lawyer’s look-see) vs. a pound of cure (letters, lawsuit, and total re-brand) consideration!

Reply
Justin Wilcox

Fair point, thanks for bringing it up Alina. That said, we can probably save ourselves a couple (hundred) bucks by using the online trademark search tool: http://1.usa.gov/YMHsyJ. If the name that users choose is one that conflicts w/ an existing trademark, perhaps then it’s time to get an official legal opinion.

Thoughts?

Reply
mattdecuir

Nicely done Justin! If you make this into a product, I’d pay to test company names.

Reply
Justin Wilcox

Thanks Matt – you’re the second person to mention that…wonder if there’s something there.

Reply
Orion Bukantis

This is awesome! I wouldn’t have thought to check associations.
I’ve been wondering whether there are better names than Logical Increments for my thing. Now I know how to find out.
Thanks for sharing, Justin!

Reply
Meaghan Nolan

Wow, this is a great approach to naming your website, and laying it out step by step is super helpful. I think the use of Mechanical Turk is a great idea, as it provides you with essentially on-demand survey results at such a low cost.

How long did you have to wait to get your 50 respondents?

Reply
Justin Wilcox

If I recall it only took one evening. Whatever it was, it wasn’t very long and if it’s taking longer than you like, you can up the payment amount. (and thanks for the question!)

Reply
markhoroszowski

Nice! When you posting on hacker news?

Reply
Liz Angoff

sweet!

Reply