Matt Mazur

Yesterday I launched Lean Domain Search, a new domain search engine, on HackerNews.

You can read the discussion here: Review my Startup – Lean Domain Search – The fastest way to find a domain name.

It started with Domain Pigeon

Lean Domain Search is a follow up to Domain Pigeon, my first Rails project which I launched (also on HackerNews) a little less than three years ago.

Lean Domain Search is what Domain Pigeon was supposed to be: you type in a search phrase and the app pairs it with other words to generate domain names and instantly shows you which are available. I even kept a notebook in my car and when I saw a company name that started or ending with something that would also work for a domain name, I’d jot it down. One problem though: I didn’t know how to do it.

It’s not hard to programatically check whether a domain name is available, but doing it in bulk and doing it fast is. When I started trying to implement bulk search for Domain Pigeon, I couldn’t figure out a way to do it well. I decided to pivot slightly and simply listed available web 2.0-style domain names instead.

Domain Pigeon did well, but I made a bunch of mistakes which eventually led to its downfall. I remember thinking foolishly “Man, making money with web apps is easy.” Domain Pigeon was never supposed to take off. It was a something I undertook to learn web development; it wasn’t supposed to be what I became known for. Yet I’d go to Philly on Rails meetups and I’d be the “Domain Pigeon guy”. And so I started working on other projects, one of which eventually became Preceden. I kept working on Domain Pigeon on the side, but as I became more and more involved in Preceden’s development, I let Domain Pigeon slide and eventually just said screw it and shut it down.

Looking back, my two big takeaways from Domain Pigeon where:

1) It’s hard to make money with web apps. If you are lucky enough or talented enough to find something that people like and will pay for, double down.

2) If you do decide to work on new projects, don’t abandon your existing ones. If you don’t have the discipline to maintain existing projects while building new ones, then don’t build new ones until you do.

Lean Domain Search aka Domain Pigeon 2.0

I launched Preceden and eventually Lean Designs, but I still had the domain name itch. I really wanted to build that original search tool and continued brainstorming ways to do it. About five months ago I decided to take a break from Preceden and Lean Designs development and go for it (without abandoning either — see #2 above).

Lean Domain Search is the result. It’s better that what I originally envisioned: it generates and checks the availability of 1,000 domain names in about 2 to 3 seconds (go check it out!).

The HackerNews launch

The launch on HackerNews went well, but it was not without issues (launch day never is).

The biggest problem was the accuracy of the results. I estimated beforehand that about 1 in 1000 available results would be actually be registered due to some quirks with the way they were being checked. That turned out to be a tad bit optimistic: it was more like 1 in 20. As an extreme example, the results for “cloud” returned 10 results, none of which were actually available. This was understandably frustrating to anyone who searched for “cloud”, found domain names that he or she got excited enough to register, and then found out it was actually registered.

As a temporary solution, I added a “Double-check availability” button that let folks confirm an available domain name was indeed available before going off to register it. This worked great on my local computer, but there was a bug in production that caused it to say that every result was registered. I didn’t notice this until a few hours later by which time 800 people had double-checked the availability of a domain name and Lean Domain Search said the equivalent of “Oh, actually it’s registered. My bad.”

There was a hectic hour of hacking after that where I troubleshot and fixed the double-check feature, but it still felt sloppy. Everyone searching for “cloud” still got those same 10 incorrect results. Eventually it clicked: why not track the double-check results and stop showing ones that come back registered?

After another few hours of hacking, I had a decent solution in place: if you double-checked the result of an available domain name and it came back as registered, Lean Domain Search would remember that and it would no longer be shown in the available search results for you or anyone else. This way the results become more and more accurate as more people use it. 1 in 1000 might never be possible, but the false positive rate should eventually be closer to 1 in 100 than 1 in 20 (and thanks to Mixpanel, I can actually track that over time).

This morning I spent a few hours cleaning up the code and making other improvements. For example, when you click a domain name it will now automatically double check the result so you don’t have to.

I also found a typo today in the Namecheap affiliate link that caused my affiliate code to be dropped so I didn’t get credit for who knows how many domain name registrations yesterday. Fun!

But it was still a good day. All in all folks used Lean Domain Search to make about 14,000 searches yesterday thanks to it sitting on the HackerNews front page for almost 20 hours. Launch day traffic is always inflated though (don’t be fooled!); the trick is getting it back up to that point over the next few months.

The mentions on Twitter about Lean Domain Search were very positive too — thank you to everyone who Tweeted about it.

Here’s a screenshot from today:

On a final note, I should be blogging a lot more in the future. If you’re not subscribed to the RSS feed, you should.

Domain Pigeon has been among the living dead for quite a while now. The site has been live, but it hasn’t been updated in about two years.

As part of my account clean up today, I’m putting a nail in it.

Here’s to you old friend.

Domain Pigeon was recently mentioned in a Smashing Magazine article titled 10 Tools For Finding, Registering And Managing Domain Names:

Domain Pigeon’s approach is different to the search tools mentioned before. It automatically generates lists of available domains as well as Twitter names. The names are displayed in different colors, depending on how many people have showed interest in a certain term. The darker the color the more popular the name. You can also order the lists by length, popularity and show only domains or Twitter usernames. Examples of generated domain names are ablebo.com, meliori.com, minecafe.com or shopshost.com.

Domain Pigeon is an ideal source of inspiration for domains and short available Twitter names.

Cool cool.

Domain Pigeon has been down for the last few days.

Recent visitors to the site saw this uninviting error message:

The error, “unitialized constant ApplicationController”, was remedied by SSHing into Dreamhost and running the following command:

rake rails:update:application_controller

… which renames application.rb to application_controller.rb

I think Dreamhost updated something on their end which caused the error.

Regardless, its back up — more domains to come shortly.

A new domain name tool just launched called Wordoid.

The thing I like the most about it is how it lets you enter in a word and then it generates available domain names around that word. Domain Pigeon, on the other hand, only lets you search names that already exist on the site.

There were a few interesting mentions of Domain Pigeon in the Hacker News thread on Wordoid’s launch:

Yeah, I was a fan of http://domainpigeon.com/ for a while, but that got a bit too restricted for me (as it became more commercial), not to mention a bit unwieldy to browse through. Bookmarked!

…

Wow, it’s like domain pigeon only way better. Awesome.

…

It’s nice. It reminds me of domain pigeon, but the words it creates seem a lot more natural.

The first point is valid (visitors can’t explore a lot of the names until they sign up). The problem, in my experience, is that its very hard to make money from a site like this purely as an affiliate. Jorge, from hotnamelist.com, can attest to that fact too.

I’ve had a new feature on the todo list since the site launched that would let you do exactly that, but I’ve let a few technical hurdles stop me from implementing it.

Maybe this is the kick I need to get around to doing it.

I’ve spent about a year working on Domain Pigeon and I’ve learned a lot in that time and I’m proud of what the site has become, but its time to start thinking about what’s next.

I’ve been tossing around a few ideas for what to work on and I think I’ve finally settled on a project. I’m very excited about it and I think it will be life changing for a lot of people.

I plan to use the following technologies to make it:

• Python/Django
• Flash/Actionscript
• jQuery
• Google Maps API

I could probably do it with Ruby on Rails and Prototype, but a lesson learned from a decade of Visual Basic work has taught me that it’s wise to learn something new for every project you undertake. In this case, I don’t have any experience with any of those things, so it’s going to take some time.

At some point in the next few months I’m also going to use the profits from Domain Pigeon to form an LLC to cover this project and any others I might start in the next few years.

I will continue to work on Domain Pigeon as time permits, but with only a few hours a day to work, this new project will be my priority. Domain Pigeon will continue to be updated daily with new domain and Twitter names and there are a few more major features I want to implement, but the days of small layout changes are at an end.

Lots to learn, lots to do.

For the last two weeks I busily worked on a new addition to Domain Pigeon that would let visitors sort the domain names by pronounceability, as determined by folks on Mechanical Turk. I was excited. It was an innovative way to help people find great domain names and it gave me an excuse to post another link to it on HackerNews.

And so, on Monday morning I posted it on HackerNews: Finding Great Domain Names using Mechanical Turk.

I eagerly awaited feedback, refreshing the comments every few minutes.

Well, things did not go as planned. You know that feeling you get when you take a test and you’re so sure that you got an A that you tell everyone how well you did, but when you get it back you find out that you actually got a D? That was Monday, kind of.

In this case, it was more like the teacher couldn’t grade the test because he couldn’t read your handwriting.

Here’s a snapshot of the comments:

Matt, your site is crawling

…

I have to say that if this weren’t HN, your site was posted, and it timed out, I would never come back again. I mean no disrespect; I really like your idea, but going to a site these days and timing out makes the company look like it runs out of a cardboard box with a 2400 baud modem.How many people trust their credit card information to a site with slow response times.

…

DreamHost? Uh oh… Methinks you aren’t getting 1/1,000th of the hits you could be. DreamHost = dog slow with two concurrent connections to a fairly light rails app, in my experience. Please tell me you’re running a VPS with them or something. You might want to consider switching hosts before you lose anymore of these hits.

…

The “guest” links don’t work.

For me, I tried to follow the link for domain names sorted by pronouncability and they did not work. I assume this is what OP meant.link href:

http://www.domainpigeon.com/domains?length=6&sort_by=pro…

error:

Oops! That Page Doesn’t Exist

You may have mistyped the address or the page may have moved.

Please contact support@domainpigeon.com if you have any questions or concerns.

…

Just clicked the register link [https://www.domainpigeon.com/users/register], got “Internal server error” with a load of info and a backtrace, which I imagine you don’t want to display.

Ugh. Domain Pigeon wasn’t loading for a lot of people and for those that were lucky enough to get through, it was about as slow as me picking up country line dancing.

A few hours later I received the following email, which really drives home the impact it was having:

I hate it when that happens…

I had come back from PayPal and clicked the complete order link, when I
got the Passenger error screen.  (Probably a security hole, too.)

Did I actually pay or not?

Thanks

The worst part was that it was probably avoidable. At first I blamed the problems on the heavy traffic and using a shared host, but I later discovered that the likely culprit was a 238 MB log file which I had been remiss in clearing.

In total I got 2 sign ups on Monday and 1 on Tuesday, just enough ($120) to cover the costs of the Mechanical Turk experiment ($106).

I can’t help but think about how many it would have been if the site had worked well.

(Cross-posted from Domain Pigeon)

Overview

• Finding great available domain names just got a little bit easier with some help from Amazon’s Mechanical Turk community.
• The community rated the pronounceability of Domain Pigeon’s entire collection of five and six letter .com domain names.
• It worked well: Check out the six letter domain names sorted by pronounceability
• Five is better than six! Sign up for an account to explore all the five letter domain names.
• Added bonus: To celebrate, we’re adding 1,000 new five letter domain names per day from Monday, May 11 through Friday, May 15, 2009 at 3pm EST.

Background

Anyone that’s attempted to search for an available domain name recently can attest to how hard it is to find a good name. If you’ve ever spent any time on Ajaxwhois.com, you know what I mean. You can futilely sit there for hours typing various combinations of words until you finally get so frustrated that you settle on something that’s long, hyphenated, unpronounceable, and has six to ten numbers tacked on the end. Not exactly your ideal name.

Domain Pigeon was created to help alleviate the pain of finding a good domain name. We find available domain names and then list them on this website so that you can browse them and find ones that appeal to you.

To be clear: it’s not difficult to find available domain names, but it is hard to find good ones. Smack your keyboard against your desk about ten times until you have something that resembles xifs-duodis9.com and chances are you’ll be able to register it. But that’s not what you want. You want something marketable. Something remarkable.

We’ve used various methods to find quality domain names. For example, one way to come up with domain names is to simply add “i” or “e” to the beginning of a word and check whether that domain is available: iPhone.com, ePhone.com, iStore.com, eStore.com, etc. We’ve got plenty of those (‘eSlanting.com,’ anyone?), but that’s not what most people want. Another method is to check every group of five letters within a word, hoping that it produces a decent name. For example, using the word “alphabetic” we could check the availability of alpha.com, lphab.com, phabe.com, habet.com, abeti.com, and betic.com.

When you do that though, you get a lot of garabge like lphab.com. It’s barely pronounceable and when added with the other names it adds a lot of clutter to the site. The problem is that its not that easy for the computer to tell which names are good and which are bad.

Until recently, I’ve simply let the visitors sort it out. I posted everything using the philosophy: Don’t like it? Don’t register it.

A Better Way

I recently saw Mike Culver, an Amazon Web Services evangelist, speak at the Philly Emerging Tech conference. I had heard of Mechanical Turk and knew the concept, but was curious to learn more so I attended his talk on it.

For the unenlightened, Mechanical Turk is, in Amazon’s words, “a marketplace for work that requires human intelligence.” Basically, its a system that enables thousands of people around the world to help you with tasks that computers have a hard time doing. In exchange for their help, you pay them some small amount. For example, its very difficult for computers to analyze the contents of a picture so you could use Mechanical Turk to tag images or to identify pornography (yes: someone will pay you to look at porn).

On the drive home that day I was thinking about ways to use it for Domain Pigeon. Then it hit me: Why not have the workers (the people who complete the tasks) help identify good domain names?

I decided that above all else, pronounceability and length were the two most important characteristics of a good domain name. If you have to spell out any portion of your domain name when telling it to somebody, it’s probably not a good name. Additionally, you want it to be as short as possible. After all, “thisisanawesomedomainnamebutyoudprobablyneverwantit.com” is easy to pronounce, but it’s hard to remember and even harder to type.

So, I went about creating a task (a Human Intelligence Task or HIT in Turk-speak) that would ask people how pronounceable thousands of different words were. The words were made up of domain names that I have already posted or plan to post on Domain Pigeon. I limited the domains to ones that were five or six letters long because alas, there are no available .com domain names less than five letters long.

Here’s a brief summary of what this entailed:

1. Create a template for the HIT:

After a worker completes a task, the person who created the task rates the workers’ performance. The better the workers do the higher their HIT approval rating will be. When you create a task, you can specify a minimum HIT approval rating which will determine the quality of the worker that can work on your task. Generally you want to keep it pretty high so you can reduce the number of garbage answers, but remember that the higher you make it, the less people who will be able to work on your task and the longer it will take to get completed. I lowered it to 0.80 to allow more participants.

As you’ll see in a second, I added 40 domain names per HIT. That means that when someone accepts one of these tasks, they have to rate the pronounceability of 40 words. I set a reward of 4 cents for this grueling task and I also said I wanted each HIT (each group of 40) to be rated 4 times. I wanted several people to rate each domain so I could then average the scores to get a more reliable rating.

2. Design the layout:

You can use their editor to create a layout, or if you’re familiar with HTML you can create your own. A basic knowledge of CSS is also helpful because, well, you should be nice to the workers and it doesn’t take much work to add a little style.

The ${domain1} format is a variable which will be replaced when I upload the data…

3. Upload the data:

When you’re happy with the price and the layout, it’s time to upload the data you want it to use.

For this task I wrote a few lines of code that would pull all of the available five and six letter domain names from my local database and output them as a Comma Separated Value (CSV) file where each row corresponds to the data used in a single HIT.

For this task, there were 40 columns and 590 rows for a total of 23,600 domains.

4. Publish and wait

This is what I see:

This is what the workers see:

5. Export results, analyze

When you’re done you can export the results, which you can them import into Excel and analyze to your heart’s (or rather to your ability’s) content.

Here’s what the analysis looked like for this this task:

The Result

You can now sort five and six letter domain names on Domain Pigeon by pronounceability. While not perfect, it does make it a lot easier to find the cream of the crop domain names on the site.

Guests: Check out the six letter domain names sorted by pronounceability
Members: Also check out the five letter domain names (Not a member? Sign up)

To celebrate the addition of this new feature to Domain Pigeon, we’re adding 1,000 five letter domain names per day for the entire week of 11 May, 2009 through 15 May, 2009.

Enjoy the domain names and happy hunting!

Your first time is always special:

I set a monthly budget of $250 and entered about 15 phrases I’d like Domain Pigeon to pop up for.

I do realize that it’s currently the third result for this search term and that paying for advertising on this page is probably unnecessary, but what the hell, let’s see what happens.

On February 8 I wrote the following:

Before I start adding more features I’m going to add credit card processing to the site. The percentage of people that follow through to Paypal to complete their registration is abysmal.

Well, I was lazy and abandoned it in favor of adding some shiny new feature.

After the CNet traffic two weeks ago I decided to take another look at the conversion rates.

To make a long story short: only 10% of the people were completing their purchases. Ouch. I think Paypal is difficult to use for people without Paypal accounts and I think most of them didn’t complete their purchases. There could have been usability problems on my end too, but I think it’s minimal in comparison.

I said enough is enough, I’m not doing anything else until I get this done. So, I spent the last two weeks adding credit card processing to the site. Hallelujah.

I have to thank Ryan Bates from Railscasts for his excellent videos on Active Merchant.  Without his tutorials this would have taken much, much longer.

Here’s what it looks like:

I’d also be remiss not to thank 37Signals, for it was the registration page for Backpack that was my inspriration for this design.

I’m thrilled that its done and that I can get back to adding new features.

P.S. This is a good read: The 5 Things I’d Tell My 21 Year Old Entrepreneurial Self.