Second, let’s get the most common complaint about OpenDNS – one that isn’t going to be discussed here any further – out of the way: Their practice of returning ads on blocked or non-existent sites in your browser, via a bogus A RR of 67.215.65.132 (if you don’t go with one of their paid options). OpenDNS is upfront about doing this, so you can decide if the trade-off is worthwhile before you sign up – and you can quit using them any time you want.

Those two preliminaries covered, here’s a case study of what I think is a serious problem with OpenDNS, plus some thoughts on how they could fix it.

Background: Recently, I helped my wife buy a domain for a personal blog; it turns out OpenDNS has tagged this domain as pornographic. Currently, the domain hosts nothing beyond an empty WordPress blog.

I can think of three possible scenarios that could be in effect here:

1. The site has been compromised. This was my first thought, although I now consider it unlikely. I’ve gone through the site and the logs pretty closely, and have been unable to find anything amiss. I’m willing to leave the possibility open that I missed something, though.
2. The domain used to be pornographic, but is no longer. This also appears unlikely, at least based on archive.org and other sites; the domain name itself also doesn’t exactly suggest a porn site, either, unless there’s some obscure slang I’m unaware of.
3. The OpenDNS classification is incorrect. The domain was accidentally or intentionally mislabeled. It appears most classification – especially for small sites like the one in question – is crowd-sourced, but only a few members of the “crowd” might tag a small site.

It’s unclear which of the above went wrong, but there are several obvious ideas that OpenDNS could implement to address this class of problems:

1. More metadata, please. For my “pornographic” domain, it would have been nice to know when it was tagged as such, what the DNS servers and relevant records were when it was tagged, what a screenshot looked like then (not now), and what the registrar data was. If it was a site compromise, this data would make my finding it that much easier. If the classification is stale, metadata would make this obvious as well.
2. Automatically-triggered classification reviews. If there are substantial changes to a domain’s registrar entry, that should trigger an automatic review of the site – especially so if the domain registration lapses, and another party purchases the domain after a period of time. The volume of users whitelisting a site in the control panel should be another signal of misclassification, although this probably won’t help obscure sites. This is an obvious, simple fix, and perhaps OpenDNS has something like this in place, but I don’t see any signs of it.
3. A better user interface. When I first visited their tagging interface, the thumbnail viewer was broken, and the “flag for review” link was missing. Both later reappeared, apparently functional, although I have no idea if or when a review will actually take place.
4. Better policing of community members. Some of the OpenDNS Community’s most prolific members tag in excess of 1,000 domains on an average day. Many of these tags appear to be of dubious quality, going far beyond anything that could be called a difference of opinion into the realm of flat-out wrong. Poor quality – or malicious – tagging is damaging OpenDNS’s product; I’m surprised they don’t appear more strongly motivated to address this.

For now, I’m going to continue to use OpenDNS’s services at home, although I’m on the lookout for a better product elsewhere. But until I see some signs that they’re addressing the issue of incorrect tags, I can no longer recommend them for professional use, which is too bad; as I said in the first paragraph, there’s a lot to like about OpenDNS.

Update, 1/2/2012: It’s been a couple weeks since I flagged the domain for OpenDNS’s review – and nothing has happened. I think it’s fair to update my conclusion: OpenDNS has a garbage-in, garbage-out problem and does not appear to be invested in the quality of their product; look elsewhere for a content-filtering tool that goes beyond hobbyist quality.

(See bottom of post for an update running similar tests on OpenDNS.)

Methods: I searched Google for keywords that I believed fell somewhere between obscure and common and collected the first ten hostnames printed on the screen. I then used local installations of dig to query a collection of DNS servers for the hostnames’ A records and collected the response times. The different resolvers used were:

• A local BIND installation (127.0.0.1, cache empty) with Comcast Internet connectivity;
• A Comcast DNS server (68.87.69.150) via Comcast Internet connectivity;
• My employer’s internal caching DNS;
• Google (8.8.8.8) via my employer’s Internet connectivity (mostly Level 3);
• Google (8.8.8.8) via Comcast; and
• Google (8.8.8.8) via an Amazon EC2 instance in us-east-1a.

Anticipating a bimodal distribution of results, I assumed high latency responses were cache misses, while low latency responses were cache hits, and categorized results correspondingly.

Limitations: Chiefly, the small number of hostnames queried. Results from a larger group of domains would be more conclusive.

Results: Given in the format of Server/Connectivity: Cache Miss/Cache Hit

• Local BIND server/Comcast: 319ms/0ms
• Comcast/Comcast: 166ms/14ms
• Google/Comcast: no misses/73ms
• Employer/Level 3: 235ms/30ms
• Google/Level 3: 204ms/44ms
• Google/EC2: 190ms/4ms

I concluded that Google/Comcast had no misses by testing another set of obscure hostnames twice each, noting that the first query was slower (~120ms) and the second query was similar in latency to the results above (70ms). (My belief is that I inadvertently pre-populated the cache for Google/Comcast by my tests elsewhere.)

Discussion:

• It’s all about cache hits. Whichever resolver gives you the most cache hits will give you the best performance; cache misses are at least an order of magnitude slower than cache hits. In this extremely limited test, the cache-hits-champion appears to be Google. Excluding Google/Comcast, where I believe I pre-populated Google’s cache, Google had a 50% cache hit rate, while Comcast and my Employer only hit 20%.
• Location, location, location. Secondary to cache hits, the closer the resolver is to you, the better. Looking at the Comcast results, it’s hard to get closer than localhost, and, as seems logical, Comcast’s resolvers have lower cached latency than Google’s. Running a local caching resolver forwarding to Google may be a desirable configuration.
• Resolver behavior matters. Comcast is notorious for poor behavior. It’s reasonable to expect that Google will be mining your DNS query data. Running a slower but directly-controlled local, non-forwarding server may be preferable for privacy and security reasons.

Update: At the suggestion of @tscalzott, I researched OpenDNS’s performance with the same set of hostnames via the same connectivity on their DNS resolvers at 208.67.222.222. This time, however, I queried the A record for each hostname twice in rapid succession to ascertain how many of my queries were served from OpenDNS’s cache. Results are in the format:

DNS Server/Connectivity: Cache miss/Cache hit – cache hit rate

• OpenDNS/Comcast: 218ms/30ms – 20%
• OpenDNS/EC2: 144ms/2ms – 10%
• OpenDNS/Level 3: 230ms/4ms – 40%

Compared to Google, OpenDNS had similar latency for cache misses and lower latency for cache hits, but appears to possibly have a lower cache hit rate. It seems likely that the latency “winner” for each user’s individual situation will depend on where they are on the Internet relative to the nearest Google and OpenDNS installations. Google’s greater cache hit rate suggests it may offer better service, but testing a larger number of hostnames would be necessary before being able to state that with any certainty.

Disclosure: I use Google’s free Apps services to host personal email, and I use their public sites (Search, Reader, News, Analytics, etc.) extensively. I recently attended a Google Apps for the Enterprise dog-and-pony show where I received a number of small tchotchkes; my wife took the notebook, I kept the pen and binder and threw the rest away. My employer uses Postini. I tried OpenDNS briefly several months back, but did not use them long-term because of limitations in my own configuration.