If you’ve visited these pages a few times, you may have noticed that there keep being new comments on the Make your own Virus and Religion sucks posts – and lately as on the iTunes Problems post as well. People seem to Google for such things and are keen to comment on them as well. Most of them seem to be idiots as well. Or, as more positive people have said, a veritable subculture has developed in that area.
A subculture that is in no way enlightening or relevant and that started consuming considerable amounts of my bandwidth. It’s not that this will leave starving at the end of the month because of the bandwidth bill – we’re talking about a Euro a month – but it’s still wrong. I shouldn’t be paying for anybody’s idiocy but my own. So, I was interested to follow a clever plan:
If Google’s inappropriate search results dump all those people on my site, why not use their ad program (just on those pages, not the rest of the site of course) to pay for the costs? No pain for me, hardly any pain for the people visiting those pages, and probably no pain for the advertisers either as I’d imagine there to be anti-virus and pro-religion ads on the pages, which a fraction of the errand visitors might find useful and the rest would consider offensive. I wouldn’t care. Brilliant idea!
So I submitted the site on some form with Google and now I received a replay that I can’t use that tool of theirs. Why?
Under construction is the reasion they give. Allegedly they require sites
to be launched and navigable to ‘qualify’ for their program. But I consider my pages to be just that (not that I know what ‘launched’ means, I mean most of Google’s own sites seem to be ‘beta’ at best…). So all this is rather annoying. In a way that we don’t expect from Google. What they do tends to be correct and well thought out rather than random and incomprehensible. I guess that day had to come…
And I’ll just add a line or two to my robots.txt file. Which should solve the problem.
Gee… religion… viruses… iTunes… all you need now is to write an entry about lesbian porn, and I think you’ll have corned the Google search results market! :-)
As a (the only?) subscriber to your comments RSS feed, I get all of these stupid comments on Religion and Viruses and yes, iTunes too. (Not to mention “I wanna die”.)
I was gonna point the problem out to you, but figured you’d be aware of it anyway. A common solution is to have your weblog software close comments to entries after a specified amount of time, such as a month or two. You might not want that, of course. Another is robots.txt, as you already said, but I’d personally rather not do that — I like people to find my blog, and often post something with the specific intent for people to discover it (when I find a long-sought solution to a common problem, for example — see also http://daringfireball.net/2004/05/writing_for_google ).
Finally, you could do some referer magic to see if people are coming straight from a search magic, and if so, remove this very form I am typing them, so they can still read comments but need to actually go to your site directly to post one on their own (you’d want to point this out, of course).
Fun sidenote: I’ve set up the server to simply block all requests that come in with ‘porn’ or ‘casino’ or similar referrer-spam terms in their referrer, so you may have to hit reload to actually see the page when coming from Google.
Sören: I was only planning to exclude those specific pages from Google. The rest of the site will stay in the index. I don’t think the world will lose any particularly useful information that way. As you guessed, I don’t like closing comments. Mainly because I’m always upset when finding comments closed on other sites myself.
Received data seems to be invalid. The wanted file does probably not exist or the guys at last.fm changed something.