A look at my logs recently led me down a few investigative Web rambles last night and today – the kind where you start of at Google, click a few links, go back, click a few more links farther down and so on.
Interesing and informative links found thus far – attention, my hubby, might have a little more tinkering around for you to do:
IMHO, the above site was actually specific and relevant. 😉
Helpfully provided by Random and Irrelevant: robots.txt not honored?
And I think I need to cross-reference robots.txt abusers/ignorers against a few such lists as this
There’s also a wealth of info at WebmasterWorld to check out in general.
