> > どっかのサイトのソースのコピペ
> > 
> > <META NAME="ROBOTS" CONTENT="NOINDEX">
> > <META NAME="ROBOTS" CONTENT="NOFOLLOW">
> > <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
> > <META NAME="ROBOTS" CONTENT="NONE">
> > <META name="robot-control" content="deny-all">
> > <META name="robot-control" content="deny-quote">
> > <META name="robot-control" content="deny-analysis">
> なるほど、nofollowとnoneも噛ませるのか
> よくわからんがrobot-controlとやらも使ってみよう
> でもこれがgoogleに効くかどうかが実際あやしいんだけどな
> よくわからん


Why is Googlebot asking for a file called robots.txt which isn't on my server?

Robots.txt is a standard document that can tell Googlebot not to download some or all information from
your web server. For information on how to create a robots.txt file, see The Robot Exclusion Standard. 

http://www.google.com/faq.html

参考：2000/09/19(火)00時14分02秒