Wednesday, February 28, 2007

Goolgle site itself is not robot friendly

Just checked
google site search engine setting

It is 1st search engine in the world, but it only allow other agent to read from news.
Really Funny!!!

User-agent: *
Allow: /searchhistory/
Disallow: /news?output=xhtml&
Allow: /news?output=xhtml
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Disallow: /catalogues
Disallow: /news
Disallow: /nwshp

...

No comments: