Thursday, May 13, 2010

View a Web Page as 'Googlebot'

Crawler

User Agent

Alexa-1

ia_archiver

Alexa-2

ia_archiver-web.archive.org

AskJeeves-Teoma

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)

Googlebot-2.1

Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot-Mozilla-2.1

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Google-AdSense-2.1

Mediapartners-Google/2.1

MSN-1.0

msnbot/1.0 (+http://search.msn.com/msnbot.htm)

Yahoo-Slurp

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

ZyBorg-1.0

Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.WISEnutbot.com)

No comments: