Crawler User Agent Alexa-1 ia_archiver Alexa-2 ia_archiver-web.archive.org AskJeeves-Teoma Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html) Googlebot-2.1 Googlebot/2.1 (+http://www.google.com/bot.html) Googlebot-Mozilla-2.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Google-AdSense-2.1 Mediapartners-Google/2.1 MSN-1.0 msnbot/1.0 (+http://search.msn.com/msnbot.htm) Yahoo-Slurp Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) ZyBorg-1.0 Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.WISEnutbot.com)
Search Engine Optimization, SEO Articles, SEO tips, Latest SEO News, SEO Updates, SEO Blog.