User-agent: Googlebot Allow:/ User-agent: Alexa (IA Archiver) Allow:/ User-agent: googlebot-image Allow:/ User-agent: googlebot-mobile Allow:/ User-agent: bingbot Allow:/ User-Agent: Mediapartners-Google Allow: / User-Agent: msnbot-media Allow: / User-agent: MSNBot Allow:/ User-agent: Slurp Allow:/ User-agent: Teoma Allow:/ User-agent: Scrubby Allow:/ User-Agent: Alexa (IA Archiver) Allow:/ User-agent: Robozilla Allow:/ User-agent: yahoo-mmcrawler Allow:/ User-Agent: Alexa (IA Archiver) Allow: / User-agent: psbot Allow:/ User-agent: Baiduspider Allow:/ User-agent: yahoo-blogs/v3.9 Allow:/ User-agent: Twitterbot Allow:/ User-agent: YandexBot Allow:/ User-agent: MJ12Bot Allow:/ Disallow: User-agent: * Disallow: / Disallow: /cgi-bin/ Disallow: /new/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /register/ Disallow: /login/ Disallow: /account/ Disallow: /wp-login.php?action=register Disallow: /admin-ajax.php User-agent: * Disallow: / User-agent: Crowsnest Disallow:/ User-agent: OpenHoseBot Disallow:/ User-agent: veoozbot Disallow:/