Sitemap: http://squeat.com/sitemap.xml # for the most part, we want bots to only hit /places/... User-agent: ShopWiki Disallow: / User-agent: * Crawl-delay: 53 Disallow: /parties/ Disallow: /people/ Disallow: /places/map/ # bots don't need maps Disallow: /images/ Disallow: /javascripts/ Disallow: /stylesheets/ Disallow: /login Disallow: /forgot_password Disallow: /register Disallow: /terms