# /robots.txt file for http://vasb.fi/ # mail webmaster@vasb.fi for constructive criticism # Block MSNBot: Crawls too often & uses too much bandwidth User-agent: msnbot Disallow: / # Often spam bots: User-agent: Java Disallow: / # Exclude some locations User-agent: * Disallow: /tmp/ Disallow: /backstage/ Disallow: /awstats/ Disallow: /cgi-bin/