############################### # # User-agent: * # # list folders robots are not allowed to index # Disallow: /images/ Disallow: /leeches/ Disallow: /cgi-bin/ Disallow: /aspnet_client/ Disallow: /oldsite/ Disallow: /stats/ Disallow: /cgi Disallow: /_private Disallow: /database # # ###############################