# All robots will spider the domain User-agent: * Disallow: # Disallow directory admin User-agent: * Disallow: admin # Disallow directory admin2 User-agent: * Disallow: admin2 # Disallow directory db User-agent: * Disallow: db