Question: If I make a file with the content below, what shouldI name it to keep spiders away from my website? # Hits many times per

If I make a file with the content below, what shouldI name it to keep spiders away from my website? # Hits many times per second, not acceptab # http://www.nameprotect.com/botinfo.html User-agent: NPBot DisalLow: / # A capture bot, downloads gazillions of p ages with no public benefit # http://www.webreaper.net/ User-agent: WebReaper Disallow: / # Wayback Machine : defaults and whether to index user-pages # FIXME: Complete the remova of this bloc k, per T7582. # User-agent: archive.org.bot # Allow: /
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
