Quote Originally Posted by Nomadic View Post
Bad bots tend to ignore the robots.txt file. The best way to block bad bots is to IP ban them. If however it is a bunch of copies on different IPs you might have to contact your host about setting a limit on how many pages can be accessed by one IP in a given time period (bad bots like to spam page requests really fast).
Very true. I was going to mention about bad bots ignoring the robots.txt file but I figured I covered the basics and didn't want to get bogged down by technicalities. One can use the htaccess file to ban bots, IP addresses, and hot-linking (to name the top three) but that is probably getting a bit too technical for here.