ModSec Taylor Made and Tweaks by Sergio

The use of any of this rules are under your own risk,

we don't assume any responsability.

User-Agent block rule:

SecRule REQUEST_HEADERS:User-Agent "@pmFromFile PuntaPirata-blackbots.txt"

"id:980001,rev:1,severity:2,log,msg:'PuntaPirata Bot Rule: Black Bot detected. '"

How to use it:

Create a TXT file called "PuntaPirata-blackbots.txt", fill in them any bad User-Agent that you like.
Here is a file that you can use to start working with: PuntaPirata-blackbots.zip
unzip the file before you can use it and check the definitions inside to modify accordingly.
To block any of the bad bots defined on the TXT file, set this rule as one of you first rules in ModSec.

How this works:

There are a lot of bad bots that crawls web pages to gather sensitive information that is written on them, this crawls doesn't respect the robot.txt file in a site and are a security risk.

A lot of sites on the web recommends to use .HTACCESS to block this bots but this only protects one directory or site, so, this rule is far better as you can block bad bots server wide.

Tweaks:

When you finish to check that the rule is working for you, we recommend that you turn off the LOG option in the rule changing "LOG" to "NOLOG", that way you will be not bothered with hundred of emails of IPs blocked by ModSecurity.

July 19th, 2010 - Sergio Cabrera