Using fail2ban to mitigate WordPress xmlrpc.php DDoS attacks

The other day my WordPress network went down. Upon investigation it turned out it was receiving a massive amount of http posts┬áto the xmlrpc.php file. Apparently there is a WordPress DDoS that uses this mechanism. It brings apache and mysql to their knees as they can’t process the posts fast enough. If you search google for WordPress xmlrpc.php DDoS you can find lot more info about this.

An temporary fix is to block all access to that file from your apache configs with something like:

<Files xmlrpc.php>
  Order allow,deny
  Deny from all
</Files>

That brought the load back to normal so I could at least access the WordPress backend.

After googling around for a solution it appeared that fail2ban could help. Luckily there is a plugin for that. WP fail2ban has two parts. The first is a plugin that enables logging of xmlrpc events and authentication events to /var/log/auth. It is important to keep these events separate from the normal http access logs as the access log file can get very large and fail2ban can raise the load significantly just processing it.

You also need to add a few configuration options to wp-config.php

define('WP_FAIL2BAN_LOG_PINGBACKS',true);
# prevent user enumeration
define('WP_FAIL2BAN_BLOCK_USER_ENUMERATION',true);
# block some obviously invalid users
define('WP_FAIL2BAN_BLOCKED_USERS','^test$');
define('WP_FAIL2BAN_BLOCKED_USERS','^organictrader$');

See the readme for more details about what they do

The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:

~# cp /var/www/wp-content/plugins/wp-fail2ban/wordpress.conf \
 /etc/fail2ban/filter.d
~#

Then edit /etc/jail.local and insert:

[wordpress]
enabled = true
filter = wordpress
logpath = /var/log/auth.log

# set the ban time to 1 hour - probably could be even higher for good measure
bantime = 3600

# needed for debian wheezy otherwise fail2ban doesn't start and reports
#   errors with the config
port = http,https

Now restart fail2ban:

~# /etc/init.d/fail2ban restart
[ ok ] Restarting authentication failure monitor: fail2ban.
~# 

Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:

2014-08-09 23:18:30,405 fail2ban.actions: WARNING [wordpress] Ban 117.195.37.14
2014-08-09 23:20:49,090 fail2ban.actions: WARNING [wordpress] Ban 78.97.220.237
2014-08-09 23:20:50,108 fail2ban.actions: WARNING [wordpress] Ban 46.108.226.105
2014-08-09 23:21:04,162 fail2ban.actions: WARNING [wordpress] Ban 120.28.140.93
2014-08-09 23:21:28,206 fail2ban.actions: WARNING [wordpress] Ban 175.142.187.77
2014-08-09 23:21:36,234 fail2ban.actions: WARNING [wordpress] Ban 88.240.97.76
2014-08-09 23:21:36,294 fail2ban.actions: WARNING [wordpress] Ban 122.177.229.110
2014-08-09 23:21:44,346 fail2ban.actions: WARNING [wordpress] Ban 89.106.102.15
2014-08-09 23:21:46,400 fail2ban.actions: WARNING [wordpress] Ban 2.122.219.188
2014-08-09 23:21:52,423 fail2ban.actions: WARNING [wordpress] Ban 95.69.53.13
2014-08-09 23:22:12,488 fail2ban.actions: WARNING [wordpress] Ban 5.12.12.66
2014-08-09 23:22:12,509 fail2ban.actions: WARNING [wordpress] Ban 182.182.89.23
2014-08-09 23:22:42,564 fail2ban.actions: WARNING [wordpress] Ban 178.36.126.249
2014-08-09 23:22:53,590 fail2ban.actions: WARNING [wordpress] Ban 36.83.125.10
2014-08-09 23:22:53,607 fail2ban.actions: WARNING [wordpress] Ban 95.231.59.185

I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.