Diary of an Emacs tragic

Category: WordPress

  • Using fail2ban to mitigate WordPress xmlrpc.php DDoS attacks

    Using fail2ban to mitigate WordPress xmlrpc.php DDoS attacks

    The other day my WordPress network went down. Upon investigation it turned out it was receiving a massive amount of http posts to the xmlrpc.php file. Apparently there is a WordPress DDoS that uses this mechanism. It brings apache and mysql to their knees as they can’t process the posts fast enough. If you search google for WordPress xmlrpc.php DDoS you can find lot more info about this.
    An temporary fix is to block all access to that file from your apache configs with something like:

    <Files xmlrpc.php>
  Order allow,deny
  Deny from all
</Files>

    That brought the load back to normal so I could at least access the WordPress backend.
    After googling around for a solution it appeared that fail2ban could help. Luckily there is a plugin for that. WP fail2ban has two parts. The first is a plugin that enables logging of xmlrpc events and authentication events to /var/log/auth. It is important to keep these events separate from the normal http access logs as the access log file can get very large and fail2ban can raise the load significantly just processing it.
    You also need to add a few configuration options to wp-config.php

    define('WP_FAIL2BAN_LOG_PINGBACKS',true);
# prevent user enumeration
define('WP_FAIL2BAN_BLOCK_USER_ENUMERATION',true);
# block some obviously invalid users
define('WP_FAIL2BAN_BLOCKED_USERS','^test

    See the readme for more details about what they do
    The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:

    ~# cp /var/www/wp-content/plugins/wp-fail2ban/wordpress.conf \
 /etc/fail2ban/filter.d
~#

    Then edit /etc/jail.local and insert:

    [wordpress]
enabled = true
filter = wordpress
logpath = /var/log/auth.log
# set the ban time to 1 hour - probably could be even higher for good measure
bantime = 3600
# needed for debian wheezy otherwise fail2ban doesn't start and reports
#   errors with the config
port = http,https

    Now restart fail2ban:

    ~# /etc/init.d/fail2ban restart
[ ok ] Restarting authentication failure monitor: fail2ban.
~#

    Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:

    2014-08-09 23:18:30,405 fail2ban.actions: WARNING [wordpress] Ban 117.195.37.14
2014-08-09 23:20:49,090 fail2ban.actions: WARNING [wordpress] Ban 78.97.220.237
2014-08-09 23:20:50,108 fail2ban.actions: WARNING [wordpress] Ban 46.108.226.105
2014-08-09 23:21:04,162 fail2ban.actions: WARNING [wordpress] Ban 120.28.140.93
2014-08-09 23:21:28,206 fail2ban.actions: WARNING [wordpress] Ban 175.142.187.77
2014-08-09 23:21:36,234 fail2ban.actions: WARNING [wordpress] Ban 88.240.97.76
2014-08-09 23:21:36,294 fail2ban.actions: WARNING [wordpress] Ban 122.177.229.110
2014-08-09 23:21:44,346 fail2ban.actions: WARNING [wordpress] Ban 89.106.102.15
2014-08-09 23:21:46,400 fail2ban.actions: WARNING [wordpress] Ban 2.122.219.188
2014-08-09 23:21:52,423 fail2ban.actions: WARNING [wordpress] Ban 95.69.53.13
2014-08-09 23:22:12,488 fail2ban.actions: WARNING [wordpress] Ban 5.12.12.66
2014-08-09 23:22:12,509 fail2ban.actions: WARNING [wordpress] Ban 182.182.89.23
2014-08-09 23:22:42,564 fail2ban.actions: WARNING [wordpress] Ban 178.36.126.249
2014-08-09 23:22:53,590 fail2ban.actions: WARNING [wordpress] Ban 36.83.125.10
2014-08-09 23:22:53,607 fail2ban.actions: WARNING [wordpress] Ban 95.231.59.185

    I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.

    define('WP_FAIL2BAN_BLOCKED_USERS','^organictrader');

    See the readme for more details about what they do
    The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:

    




    Then edit /etc/jail.local and insert:
    




    




    Now restart fail2ban:
    




    




    Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:
    




    




    I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.
    




    );
    




    See the readme for more details about what they do
    The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:
    




    




    Then edit /etc/jail.local and insert:
    




    




    Now restart fail2ban:
    




    




    Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:
    




    




    I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Facebook selecting wrong thumbnail for WordPress links
    
			
    Does Facebook keep selecting the wrong thumbnail for WordPress links for you? The solution is to give Facebook some extra instruction about what image to use for the thumbnail, using open graph.
    
If you use a static frontpage, it’s a simple matter of adding something like:
    

    
<meta property="og:image" content="http://samplesite.com/files/2014/05/web-thumb.png" />
    

    
to the Full meta tags of your front page.
    
You can check what facebook will do with your site by using their link debugger: https://developers.facebook.com/tools/debug
    
WordPress >= 3.9.1 seems to do the right thing for posts according to my testing.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    increase wordpress Maximum upload file size beyond 8MB.
    
			
    There is at least three places you need to set the maximum upload file size in WordPress.
    
First check your php.ini
    
upload_max_filesize = 64M;
    
post_max_size = 64M;
    
Restart Apache after making this change.
    
Then if you are using WordPress multi-site, you need change a setting in the backend. Visit My Sites -> Network Admin -> Dashboard -> Settings -> Network Settings. Then scroll down to the Upload Settings area and change that.
    
Thanks to dorr13 for that tip.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Setting alternate hreflang in WordPress
    
			
    UPDATE 2013-05-03:
    
After writing this I found out my approach was incorrect. I have since removed the code as it was not useful.
    
Dan’s team was kind enough to point me to Virgin Australia‘s site. If you inspect the source and search for hreflang, you’ll see how it should be done.
    
/UPDATE
    
After seeing the excellent talk by Dan Petrovic at the Melbourne WordCamp, I went to find out more about the <link rel=”alternate” hreflang=”en-AU” href=”http://somesite.com.au/” />. He has an article describing it. Essentially if you have multiple sites that have similar content (but not exactly the same) then you can instruct google that the sites serve different regions by using the rel=alternate hreflang=”en=gb” options for the link tag.
    
A common scenario that you might find this useful would be in online stores where there is a different store for each country.
    
eg. for site somesite.com.au you want a <link> tag like this:
    
<link rel=”alternate” hreflang=”en-AU” href=”http://somesite.com.au/” />
    
and for somesite.com you might want a tag like this:
    
<link rel=”alternate” hreflang=”en-US” href=”http://somesite.com/” />
    
and you want to remove the rel=”canonical” option too.
    
I couldn’t find a plugin that would implement this in WordPress so I wrote one and put it up on github. Yoast does not have an option for this as far as I can tell.
    
At some point I might put it up on wordpress.org too if it seems like there is interest in it.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    OAuth 2.0 in emacs – Part 1
    
			
    I want to write something in emacs to let me edit WordPress posts directly. There is of course the blogger-mode in emacs, but I’ve never managed to make that work. Then I noticed that JetPack in WordPress has a JSON interface. Supposedly it will let me do stuff to my blog via a REST interface.
    
What better way to learn something than try and learn 10 new things at once?! I mentioned my plan on #emacs and Nic Ferrier asked me to document the journey.
    
First thing is that your app needs to use oauth2 to authenticate. I tried the sample code given on wordpress.com. Before you can do that though, you need to log into wordpress.com and create an “app” and enable the JetPack JSON api module.
    

    <?php
 $curl = curl_init( "https://public-api.wordpress.com/oauth2/token" );
 $curl_setopt( $curl, CURLOPT_POST, true );
 $curl_setopt( $curl, CURLOPT_POSTFIELDS, array( 'client_id' =--> XXXX,
'redirect_uri' => 'https://emacstragic.net',
'client_secret' => biglongsecretstring,
'code' => $_GET['code'], // The code from the previous request
'grant_type' => 'authorization_code'
) );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1);
$auth = curl_exec( $curl );
$secret = json_decode($auth);
$access_key = $secret->access_token;
echo "auth: $auth";
echo "secret: $secret";
echo "access_key: $access_key";
?>

    

    This results in:
    

    
auth: {"error":"invalid_request","error_description":"The required \"code\" parameter is missing."}
    
Catchable fatal error: Object of class stdClass could not be converted to string in /Users/jason/Dropbox/projects/emacs-wordpress/test.php on line 17
    

    
not very helpful. I then decided to see whats available in emacs. And low and behold Julien Danjou has written an OAuth 2.0 library for emacs. It’s available in ELPA in emacs >= 24
    
I downloaded and installed it no problems but the documentation lacks any kind of working example that I can see.
    
My first attempt was to run the oauth2-request-authorization function, but it required strange parameters. (oauth2-request-authorization AUTH-URL CLIENT-ID &optional SCOPE STATE
    
REDIRECT-URI)     What the hell are scope and state?
    
Then looking through the source I found oauth2-request-access with a more promising (oauth2-request-access TOKEN-URL CLIENT-ID CLIENT-SECRET CODE
    
&optional REDIRECT-URI)
    
So I tried that: (oauth2-request-access "https://public-api.wordpress.com/oauth2/token" "XXXX" "longsecretcode" "CODE" "https://emacstragic.net" )
    
which returned [cl-struct-oauth2-token nil nil "XXXX" "longsecretcode" nil nil "https://public-api.wordpress.com/oauth2/token" ((error_description . "Invalid authorization_code.") (error . "invalid_grant"))]
    
That looks promising in a way. At least its doing something!
    
Stay tuned for part 2.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Piwik GPL web analytics
    
			
    Piwik is an open source (GPLed) web analytics program. It looks very comprehensive. I will endeavour to install it and test it out soon. I’ll report back on it when I do.
    
There is also a WordPress plugin for Piwik.
    
 
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Apache Virtual Host configuration for a Networked WordPress Installation
    
			
    Direct any URL request that Apache receives to the WordPress installation. You need to do it if  you are setting up a WordPress Network multi-site installation that has sites with their own unique domain names. e.g. site1.org, site2.com, someothersite.co.uk etc.
    

    /etc/apache2/sites-enabled$ ls -al
total 8
drwxr-xr-x 2 root root 4096 Jul 25 13:18 .
drwxr-xr-x 7 root root 4096 Jul 24 12:28 ..
lrwxrwxrwx 1 root root 40 Jul 24 12:06 000-wordpress-network-ssl -> ../sites-available/wordpress-network-ssl
lrwxrwxrwx 1 root root 36 Jul 24 12:02 010-wordpress-network -> ../sites-available/wordpress-network

    

    Order of the files is very important. wordpress-network contents below:
    

    <VirtualHost *:80>
UseCanonicalName Off
ServerAlias *.examplehost.com examplehost.com
ServerName examplehost.com
DocumentRoot /var/www
Options All
ServerAdmin myname@examplehost.com
# Store uploads of www.domain.com in /srv/www/wp-uploads/\
    <VirtualHost *:80>
UseCanonicalName Off
ServerAlias *.examplehost.com examplehost.com
ServerName examplehost.com
DocumentRoot /var/www
Options All
ServerAdmin myname@examplehost.com
# Store uploads of www.domain.com in /srv/www/wp-uploads/$0
RewriteEngine On
RewriteRule ^/wp-uploads/(.*)$ /var/www/wp-uploads/%{HTTP_HOST}/$1
# try and make server-status return server status
#RewriteRule ^/server-status - [L]
RewriteCond %{REQUEST_URI} !=/server-status
<Location /server-status>
SetHandler server-status
Order Deny,Allow
# Deny from all
# Allow from localhost
Allow from all
</Location>
<Directory />
Options FollowSymLinks
AllowOverride All
</Directory>
CustomLog /var/log/apache2/access.log vhost_combined
ErrorLog /var/log/apache2/error.log
# this is needed when activating multisite, WP needs to to a
# fopen("http://randomname.domain.com") to verify
# that apache is correctly configured
php_admin_flag allow_url_fopen on
</VirtualHost>

    
RewriteEngine On
RewriteRule ^/wp-uploads/(.*)$ /var/www/wp-uploads/%{HTTP_HOST}/\
# try and make server-status return server status
#RewriteRule ^/server-status - [L]
RewriteCond %{REQUEST_URI} !=/server-status
<Location /server-status>
SetHandler server-status
Order Deny,Allow
# Deny from all
# Allow from localhost
Allow from all
</Location>
<Directory />
Options FollowSymLinks
AllowOverride All
</Directory>
CustomLog /var/log/apache2/access.log vhost_combined
ErrorLog /var/log/apache2/error.log
# this is needed when activating multisite, WP needs to to a
# fopen("http://randomname.domain.com") to verify
# that apache is correctly configured
php_admin_flag allow_url_fopen on
</VirtualHost>

    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Migrating single site WordPress installations to a multisite network
    
			
    I have seen this question about migrating to a networked multisite WordPress install come up more than once so I thought I’d write down my experience. I used to host about 10 individual WordPress installations. Managing them and keeping them all up to date was a pain, and that meant I always had sites that were out of date. So a few months ago I spent some time migrating them all to a new WordPress network installation. That allows you to host multiple sites all in the one WordPress installation. The only problem is that its quite cumbersome to migrate sites into the new site.
    
I’ve tried to document the overall process. If anything isn’t clear please let me know and I’ll and elaborate.
    

      

    1. Backup all your old sites and databases
      2. 

    2. Go to each site and export the site
      3. 

    3. Select a suitable main domain name for your new networked WordPress site. When you create a new blog in the network, the admin of the blog will get an email from your multi site saying it was set up, so choose something that is suitable bearing that in mind.
      4. 

    4. Set up the multi site with subdomain (not sub folder).
      5. 

    5. Set up apache virtual hosts to direct ALL host traffic to the wp install.
      6. 

    6. Set up subdomain sites for each site you wish to import.
      7. 

    7. Install the WordPress MU Domain Mapping plugin
      8. 

    8. For each subdomain:

        

      1. Import the old blog
        2. 

      2. re-delegate the domain or set your hosts file up to point the domain name to your new WordPress network.
        3. 

      3. On the Info tab in your network settings for the domain, change the Domain to the actual domain you want,
        4. 

      4. On settings tab, change siteurl to the blogs correct domain,
        5. 

      5. Change “home” to the correct domain
        6. 

      6. Check through the list and fix up any other instances of incorrect domain. Save the changes.
        7. 

      7. Note down the site id (from the url. something like: http://mynewmultisite/wp-admin/network/site-settings.php?id=4  means site ID is 4)
        8. 

      8. Go to Settings/Domains and in the New Domain section, enter the site ID and correct domain for the new site
        9. 

      9. Click save and that’s it – the new domain should be working. Test it out.
        10. 

      10. Fix up all the old links to images. they will currently be links with the old subdomian in them. You can fix them up with the most excelent Search and Replace plugin.
        11. 

      

      9. 

    9. That’s it, you should be done.
      10. 

    

    It’s a fairly long winded process but it can be done and is worth the effort. Running a networked site instead of many individual installs saves a lot of time in upgrades and  maintenance.
    
Moving across the plugins and themes is also a bit of a pain. You just have to manually do it per site. There are plugins that are supposed to help move widget settings across but I had limited success with them. “Widget Data – Setting Import/Export Plugin” seemed to add the settings not replace them so you end up with duplicates of widgets and things. it may have improved since I tried it.
    
I don’t like to site wide activate plugins but WordPress does not have an option like with themes, to only enable a plugin on a per site basis. There is however a plugin for that: Multisite Plugin Manager. With that you can turn on plugins for each site in the Network Admin/Edit Site area. Then you have to go to the site’s dashboard and activate the plugin in there afterwards.
    
Another useful plugin for a networked site is called “Network Plugin Auditor” it adds a column to the Network/Admin/Plugins page that shows which sites are using each plugin. It also also shows the converse on the Sites page.
    
There are tools to assist this process also. One such tool that comes highly recommended, that I have not personally used is Backup Buddy. If you have more than a few sites to migrate it may well be worth investing in something like that to help you.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Ordered list with letters in Twenty Twelve
    
			
    UPDATE: It turns out I was wrong. type= is deprecated. The correct way to do this is style="list-style-type:lower-alpha". I updated my post to reflect that.
    
It appears there is a bug in Twenty Twelve as of this writing where specifying the type of an ordered list is not respected. So you can’t get an OL of letters. The second list below should have letters.
    

      

    1. first line
      2. 

    2. second line
      3. 

    

      

    1. first line
      2. 

    2. second line
      3. 

    

      
	
    1. first line
      2. 
	
    2. second line
      3. 

    

      
	
    1. first line
      2. 
	
    2. second line
      3. 

    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    A word on WordPress security
    
			
    First of all, I am not an expert on security. So please check this info for yourself, and don’t be shy to suggest improvements either.
    
In order to keep your WordPress site secure, one of the things you can do is slow down a potential brute force attack, so the attacker can try fewer login/password combinations per second.
    
There is an excellent plugin by convissor over at wordpress.org called Login Security Solution that does just that. I have been running it for a while now and it is very good. It emails you if your site is under attack and informs you about what its doing. It also ensures users have secure passwords. I have mentioned it before.
    
Recently I have had a few attacks and encountered a minor problem with the plugin. Daniel has been extremely proactive in tracking down the issue. But during the process I found that the attacks were coming from 3 IPs.
    
That made me think I should ban those IPs. And that led me to fail2ban. This is an excellent tool that monitors for failed login attempts and simply temporarily bans that IP using your firewall.
    
In debian you install it with aptitude install fail2ban. If you want to enable apache monitoring, you need to add:
    

    [apache]
enabled = true

    

    to /etc/fail2ban/jail.local
    
restart fail2ban and that’s it. Within minutes of me activating it, it had already banned a few IP addresses due to failed ssh login attempts.