Diary of an Emacs tragic

Category: Uncategorized

  • BPay View feedback

    Dear CommBank,
    While I think NetBank is great, the BPayView part seriously lets it down. I tweeted my feelings about this and you asked for feedback, so here it is:
    1. It silently lets the biller prevent you from viewing older bills. eg. I cannot view Sydney Water bills older than 6 months. but you don’t know this until you try to view the bill, you seen the “This bill is unable to be opened at this time.” and then you call tech support and they inform you that Sydney Water prevents bills older than 6 months from being viewed.
    2. It auto archives bills after 18 months. This should be at least 2 years if not 7 in my opinion
    3. if you open a bill and try to save the PDF, the PDF is not named something sensible so you can quickly save it and move on to the next bill, making saving bills a tiresome slow process.
    4. There should be an option to email the bills to you as a PDF as and when they become available.
    5. The archived bills view is useless a it does not show the date of the bill or anything useful about the bill except the amount. This is maddeningly annoying.
    6. You cannot open archived bills. well this wouldn’t be such a problem if points 2,4 and 5 were taken care of.

  • Letter to the minister for small business about slow Internet in Australia

    Today I wrote to the minister for small business complaining about the crap Internet in this country. I urge you to write to your MPs also if you think the lack of NBN is a problem.

    Dear Mr Billson,
    I’d just like to express my concern over the woeful state of Internet in Australia.
    As a small business owner, I’m finding it prohibitively expensive to get a decent Internet connection for my business and since the government scuttled the NBN its looking increasingly like fast Internet is basically only for big businesses or people with a lot of money.
    Please can you do something to get cheap fast Internet for small business?
    I did write to Mr Turnbull about this and the best he could come up with was to blame Labor. I’m sick of hearing excuses. I need action in order for my business to remain viable and continue providing employment to Australian Tax Payers. Fast and cheap Internet is part of what I need to do that.
    Thanks for listening

  • I wrote to Turnbull to complain about the lack of NBN. Here is his reply

    After traveling to New Zealand for #lca2015 and experiencing fibre to the home at my friends house, I decided to write to Malcolm Turnbull to express my concern about the slow rollout of the NBN in Australia.
    He actually personally replied directly to my email only 4 hours later. In the interest of openness I am putting the whole email below. I have rearranged it to fix the topposting.
    The tl;dr is “It’s all Labor’s fault”
    My email to Mal. (I think we can say we are on first name terms now)

    From: Jason Lewis
    Sent: Tuesday, 20 January 2015 1:52 PM
    To: Turnbull, Malcolm (MP)
    Subject: NBN Roll-out
    Dear Mr Turnbull,
    I’m writing to express my concern with the woefully slow roll-out of the
    NBN.
    I’ve recently been travelling in other countries where fibre to the home
    is already commonplace and it makes the Australian broadband offering
    feel terribly inadequate.
    I feel this will lead to a further reduction in Australia’s
    international competitiveness.
    Please devote more resources to speeding up the roll-out.
    Thanks,
    Jason

    And his response:

    Dear Jason,
    Thanks for your candid comments.
    Unfortunately Labor significantly underestimated the cost and complexity of this project and, as a result, released rollout schedules that were unrealistic and inaccurate.
    For instance, Labor originally forecast that 2.7 million houses would be passed by fibre by 30 June 2014. In NBN Co’s last Corporate Plan released in August 2012, it was revised to 1.3 million houses passed by fibre. The comprehensive Strategic Review completed in December 2013 however, found that the NBN would only pass 467,000 houses with fibre by 30 June 2014. The actual number reached by 1 July 2014 was 492,000 premises – less than one-fifth of the original target.
    I highlighted the problems which Labor created with their false rollout expectations in a recent blog available here: www.malcolmturnbull.com.au/media/trying-to-make-sense-of-the-confused-and-confusing-mr-clare1
    In addition, the Government recently undertook a Broadband Availability and Quality Report, which found that there are more than 1.6 million premises across Australia with very poor or no fixed broadband access at all. However, Labor made no effort to prioritise these areas in their rollout.
    The Government has instructed NBN Co to revise its current rollout schedule to meet three key objectives:

    1. To ensure that the underserved areas in Australia are prioritised in the NBN Co rollout. On average, areas with very poor broadband will receive upgrades two years sooner.
    2. To ensure that the NBN upgrades are delivered sooner and more affordably, by using a mix of technologies. The Strategic Review found that under the model adopted by the Coalition, the project will be finished four years sooner than would have otherwise been the case.
    3. To ensure that information provided in the public domain is accurate and can be relied upon by businesses and households waiting for broadband upgrades.

    The fact is that the NBN, up to the election, had reached only three per cent of Australian premises after four years and $6.4 billion of funding.
    NBN Co is now determining which technologies are most cost effective and should be utilised on an area-by-area basis so as to minimise peak funding, optimise economic returns and deliver broadband upgrades sooner. This is highly demanding and complex work which involves renegotiating deals with Telstra and Optus to take over portions of their fixed line networks. Naturally this is taking time to complete.
    In the meantime, NBN Co has continued to expand its fibre network in areas where construction contracts had been signed at the time of the election. Across the country, more than 500,000 premises have been passed by NBN fibre and work is underway to extend the network to reach a further 600,000 premises.
    Sites that are not currently represented on the NBN Co maps are being reviewed in line with rollout priorities. Importantly, the network will continue to be built on a state-of-the-art platform, but will use existing infrastructure where it makes sense to do so. In areas where work has not commenced, NBN Co will be making decisions about construction on the basis of review findings. NBN Co is significantly advanced in planning the multi-technology mix and rollout plans will be publicly released once they are completed. More information about the NBN rollout will be published by NBN Co on its website www.nbnco.com.au when it is available.
    You may also be interested to know that the Government has secured the progressive transfer of the necessary parts of Telstra’s copper and HFC (pay TV) cable networks to NBN Co at no additional expense. Telstra’s 1.4 million shareholders have been ‘kept whole’ in keeping with the Government’s pre-election commitment.
    These agreements are a key milestone in shifting the NBN to the Multi Technology Mix the company has determined is its optimal strategy. The December 2013 Strategic Review found the shift to a multi-technology NBN will reduce its cost by $30 billion, and save at least four years in construction time.
    Importantly, under these agreements, NBN Co will make use of Telstra’s copper and HFC networks (i.e. the network used to deliver pay television) rather than decommissioning and wasting these assets, as Labor had planned.
    Yours sincerely,
    Malcolm Turnbull

  • #lca2015: Using FOI to get source code

    Michael Cordover has spent significant time and money chasing down the code used for counting Australian Election results. He goes into the reasons why that is important. Another great talk from #lca2015.

  • I just donated to Wikipedia

    After reading a blog post from someone who donated to Wikipedia because they use it a lot, I realised I use it a lot too, so I decided to donate too. Unfortunately I can’t quite remember who’s post it was, but thank you whoever you are!
    I donated $30. You should too if you like Wikipedia, and you use it.

  • Apache [warn] RSA server certificate CommonName (CN) does NOT match server name!?

    Apache was giving me lots of errors like this on my host that has multiple virtual hosts on it.

    [Mon Nov 10 13:54:01 2014] [warn] RSA server certificate CommonName (CN) `www.xxx.com.au' does NOT match server name!?
[Mon Nov 10 13:54:01 2014] [warn] RSA server certificate CommonName (CN) `www.yyy.com.au' does NOT match server name!?
[Mon Nov 10 13:54:01 2014] [warn] RSA server certificate CommonName (CN) `www.zzz.co.uk' does NOT match server name!?
[Mon Nov 10 13:54:01 2014] [warn] RSA server certificate CommonName (CN) `www.cocolo.com.au' does NOT match server name!?
[Mon Nov 10 13:54:01 2014] [warn] RSA server certificate CommonName (CN) `www.aaa.com.au' does NOT match server name!?
[Mon Nov 10 13:54:01 2014] [notice] Apache/2.2.22 (Debian) PHP/5.4.4-14+deb7u14 mod_ssl/2.2.22 OpenSSL/1.0.1e configured -- resuming normal operations

    After a little investigation it turned out the CN in the certificate was set to www.domain but the vhost’s ServerName was set to www-less.
    The solution turned out to be to ensure the vhost ServerName == the CN in the certificate. You can examine certificate with:

    certtool -i < domain.cert

    And in the vhost, make sure there is a ServerAlias for the www-less version:

    ServerName www.domainname.com.au
ServerAlias domainname.com.au
.
.
.

    a few less warnings in the logs now. Phew!

  • Using fail2ban to mitigate WordPress xmlrpc.php DDoS attacks

    Using fail2ban to mitigate WordPress xmlrpc.php DDoS attacks

    The other day my WordPress network went down. Upon investigation it turned out it was receiving a massive amount of http posts to the xmlrpc.php file. Apparently there is a WordPress DDoS that uses this mechanism. It brings apache and mysql to their knees as they can’t process the posts fast enough. If you search google for WordPress xmlrpc.php DDoS you can find lot more info about this.
    An temporary fix is to block all access to that file from your apache configs with something like:

    <Files xmlrpc.php>
  Order allow,deny
  Deny from all
</Files>

    That brought the load back to normal so I could at least access the WordPress backend.
    After googling around for a solution it appeared that fail2ban could help. Luckily there is a plugin for that. WP fail2ban has two parts. The first is a plugin that enables logging of xmlrpc events and authentication events to /var/log/auth. It is important to keep these events separate from the normal http access logs as the access log file can get very large and fail2ban can raise the load significantly just processing it.
    You also need to add a few configuration options to wp-config.php

    define('WP_FAIL2BAN_LOG_PINGBACKS',true);
# prevent user enumeration
define('WP_FAIL2BAN_BLOCK_USER_ENUMERATION',true);
# block some obviously invalid users
define('WP_FAIL2BAN_BLOCKED_USERS','^test

    See the readme for more details about what they do
    The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:

    ~# cp /var/www/wp-content/plugins/wp-fail2ban/wordpress.conf \
 /etc/fail2ban/filter.d
~#

    Then edit /etc/jail.local and insert:

    [wordpress]
enabled = true
filter = wordpress
logpath = /var/log/auth.log
# set the ban time to 1 hour - probably could be even higher for good measure
bantime = 3600
# needed for debian wheezy otherwise fail2ban doesn't start and reports
#   errors with the config
port = http,https

    Now restart fail2ban:

    ~# /etc/init.d/fail2ban restart
[ ok ] Restarting authentication failure monitor: fail2ban.
~#

    Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:

    2014-08-09 23:18:30,405 fail2ban.actions: WARNING [wordpress] Ban 117.195.37.14
2014-08-09 23:20:49,090 fail2ban.actions: WARNING [wordpress] Ban 78.97.220.237
2014-08-09 23:20:50,108 fail2ban.actions: WARNING [wordpress] Ban 46.108.226.105
2014-08-09 23:21:04,162 fail2ban.actions: WARNING [wordpress] Ban 120.28.140.93
2014-08-09 23:21:28,206 fail2ban.actions: WARNING [wordpress] Ban 175.142.187.77
2014-08-09 23:21:36,234 fail2ban.actions: WARNING [wordpress] Ban 88.240.97.76
2014-08-09 23:21:36,294 fail2ban.actions: WARNING [wordpress] Ban 122.177.229.110
2014-08-09 23:21:44,346 fail2ban.actions: WARNING [wordpress] Ban 89.106.102.15
2014-08-09 23:21:46,400 fail2ban.actions: WARNING [wordpress] Ban 2.122.219.188
2014-08-09 23:21:52,423 fail2ban.actions: WARNING [wordpress] Ban 95.69.53.13
2014-08-09 23:22:12,488 fail2ban.actions: WARNING [wordpress] Ban 5.12.12.66
2014-08-09 23:22:12,509 fail2ban.actions: WARNING [wordpress] Ban 182.182.89.23
2014-08-09 23:22:42,564 fail2ban.actions: WARNING [wordpress] Ban 178.36.126.249
2014-08-09 23:22:53,590 fail2ban.actions: WARNING [wordpress] Ban 36.83.125.10
2014-08-09 23:22:53,607 fail2ban.actions: WARNING [wordpress] Ban 95.231.59.185

    I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.

    define('WP_FAIL2BAN_BLOCKED_USERS','^organictrader');

    See the readme for more details about what they do
    The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:

    




    Then edit /etc/jail.local and insert:
    




    




    Now restart fail2ban:
    




    




    Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:
    




    




    I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.
    




    );
    




    See the readme for more details about what they do
    The second part is enabling filters and gaols in fail2ban. Luckily this is also provided by the WP fail2ban plugin. Copy the wordpress.conf file from the wp-fail2ban directory to the fail2ban config directory:
    




    




    Then edit /etc/jail.local and insert:
    




    




    Now restart fail2ban:
    




    




    Remove the block on the xmlrpc.php file from your apache config and restart apache. Then you should see in your fail2ban logs something like:
    




    




    I found however that I was being hit from over 1800 unique IP addresses and despite fail2ban successfully banning them, it was taking too long to ban enough that the load would return to normal so I re-blocked the xmlrpc.php file for 24 hours. After that, I enabled it and it seemed as though the DDoS had gone away. So far so good.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
    Howto quickly find your Beaglebone Black’s IP address
    
			
    Howto quickly find your Beaglebone Black’s IP address
    
			
    Whenever I connect my Beaglebone Black (BBB) to a network, I have to work out it’s IP address so I can ssh into it. This can be tricky. Some of your options are:
    

      

    1. connect to the serial terminal, or connect over the usb network interface which gives the BBB the address 192.168.7.2, log in and issue the command ip addr.
      2. 

    2. use nmap to try and search out its IP address on your subnet but this is time consuming and not very accurate I have found.
      3. 

    3. use avahi-browse -rat (thanks Madox for that tip.)
      4. 

    

    Last night I came up with a Better Way™. Rather than trying to determine the BBB’s address, why not use a fully qualified domain name and a dynamic dns service? I could then just type ssh myfqdn.com or whatever and log in. Think how that would simplify one’s life!
    
To implement this, set up a dynamic DNS somewhere with a FQDN for your BBB. If you happen to have your own domain name already you can use a sub-domain from that. I think its fairly common for DNS hosts to offer an API to update your IP address. I happen to use Rimu Hosting and they have their own simple web api.
    
Then you just need to write a little script to update the IP address every time the DHCP client receives a new IP address, and drop it into /etc/dhcp/dhclient-exit-hooks.d/
    
Here is my script. This will only work with Rimu Hosting as they have their own privately developed API, and you’d need to insert your own KEY into the script.
    

    #!/bin/bash
# update ip address with rimu hosting. See https://rimuhosting.com/dns/dyndns.jsp
if [[ ! -z ${new_ip_address} ]]
then
   echo $(date +"%F %T") ${new_ip_address} >> /root/ddns.txt
   curl "https://rimuhosting.com/dns/dyndns.jsp?action=SET&name=clock.emacstragic.net&value=${new_ip_address}&type=A&api_key=XXX"
else
    echo "got no ip"
fi

    

    Update:
    

    I discovered this didn’t work at home. Turns out that dnsmasq in OpenWRT is set to ignore this kind of dns request, due to potential security risks. There is a solution to that. Add a list rebind_domain line to your /etc/config/dhcp line on the router.
    

    config dnsmasq
	.
	.
	.
	list    rebind_domain 'clock.emacstragic.net'

    

    Thanks to Michal Čihař for the solution to that.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
			
    Facebook selecting wrong thumbnail for WordPress links
    
			
    Does Facebook keep selecting the wrong thumbnail for WordPress links for you? The solution is to give Facebook some extra instruction about what image to use for the thumbnail, using open graph.
    
If you use a static frontpage, it’s a simple matter of adding something like:
    

    
<meta property="og:image" content="http://samplesite.com/files/2014/05/web-thumb.png" />
    

    
to the Full meta tags of your front page.
    
You can check what facebook will do with your site by using their link debugger: https://developers.facebook.com/tools/debug
    
WordPress >= 3.9.1 seems to do the right thing for posts according to my testing.
    

    
			
    
		
    
		
	
  • 
		
		
    
			
    Backup your Beaglebone Black
    
			
    Backup your Beaglebone Black
    
			
    Using a method I previously wrote about, it’s quite easy to backup your Beaglebone Black over the network.
    

    ssh root@bbb 'tar cf - / 2>/dev/null' | pv -cN tar \
   | bzip2 | pv -cN bzip2 > bbb-backup-tar.bz2

    

    It runs the bzip compression locally because presumably it will be faster than on the Beaglebone Black. I didn’t actually test that hypothesis though.
    
pv gives nice little indicators:
    

        bzip2: 1.81MB 0:00:12 [ 995kB/s] [      <=>                               ]
      tar: 36.2MB 0:00:12 [3.84MB/s] [                             <=>        ]