Collectd causing rrd illegal attempt to update using time errors

I found collectd causing rrd illegal attempt to update using time errors. I was seeing a whole load of lines like this in my syslog:
Aug 20 16:27:12 mythbox collectd[32167]: rrdtool plugin: rrd_update_r (/var/lib/collectd/rrd/mythbox/df-root/df_complex-free.rrd) failed: /var/lib/collectd/rrd/mythbox/df-root/df_complex-free.rrd: illegal attempt to update using time 1345444032 when last update time is 1345444032 (minimum one second step)

It was adding one message like that every second so my logs were completely full of it. Google didn’t reveal much except that this sort of error is either because there are two instances of RRD trying to write the RRD database at the same time, or that my server’s date and time are way out of sync. Neither of these were true in my case.
I asked on #collectd on freenode and a very nice person by the name of tokkee told me that it’s a known issue of sorts. The df plugin for collectd uses /proc/mount to determine which drives to check free space on and if / is in there twice, it tries to update the entry for / twice and causes the problem.
The solution is to add the following to the /etc/collectd/collectd.conf file:

<Plugin df>
        FSType "rootfs"
        IgnoreSelected true
</Plugin>

Then I restarted collectd and my logs were peaceful again.
Update 2014-04-10:
I was getting these errors again on one of my VPS hosts. In this instance, / only appeared once in /proc/mounts but /run was in there multiple times:

root@new:/etc/collectd# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,relatime,errors=remount-ro,data=ordered 0 0
devtmpfs /dev devtmpfs rw,relatime,size=1085360k,nr_inodes=271340,mode=755 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=217328k,mode=755 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=460860k 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0
root@new:/etc/collectd#

The solution is to ignore tmpfs instead of rootfs:

<Plugin df>
        FSType "tmpfs"
        IgnoreSelected false
</Plugin>

13 comments

    1. I see that wasn’t clear from my post. Add it to the /etc/collectd/collectd.conf file. I’ve updated my post to reflect that.

  1. Awesome! Didn’t think having a /run/shm and /run/lock would cause this problem.. based on the previously known bugs.. THANKS!

  2. Hello! It didn’t saved mine 🙁 I have added the few lines but what do you mean by “restarted collectd”? The problem remains in my case…

  3. I see this is now the default config but I’m still seeing this issue? Anyone else? But only on one host and it’s an Azure Ubuntu host…

Leave a comment

Your email address will not be published. Required fields are marked *