I found collectd causing rrd illegal attempt to update using time errors. I was seeing a whole load of lines like this in my syslog:
Aug 20 16:27:12 mythbox collectd: rrdtool plugin: rrd_update_r (/var/lib/collectd/rrd/mythbox/df-root/df_complex-free.rrd) failed: /var/lib/collectd/rrd/mythbox/df-root/df_complex-free.rrd: illegal attempt to update using time 1345444032 when last update time is 1345444032 (minimum one second step)
It was adding one message like that every second so my logs were completely full of it. Google didn’t reveal much except that this sort of error is either because there are two instances of RRD trying to write the RRD database at the same time, or that my server’s date and time are way out of sync. Neither of these were true in my case.
I asked on #collectd on freenode and a very nice person by the name of tokkee told me that it’s a known issue of sorts. The df plugin for collectd uses /proc/mount to determine which drives to check free space on and if / is in there twice, it tries to update the entry for / twice and causes the problem.
The solution is to add the following to the /etc/collectd/collectd.conf file:
FSType "rootfs" IgnoreSelected true
Then I restarted collectd and my logs were peaceful again.
I was getting these errors again on one of my VPS hosts. In this instance, / only appeared once in /proc/mounts but /run was in there multiple times:
root@new:/etc/collectd# cat /proc/mounts rootfs / rootfs rw 0 0 /dev/root / ext3 rw,relatime,errors=remount-ro,data=ordered 0 0 devtmpfs /dev devtmpfs rw,relatime,size=1085360k,nr_inodes=271340,mode=755 0 0 tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=217328k,mode=755 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=460860k 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0 root@new:/etc/collectd#
The solution is to ignore tmpfs instead of rootfs:
FSType "tmpfs" IgnoreSelected false
13 responses to “Collectd causing rrd illegal attempt to update using time errors”
add it where please ?
I see that wasn’t clear from my post. Add it to the /etc/collectd/collectd.conf file. I’ve updated my post to reflect that.
Your blogpost saved my day 🙂
Glad to hear it!
It saved my day too 🙂 Thanks
Saved my day, thanks a lot!
thank’s for sharing this tips ! 🙂
Awesome, simple solution to an annoying problem!
Awesome! Didn’t think having a /run/shm and /run/lock would cause this problem.. based on the previously known bugs.. THANKS!
solved my problem – thanks!
Hello! It didn’t saved mine 🙁 I have added the few lines but what do you mean by “restarted collectd”? The problem remains in my case…
I see this is now the default config but I’m still seeing this issue? Anyone else? But only on one host and it’s an Azure Ubuntu host…