Persistent RAM disk
Introduction
Deprecated
We now recommend the use of enterprise-class NVMe SSDs instead of RAM disks for ease of use and manageability.
This guide will help you to create a persistent RAM disk on Ubuntu/Debian Linux variants to help speed up Observium on servers with lots of RAM.
I/O load is a serious issue for the kind of data storage that Observium performs. During polling sessions with multiple parallel pollers the I/O subsystem of the server can become a severe bottleneck. The load not only affects reads and writes to RRD files, but also reads and writes for the MySQL server, which can drastically slow the web interface.
We use the tmpfs file system on Linux which creates a filesystem in RAM.
RRD files are copied at boot-up from hard disk to the RAM disk. They can optionally be synced periodically to the hard disk to reduce the amount of data lost if the server crashes or reboots.
Setup
First, create a mountpoint for the disk :
mkdir /mnt/ramdisk
Secondly, add this line to /etc/fstab
in to mount the drive at boot-time.
tmpfs /mnt/ramdisk tmpfs defaults,size=8192M 0 0
Change the size
option in the above line to easily accomodate the amount of RRD files you'll have. Don't worry, it doesn't allocate all of that space immediately, but only as it's used. It's safe to use up to half of your RAM, perhaps more if your system has a lot of ram that's not being used.
Mount the new filesystem
mount /mnt/ramdisk
Check to see that it's mounted
mount
df -h
You should see these entries in mount
and df
output
tmpfs on /mnt/ramdisk type tmpfs (rw,relatime,size=8388608k)
tmpfs 8.0G 0.0G 8.0G 0% /mnt/ramdisk
Next we need to create a directory to store the backup copies of the RRD files.
mkdir /var/ramdisk-backup
You can put it wherever you like, so long as you change the script we create below to reflect the new location.
Create a script at /etc/init.d/ramdisk
with the following contents
#! /bin/sh
# /etc/init.d/ramdisk
#
case "$1" in
start)
echo "Copying files to ramdisk"
rsync -av /var/ramdisk-backup/ /mnt/ramdisk/
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
;;
sync)
echo "Synching files from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
;;
stop)
echo "Synching logfiles from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
;;
*)
echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
exit 1
;;
esac
exit 0
Now set this up to run at startup:
update-rc.d ramdisk defaults 00 99
Either move your RRDs to /var/ramdisk-backup/observium_rrd
and then load them into the ram disk:
mv /opt/observium/rrd /var/ramdisk-backup/observium_rrd
/etc/init.d/ramdisk start
Or move your RRDs to the ram disk itself and then sync them out to the backup:
mv /opt/observium/rrd /mnt/ramdisk/rrd
/etc/init.d/ramdisk sync
Now either symlink /mnt/ramdisk/rrd
to /opt/observium/rrd
, or change the configuration so the rrds are loaded from the ramdisk path.
You can put ramdisk sync into crontab to periodically sync your ram disk back to the hard disk:
2 * * * * root /etc/init.d/ramdisk sync >> /dev/null 2>&1
Other Ideas
Multiple Copies
You could even keep multiple copies in the event that something gets corrupted.
2 * * * * root rm-rf /var/ramdisk-backup-b; mv /var/ramdisk-backup /var/ramdisk-backup-b; /etc/init.d/ramdisk sync >> /dev/null 2>&1
This system would work particularly well paired with a small SSD. It would have relatively fast read/write times at boot. It would be recommended to keep a copy of RRDs on the hard disk too for protection in the event that the SSD becomes corrupted.
Tar instead of Rsync
To reduce the time it takes to dump and restore the ramdisk, we use use tar and gzip to reduce the size of the data being read/written to the physical disk. RRDs are very compressible, and seem to get down to ~10% of their original size.
In the script below I've also included a backup of the backup, incase the server crashes whilst writing the tar.gz file.
#! /bin/sh
# /etc/init.d/ramdisk
#
case "$1" in
start)
echo "Copying files to ramdisk"
cd /mnt
tar zxvf ramdisk-backup.tar.gz
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
;;
sync)
echo "Synching files from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
cd /mnt
mv -f ramdisk-backup.tar.gz ramdisk-backup-old.tar.gz
tar zcvf ramdisk-backup.tar.gz ramdisk
;;
stop)
echo "Synching logfiles from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
tar zcvf ramdisk-backup.tar.gz ramdisk
;;
*)
echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
exit 1
;;
esac
exit 0
LZOP instead of Gzip
If you've installed the lzop compressor, you can use it instead of gzip. It was 4 times faster for me.
Try this script
#! /bin/sh
# /etc/init.d/ramdisk
#
case "$1" in
start)
echo "Copying files to ramdisk"
cd /mnt
tar --lzop -xvf ramdisk-backup.tar.lzop
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
;;
sync)
echo "Synching files from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
cd /mnt
mv -f ramdisk-backup.tar.lzop ramdisk-backup-old.tar.lzop
tar --lzop -cvf ramdisk-backup.tar.lzop ramdisk
;;
stop)
echo "Synching logfiles from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
tar --lzop -cvf ramdisk-backup.tar.lzop ramdisk
;;
*)
echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
exit 1
;;
esac
exit 0