Persistent RAM disk

Introduction

Deprecated

We now recommend the use of enterprise-class NVMe SSDs instead of RAM disks for ease of use and manageability.

This guide will help you to create a persistent RAM disk on Ubuntu/Debian Linux variants to help speed up Observium on servers with lots of RAM.

I/O load is a serious issue for the kind of data storage that Observium performs. During polling sessions with multiple parallel pollers the I/O subsystem of the server can become a severe bottleneck. The load not only affects reads and writes to RRD files, but also reads and writes for the MySQL server, which can drastically slow the web interface.

We use the tmpfs file system on Linux which creates a filesystem in RAM.

RRD files are copied at boot-up from hard disk to the RAM disk. They can optionally be synced periodically to the hard disk to reduce the amount of data lost if the server crashes or reboots.

Setup

First, create a mountpoint for the disk :

mkdir /mnt/ramdisk

Secondly, add this line to /etc/fstab in to mount the drive at boot-time.

tmpfs           /mnt/ramdisk tmpfs      defaults,size=8192M 0 0

Change the size option in the above line to easily accomodate the amount of RRD files you'll have. Don't worry, it doesn't allocate all of that space immediately, but only as it's used. It's safe to use up to half of your RAM, perhaps more if your system has a lot of ram that's not being used.

Mount the new filesystem

mount /mnt/ramdisk

Check to see that it's mounted

mount
df -h

You should see these entries in mount and df output

tmpfs on /mnt/ramdisk type tmpfs (rw,relatime,size=8388608k)

tmpfs                 8.0G  0.0G  8.0G   0% /mnt/ramdisk

Next we need to create a directory to store the backup copies of the RRD files.

mkdir /var/ramdisk-backup

You can put it wherever you like, so long as you change the script we create below to reflect the new location.

Create a script at /etc/init.d/ramdisk with the following contents

 #! /bin/sh 
 # /etc/init.d/ramdisk
 #

 case "$1" in
   start)
     echo "Copying files to ramdisk"
     rsync -av /var/ramdisk-backup/ /mnt/ramdisk/
     echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
     ;;
   sync)
     echo "Synching files from ramdisk to Harddisk"
     echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
     rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
     ;;
   stop)
     echo "Synching logfiles from ramdisk to Harddisk"
     echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
     rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
     ;;
   *)
     echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
     exit 1
     ;;
 esac

 exit 0

Now set this up to run at startup:

update-rc.d ramdisk defaults 00 99

Either move your RRDs to /var/ramdisk-backup/observium_rrd and then load them into the ram disk:

mv /opt/observium/rrd /var/ramdisk-backup/observium_rrd
/etc/init.d/ramdisk start

Or move your RRDs to the ram disk itself and then sync them out to the backup:

mv /opt/observium/rrd /mnt/ramdisk/rrd
/etc/init.d/ramdisk sync

Now either symlink /mnt/ramdisk/rrd to /opt/observium/rrd, or change the configuration so the rrds are loaded from the ramdisk path.

You can put ramdisk sync into crontab to periodically sync your ram disk back to the hard disk:

2 * * * * root        /etc/init.d/ramdisk sync >> /dev/null 2>&1

Other Ideas

Multiple Copies

You could even keep multiple copies in the event that something gets corrupted.

2 * * * * root        rm-rf /var/ramdisk-backup-b; mv /var/ramdisk-backup /var/ramdisk-backup-b; /etc/init.d/ramdisk sync >> /dev/null 2>&1

This system would work particularly well paired with a small SSD. It would have relatively fast read/write times at boot. It would be recommended to keep a copy of RRDs on the hard disk too for protection in the event that the SSD becomes corrupted.

Tar instead of Rsync

To reduce the time it takes to dump and restore the ramdisk, we use use tar and gzip to reduce the size of the data being read/written to the physical disk. RRDs are very compressible, and seem to get down to ~10% of their original size.

In the script below I've also included a backup of the backup, incase the server crashes whilst writing the tar.gz file.

 #! /bin/sh 
 # /etc/init.d/ramdisk
 #

 case "$1" in
   start)
     echo "Copying files to ramdisk"
     cd /mnt
     tar zxvf ramdisk-backup.tar.gz
     echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
     ;;
   sync)
     echo "Synching files from ramdisk to Harddisk"
     echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
     cd /mnt
     mv -f ramdisk-backup.tar.gz ramdisk-backup-old.tar.gz
     tar zcvf ramdisk-backup.tar.gz ramdisk
     ;;
   stop)
     echo "Synching logfiles from ramdisk to Harddisk"
     echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
     tar zcvf ramdisk-backup.tar.gz ramdisk
     ;;
   *)
     echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
     exit 1
     ;;
 esac

 exit 0

LZOP instead of Gzip

If you've installed the lzop compressor, you can use it instead of gzip. It was 4 times faster for me.

Try this script

 #! /bin/sh
 # /etc/init.d/ramdisk
 #

 case "$1" in
  start)
    echo "Copying files to ramdisk"
    cd /mnt
    tar --lzop -xvf ramdisk-backup.tar.lzop
    echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
    ;;
  sync)
    echo "Synching files from ramdisk to Harddisk"
    echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
    cd /mnt
    mv -f ramdisk-backup.tar.lzop ramdisk-backup-old.tar.lzop
    tar --lzop -cvf ramdisk-backup.tar.lzop ramdisk
    ;;
  stop)
    echo "Synching logfiles from ramdisk to Harddisk"
    echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
    tar --lzop -cvf ramdisk-backup.tar.lzop ramdisk
    ;;
  *)
    echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
    exit 1
    ;;
 esac

 exit 0