Moving Log and Temporary Files To a RAM Drive

From Domoticz
Jump to: navigation, search

Objective:

Move most frequently written files to a ram drive to reduce SD Card wear. This article will focus on the domoticz log and some temporary files as well as the openzwave log, that we will have reside in a RAM drive mounted as a new directory /tmp/log.
With respect to the size of the RAM drive, 20MB is used as a standard trade off considering the usual size of the domoticz and openzwave temporary files, but this may have to be adjusted up or down based on the particular setup.
With respect to Domoticz, besides the log file, there are two files that can be moved to the RAM drive:
/home/pi/domoticz/domoticz.db-wal
/home/pi/domoticz/domoticz.db-shm
These are SQLite temporary files that are created when Domoticz is launched and deleted when it is stopped.
From a SD Card wear perspective, the domoticz.db-shm is a real offender… it is relatively big (usually bigger than Domoticz's database file) and being written to disk each time a data is changed in Domoticz, which can sometimes happen many times per minute.
It might be possible to have Domoticz configured to change the location of these files, but for simplicity purposes, it is just possible to create symbolic links to the RAM drive mount point… Unfortunately, it seems that these links are deleted on Domoticz's shutdown, so a solution is to modify the domoticz.sh startup script to create these links each time Domoticz is started.
Requisite: SSH access to the RPi system and decent knowledge of the bash shell, including the nano editor (or any other of your choice). It is also assumed that you are user “pi” and domoticz is installed in /home/pi/domoticz

This article is based on several sources to be found on this wiki or the internet, including: Setting_up_a_RAM_drive_on_Raspberry_Pi

ALL THIS IS DONE AT YOUR OWN RISK AND ASSUMES YOU KNOW WHAT YOU ARE DOING. YOU MAY BRICK YOUR SYSTEM OR DOMOTICZ INSTALL.


STEP 1 – Create the RAM disk and mount it
Create the mount point:

mkdir /tmp/log

Make a backup copy of the system file we are going to edit, in case anything bad happens with the edit

sudo cp -p /etc/fstab  /etc/fstab.save

Edit the /etc/fstab file:

sudo nano /etc/fstab

and add the following line:

tmpfs	        /tmp/log        tmpfs   nodev,nosuid,size=20M 0 0

then save, quit the editor and mount the RAM drive:

sudo mount /tmp/log

Check the file system

df -h

You should now see among the mount points something similar to this line

tmpfs               20M    3,9M   17M  20% /tmp/log


STEP 2 – change Domoticz log and move temporary files to the newly created mount point:

Stop domoticz:

sudo /etc/init.d/domoticz.sh stop

or

sudo systemctl stop domoticz.service

Edit the /etc/init.d/domoticz.sh script file:

sudo nano /etc/init.d/domoticz.sh

and edit the domoticz deamon logging parameters (default at system setup being #DAEMON_ARGS="$DAEMON_ARGS -log /tmp/domoticz.txt", also make sure you uncomment that line by removing the leading # character!) to now show:

DAEMON_ARGS="$DAEMON_ARGS -log /tmp/log/domoticz.log -loglevel 1"

Note that the –loglevel parameter above can be omitted or obviously changed if you want…

Now in light of the above comments on the SQLite temporary files, you need to edit the “start” section of the script to add a few lines as per below in bold:

#
# Function that starts the daemon/service
#
do_start()
{
       # edit to move SQLite temp files to RAM drive
       ln -sf /tmp/log/domoticz.db-shm /home/pi/domoticz
       ln -sf /tmp/log/domoticz.db-wal /home/pi/domoticz
       # end of edit
       # Return
       #   0 if daemon has been started
       #   1 if daemon was already running
       #   2 if daemon could not be started
       start-stop-daemon --chuid $USERNAME --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
               || return 1
       start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
               $DAEMON_ARGS \
               || return 2
}

Then save and quit the editor


STEP 3 – Only if you are using a zwave network and wish openzwave to log to the RAM drive.

Note that you will lose openwave logs in case of a system reboot if you do this. The reason I decided to do this is that I noted that when a few of my zwave devices recently failed, openwave logs a deluge of data when it polls the dead devices…

While domoticz is still stopped,

sudo rm /home/pi/domoticz/Config/OZW_Log.txt
ln -sf /tmp/log/OZW_Log.txt /home/pi/domoticz/Config


STEP 4: Apply the changes

Restart Domoticz:

sudo /etc/init.d/domoticz.sh start

or

sudo systemctl start domoticz.service

if prompted to do a systemctl daemon-reload you should do it…

You should now see all the relevant files in your /tmp/log/ directory… Here is an example:

ls /tmp/log -l
-rw-r--r-- 1 pi   pi     32768 mars   8 11:31 domoticz.db-shm
-rw-r--r-- 1 pi   pi   5236552 mars  13 10:59 domoticz.db-wal
-rwxrwxrwx 1 root root    1909 mars  13 10:19 domoticz.log
-rw-r----- 1 root root 1069621 mars  11 10:55 OZW_Log.txt 

Last point… I did not address the size issues… overtime, these files may grow too big for the RAM drive… It is not an issue for me as I am rotating these logs daily as described at [1] (in French but may be an English version exists somewhere), but you could otherwise be faced with a disk full issue if you do not maintain the logs size somehow…