backup every 12 hours on qnap ftp

On various Hardware and OS systems: pi / windows / routers / nas, etc
guantolento
Posts: 116
Joined: Saturday 01 October 2016 14:48
Target OS: Raspberry Pi
Domoticz version: LastBeta
Location: Italy BG
Contact:

backup every 12 hours on qnap ftp

Postby guantolento » Wednesday 21 December 2016 1:45

good evenig. actually i use the domoticz on my rpi2, i have some script LUA made with the built in domoticz and some script in BASH in another folder.
i want to create a script to save on an external ftp a full backup. in the wiki i can see the procedure to create the script, but i don't understand what folder i need to backup. What do you recommend ? a full SD backup or some single folder ? thanks a lot.
RPI2 + Z Stick Gen 5 + System Alive Checker + Weather Underground + MB Sensor + Dummy + 1 Wall Plug NodOn + 3 Danfoss Room Sensor + 7 Danfoss Z Thermostat + 1 Swiid Chord + 9 Fibaro FGS222 2x1.5Kw

Egregius
Posts: 1906
Joined: Thursday 09 April 2015 12:19
Target OS: NAS (Synology & others)
Domoticz version: 3.6312
Location: Beitem, BE
Contact:

Re: backup every 12 hours on qnap ftp

Postby Egregius » Wednesday 21 December 2016 2:29

I only backup the complete domoticz folder to a versioned system with rsync.

guantolento
Posts: 116
Joined: Saturday 01 October 2016 14:48
Target OS: Raspberry Pi
Domoticz version: LastBeta
Location: Italy BG
Contact:

Re: backup every 12 hours on qnap ftp

Postby guantolento » Wednesday 21 December 2016 16:04

How you can save in this way? ???what do you setting in domoticz? ????

Inviato dal mio GT-I9301I utilizzando Tapatalk
RPI2 + Z Stick Gen 5 + System Alive Checker + Weather Underground + MB Sensor + Dummy + 1 Wall Plug NodOn + 3 Danfoss Room Sensor + 7 Danfoss Z Thermostat + 1 Swiid Chord + 9 Fibaro FGS222 2x1.5Kw

User avatar
emme
Posts: 256
Joined: Monday 27 June 2016 11:02
Target OS: Raspberry Pi
Domoticz version: latest
Location: Milano, Italy
Contact:

Re: backup every 12 hours on qnap ftp

Postby emme » Wednesday 21 December 2016 16:24

Egregius wrote:I only backup the complete domoticz folder to a versioned system with rsync.

I'm curious too....
I tried an FTP copy but there are a lot of locked files (the db for example :P)
That's one small step for a programmer, one giant leap for me!
- - - - -
Raspberry Pi3 + AeonStick G5 + RFXCOMM433 + iRTrans Server + NodeJS
There's no stress in a mess!!

Egregius
Posts: 1906
Joined: Thursday 09 April 2015 12:19
Target OS: NAS (Synology & others)
Domoticz version: 3.6312
Location: Beitem, BE
Contact:

Re: backup every 12 hours on qnap ftp

Postby Egregius » Wednesday 21 December 2016 17:19

rsync doesn't seem to have issues with that.

Code: Select all

SOURCE="[email protected]/domoticz/"
DESTINATION="/volume1/homes/guy/backup/domoticz"
LOGFILE=/volume1/web/secure/backups/logs/domoticz.$NOW.txt
echo ------------------------------------ START domoticz -- $(date +"%Y-%m-%d %H:%M:%S") UTC+2 | tee -a $LOGFILE
rsync -aP --exclude-from '/volume1/homes/guy/backup/excludedfiles.txt' --stats --delete-after --ignore-errors --links --link-dest="$DESTINATION/__prev/" "$SOURCE" "$DESTINATION/$NOW" | tee -a $LOGFILE
rm -f "$DESTINATION/__prev"
ln -s "$NOW" "$DESTINATION/__prev"
echo ------------------------------------ END  domoticz -- $(date +"%Y-%m-%d %H:%M:%S") UTC+2 | tee -a $LOGFILE

Creates a folder in DESTINATION for each day the script has run.
Folder size is only the updated files as the rest are symlinks.

I do have an exclusion list wich hold this:

Code: Select all

var/domoticz.db-shm
var/domoticz.db-wal
var/domoticz.log
var/domoticz.pid
var/ozwcp.poll.XXXXXX.xml
var/ozwcp.topo.XXXXXX
var/zwscene.xml
pi/domoticz/domoticz.db-shm
pi/domoticz/domoticz.db-wal
pi/domoticz/domoticz.log
pi/domoticz/Config/OZW_Log.txt
pi/domoticz/Config/zwscene.xml

2 paths in it because I also backup a Domoticz Pi from a friend

sincze
Posts: 520
Joined: Monday 02 June 2014 22:46
Target OS: Linux
Domoticz version: 3.6179
Location: Netherlands
Contact:

Re: backup every 12 hours on qnap ftp

Postby sincze » Wednesday 21 December 2016 18:48

I can confirm rsync is working.... nicely
or you can run a .sh ...
- stop domoticz.
- backup
- start domoticz.

:D

Egregius
Posts: 1906
Joined: Thursday 09 April 2015 12:19
Target OS: NAS (Synology & others)
Domoticz version: 3.6312
Location: Beitem, BE
Contact:

Re: backup every 12 hours on qnap ftp

Postby Egregius » Wednesday 21 December 2016 19:21

No need to stop domoticz, backup is complete with rsync without stopping it.

SweetPants
Posts: 1025
Joined: Friday 12 July 2013 21:24
Target OS: Linux
Domoticz version: V3.6328
Location: The Netherlands
Contact:

Re: backup every 12 hours on qnap ftp

Postby SweetPants » Wednesday 21 December 2016 19:48

Egregius wrote:No need to stop domoticz, backup is complete with rsync without stopping it.

isn't that dangerous with an open database?

Egregius
Posts: 1906
Joined: Thursday 09 April 2015 12:19
Target OS: NAS (Synology & others)
Domoticz version: 3.6312
Location: Beitem, BE
Contact:

Re: backup every 12 hours on qnap ftp

Postby Egregius » Wednesday 21 December 2016 19:59

I restored at least 10 times without issues. And IF one got corrupt I'l pick one from the day before :D

guantolento
Posts: 116
Joined: Saturday 01 October 2016 14:48
Target OS: Raspberry Pi
Domoticz version: LastBeta
Location: Italy BG
Contact:

Re: backup every 12 hours on qnap ftp

Postby guantolento » Saturday 24 December 2016 19:56

Merry christmas,
actually i take this action:
- disable the automatic backup from the web gui of domoticz
- create a bash script to backup every day at 13:00 "domoticz" folder
- create a bash script to backup every day at 13:30 "script" folder
- create a bash script to backup every day at 14:00 domoticz database
This script save to an ftp QNAP-GL01, the datastore of this is on 2 hard disk in raid 1 (mirror). The datastore of QNAP-GL01 is replicated on QNAP-GL02 by a process rsync every 12 hours. Someone can tell me if i need to save another folder from the RPI2 ???? thanks a lot.
Last edited by guantolento on Saturday 24 December 2016 19:59, edited 1 time in total.
RPI2 + Z Stick Gen 5 + System Alive Checker + Weather Underground + MB Sensor + Dummy + 1 Wall Plug NodOn + 3 Danfoss Room Sensor + 7 Danfoss Z Thermostat + 1 Swiid Chord + 9 Fibaro FGS222 2x1.5Kw

guantolento
Posts: 116
Joined: Saturday 01 October 2016 14:48
Target OS: Raspberry Pi
Domoticz version: LastBeta
Location: Italy BG
Contact:

Re: backup every 12 hours on qnap ftp

Postby guantolento » Saturday 24 December 2016 19:57

emme wrote:
Egregius wrote:I only backup the complete domoticz folder to a versioned system with rsync.

I'm curious too....
I tried an FTP copy but there are a lot of locked files (the db for example :P)


i don't have this problem, i save to my FTP
RPI2 + Z Stick Gen 5 + System Alive Checker + Weather Underground + MB Sensor + Dummy + 1 Wall Plug NodOn + 3 Danfoss Room Sensor + 7 Danfoss Z Thermostat + 1 Swiid Chord + 9 Fibaro FGS222 2x1.5Kw

litfiba
Posts: 5
Joined: Wednesday 07 December 2016 19:48
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: backup every 12 hours on qnap ftp

Postby litfiba » Thursday 29 December 2016 15:27

Hi guys,

A simple script I use to bachup DB or entire Domoticz folder.. Daily, weekly, Monthly.. I keep only last three backup of each..

3 3 * * * /home/pi/domoticz/scripts/script_backup.sh J DB
3 4 * * 0 /home/pi/domoticz/scripts/script_backup.sh S DB
3 5 1 * * /home/pi/domoticz/scripts/script_backup.sh M DB
3 1 * * 1 /home/pi/domoticz/scripts/script_backup.sh S SYSTEM
3 2 2 * * /home/pi/domoticz/scripts/script_backup.sh M SYSTEM

#!/bin/bash
# LOCAL/FTP PARAMETERS
SERVER="192.168.0.xxx" # IP of Synology NAS, used for ftp
USERNAME="xxx" # FTP username of Network disk used for ftp
PASSWORD="xxx" # FTP password of Network disk used for ftp
DESTDIRNAS="/Public/domoticz/" # Path to your NAS backup folder
DOMO_IP="192.168.0.yyy" # Domoticz IP
DOMO_PORT="8080" # Domoticz port
DOMO_USER="yyy"
DOMO_PASS="yyy"
### END OF USER CONFIGURABLE PARAMETERS
param=$1 # 1st parameter will be for example 'J' for day, 'S' for week or 'M' for month
mode=$2 # 2nd parameter could be 'DB' for DB only backup or 'SYSTEM' for entire domoticz bacup
BACKUPFILE=$mode"_"$param"_1"
BACKUPFILEGZ="$BACKUPFILE".gz
##delete last - rename hold one
curl -v -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS" -X "DELE "$mode"_"$param"_3.gz"
curl -v -u"$USERNAME:$PASSWORD" "ftp://$SERVER" -Q "RNFR $DESTDIRNAS/"$mode"_"$param"_2.gz" -Q "RNTO $DESTDIRNAS/"$mode"_"$param"_3.gz"
curl -v -u"$USERNAME:$PASSWORD" "ftp://$SERVER" -Q "RNFR $DESTDIRNAS/"$mode"_"$param"_1.gz" -Q "RNTO $DESTDIRNAS/"$mode"_"$param"_2.gz"
### Create backup and ZIP it
#service domoticz.sh stop # not needed
case "$mode" in "DB") # case of DB only backup
/usr/bin/curl -s http://$DOMO_USER:$DOMO_PASS@$DOMO_IP:$ ... tabase.php > /tmp/$BACKUPFILE
#service domoticz.sh start # not needed
gzip -9 /tmp/$BACKUPFILE
### Send to Network disk through FTP
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILEGZ" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
### Remove temp backup file
/bin/rm /tmp/$BACKUPFILEGZ
;;
"SYSTEM") # case of entire domoticz backup
tar -zcvf /tmp/$BACKUPFILE.tar.gz /home/pi/domoticz/
#service domoticz.sh start # not needed
### Send to Network disk through FTP
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILE.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
### Remove temp backup file
/bin/rm /tmp/$BACKUPFILE.tar.gz
;;
esac
### Done!


Return to “Installation, Compiling, Permissions, Security and Starting”

Who is online

Users browsing this forum: No registered users and 1 guest