Memory Leakage?

Please use template to report bugs and problems. Post here your questions when not sure where else to post
Only for bugs in the Domoticz application! other problems go in different subforums!
Forum rules
Before posting here, make sure you are on the latest Beta or Stable version.
If you have problems related to the web gui, clear your browser cache + appcache first.

Use the following template when posting here:

Version: xxxx
Platform: xxxx
Plugin/Hardware: xxxx
Description:
.....

If you are having problems with scripts/blockly, always post the script (in a spoiler or code tag) or screenshots of your blockly

If you are replying, please do not quote images/code from the first post

Please mark your topic as Solved when the problem is solved.
User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Thursday 07 December 2017 21:07

heggink wrote:
Thursday 07 December 2017 16:35
Btw, you can use monit to monitor process and system and restart processed when they exceed a threshold. Prevents locks.
This a workaround, not a solution or fix.
I use the workaround of stopping and starting domoticz by logrotate every day.

A solution/fix would be appreciated.
I noticed the memory leakage when I updated my stable domoticz to the latest stable on 31th of July.
One of the big changes then was the integration of dzVents. So it might be worthy to look the changes in the latest stable.
Last edited by EddyG on Thursday 07 December 2017 21:11, edited 1 time in total.
Regards,
Eddy

User avatar
heggink
Posts: 483
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V3.9530
Location: NL
Contact:

Re: Memory Leakage?

Post by heggink » Thursday 07 December 2017 21:10

Agree that it's no solution in any way. There is a bug registered for this. Seems to be caused by the webserver changes.
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

User avatar
heggink
Posts: 483
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V3.9530
Location: NL
Contact:

Re: Memory Leakage?

Post by heggink » Thursday 14 December 2017 17:31

For those on the BETA channel, the webserver fix has been implemented and the memory leak there seems to have been resolved.
https://github.com/domoticz/domoticz/issues/1955
Beta should no longer leak memory. Many thanks to everyone involved and @rkluwen for providing the fix.
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Thursday 14 December 2017 17:44

Nice that it is solved in beta. :D
I read your comment about put in an issue for the latest stable.
It might be that my memory leak in stable is already gone in the beta, so much has changed over time.
Is there a way that I could update to the latest beta, check if I still have the problem, and revert back to stable if not?
Does anyone know if the changes to the database in beta 'hurt' in the stable release?
Regards,
Eddy

User avatar
waaren
Posts: 807
Joined: Tuesday 03 January 2017 15:18
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Netherlands
Contact:

Re: Memory Leakage?

Post by waaren » Thursday 14 December 2017 17:51

Hi EddyG,

I switched a number of times between 8153 and later beta's without any problems but it is always more safe to make a copy of your domoticz.db before updating.
Raspberry (debian stretch via berryboot on Synology DS916+) , Domoticz (almost) latest Beta, , dzVents 2.6, RFLink, RFXtrx433e, P1, Youless, Harmony, Hue, Yeelight, Xiaomi, HomeWizard, Zwave, Amazon echo

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Thursday 14 December 2017 20:19

But I am talking switching between V3.8153 which has db version 116 and the latest beta which has db version 122
And I saw some on table Cameras, Hardware, DeviceStatus, Meter and a view others.
Especially the latest update to db version 122 has a lot of changes.
I was wondering if someone could comment on those changes if I use the latest stable Domoticz executable.
I am doing this on a production system that is totally tuned in.
To find out if my memory leakage is solved I have to run the beta for longer time.
I do not want to lose data on my production system.
Regards,
Eddy

User avatar
heggink
Posts: 483
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V3.9530
Location: NL
Contact:

Re: Memory Leakage?

Post by heggink » Thursday 14 December 2017 20:48

Hi EddyG,
As said, best to shut down domoticz, backup the DB, backup the domoticz folder and upgrade.
I have switched between stable and beta a number of times over the past 12 months (moving form stable to beta) due to fixes in Beta that I really wanted. I burnt my fingers a couple of times upgrading when I should not have done that (Nov 9th when the webserver mem leak kicked in). On a whole, I find the beta's stable enough and pretty much as stable and 'stable'. As long as you keep a decent backup, you should be fine.
H
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Thursday 14 December 2017 22:20

I tried for an hour the beta release. It looks (99%) stable in memory leakage.
But I had to revert back. My dzVents scripts did not work anymore.
The checkbox on the 'Other' tab could not be unchecked I have to look into that.
Regards,
Eddy

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Friday 15 December 2017 10:18

Got it working. Chrome cache was not cleared properly.
Regards,
Eddy

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Thursday 21 December 2017 20:30

After the Websockets bug was solved it looks like there is still Memory Leakage introduced before the latest stable.
I opened an issue. Please contribute in finding the cause.
Until now I found that it has something to do with the event system and/or dzVents.

https://github.com/domoticz/domoticz/issues/2000
Regards,
Eddy

User avatar
Siewert308SW
Posts: 607
Joined: Monday 29 December 2014 16:47
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands - Zoutkamp (gn)
Contact:

Re: Memory Leakage?

Post by Siewert308SW » Friday 29 December 2017 11:46

If all goes well then it should be fixed soon in the beta version.
Made a pull request and awaiting approval.
https://github.com/domoticz/domoticz/issues/2000
Setup:
- RPi3 - PIco HV3.0A / Domo Beta 3.9XXX / RFXtrx433E / Aeotec Gen5
- RPi3 - PIco HV3.0A / PiHole / PiVPN / NAS / Print Server
- Youless Elec&Gas
- FI9803P Cams
- KD101 detectors
- Zwave & KaKu

MyDomoticz scripts: @GitHub
PIco UPS Toolkit: @GitHub

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Saturday 30 December 2017 20:32

Just back from ski holliday. I compiled version v3.8797 and deleted the 2 lines. I suppose that's what you did too.

Code: Select all

-	if (lua_state == NULL)
-		return NULL;
I started at around 17:00 with some testing.
Now some 2 hours later I notice still some increase in memory consumption by domoticz.
It might be early to tell but it looks that the problem is not totally away.
Tomorrow I will know more.
Regards,
Eddy

User avatar
Siewert308SW
Posts: 607
Joined: Monday 29 December 2014 16:47
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands - Zoutkamp (gn)
Contact:

Re: Memory Leakage?

Post by Siewert308SW » Saturday 30 December 2017 20:44

EddyG wrote:
Saturday 30 December 2017 20:32
Just back from ski holliday. I compiled version v3.8797 and deleted the 2 lines. I suppose that's what you did too.

Code: Select all

-	if (lua_state == NULL)
-		return NULL;
I started at around 17:00 with some testing.
Now some 2 hours later I notice still some increase in memory consumption by domoticz.
It might be early to tell but it looks that the problem is not totally away.
Tomorrow I will know more.
I think your looking at the wrong memory.
It's not the total memory or memory used by Domoticz you should look at.
It's the virtual memory you have to look at.
The memory used by Domoticz will drop if needed over time, sometimes it will take a week or if the system is heavily earlier.
Or like my system it can take a 3weeks.
It's the Virtual Memory which you should look at, at my system it's stable now while before the fix it was increasing.
And never released and causing the system to crash as it was allocated to Domoticz and no other process could/may use it.
Setup:
- RPi3 - PIco HV3.0A / Domo Beta 3.9XXX / RFXtrx433E / Aeotec Gen5
- RPi3 - PIco HV3.0A / PiHole / PiVPN / NAS / Print Server
- Youless Elec&Gas
- FI9803P Cams
- KD101 detectors
- Zwave & KaKu

MyDomoticz scripts: @GitHub
PIco UPS Toolkit: @GitHub

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Saturday 30 December 2017 21:02

I know. I read all the items in the issue.
I log every minute both RSS and VSZ to a text file which I process into a graph, like I have done before.
For now it seems that both RSS and VSZ are increasing.
Tomorrow I will have more data.
Regards,
Eddy

User avatar
Siewert308SW
Posts: 607
Joined: Monday 29 December 2014 16:47
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands - Zoutkamp (gn)
Contact:

Re: Memory Leakage?

Post by Siewert308SW » Saturday 30 December 2017 21:05

EddyG wrote:
Saturday 30 December 2017 21:02
I know. I read all the items in the issue.
I log every minute both RSS and VSZ to a text file which I process into a graph, like I have done before.
For now it seems that both RSS and VSZ are increasing.
Tomorrow I will have more data.
Could you/will you be so kind to share that script?
So i can do the same and we both have the same report method.

edit:
Btw this is my memory graph.
You see it rising and the a sudden drop so memory is released.
Never saw that happen, only saw it rising and never released.
chart.png
chart.png (78.63 KiB) Viewed 462 times
Setup:
- RPi3 - PIco HV3.0A / Domo Beta 3.9XXX / RFXtrx433E / Aeotec Gen5
- RPi3 - PIco HV3.0A / PiHole / PiVPN / NAS / Print Server
- Youless Elec&Gas
- FI9803P Cams
- KD101 detectors
- Zwave & KaKu

MyDomoticz scripts: @GitHub
PIco UPS Toolkit: @GitHub

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Saturday 30 December 2017 21:21

It's a really quick and dirty script, which runs by cron every minute.

Code: Select all

#!/bin/bash
output=`ps aux|grep "domoticz/domoticz"|grep -v grep`
echo -n $output >> /media/RPiSys/Memory_Leagage.txt
date >> /media/RPiSys/Memory_Leagage.txt
I read the output file into Excel and use only the VSZ, RSS and time columns for the graph.
If you really want to monitor the memory of a process do a "cat /proc/<pid>/status"
Log all the "Kb" fields to a file and make some graphs.

Your graph shows RSS and not VSZ when I look at the numbers.
Last edited by EddyG on Saturday 30 December 2017 21:33, edited 1 time in total.
Regards,
Eddy

User avatar
Siewert308SW
Posts: 607
Joined: Monday 29 December 2014 16:47
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands - Zoutkamp (gn)
Contact:

Re: Memory Leakage?

Post by Siewert308SW » Saturday 30 December 2017 21:32

thx...
Will give it a shot at run till tomorrow so we can compare.
Setup:
- RPi3 - PIco HV3.0A / Domo Beta 3.9XXX / RFXtrx433E / Aeotec Gen5
- RPi3 - PIco HV3.0A / PiHole / PiVPN / NAS / Print Server
- Youless Elec&Gas
- FI9803P Cams
- KD101 detectors
- Zwave & KaKu

MyDomoticz scripts: @GitHub
PIco UPS Toolkit: @GitHub

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Sunday 31 December 2017 10:55

This is my Memory Usage of Domoticz after more than 12 hours uptime.
It looks to me that the problem is not solved.
Attachments
2017-12-31_09-33-12.png
2017-12-31_09-33-12.png (43.68 KiB) Viewed 427 times
Regards,
Eddy

User avatar
Siewert308SW
Posts: 607
Joined: Monday 29 December 2014 16:47
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands - Zoutkamp (gn)
Contact:

Re: Memory Leakage?

Post by Siewert308SW » Sunday 31 December 2017 12:21

Thats weird...
Here is mine after 12hr running.
mem_log.png
mem_log.png (12.25 KiB) Viewed 413 times
Setup:
- RPi3 - PIco HV3.0A / Domo Beta 3.9XXX / RFXtrx433E / Aeotec Gen5
- RPi3 - PIco HV3.0A / PiHole / PiVPN / NAS / Print Server
- Youless Elec&Gas
- FI9803P Cams
- KD101 detectors
- Zwave & KaKu

MyDomoticz scripts: @GitHub
PIco UPS Toolkit: @GitHub

User avatar
EddyG
Posts: 275
Joined: Monday 02 November 2015 6:54
Target OS: Raspberry Pi
Domoticz version: 4.9788
Location: Rhenen, Netherlands
Contact:

Re: Memory Leakage?

Post by EddyG » Sunday 31 December 2017 12:29

I took in my chart the difference with the start of domoticz, so the graph tells you how much it increases from the initial Kb's
I just had to reboot my Raspberry Pi, so I start again and let it running for some days now.
Regards,
Eddy

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 5 guests