Python plugin memory management

Python and python framework
Post Reply
User avatar
heggink
Posts: 622
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V4.9999
Location: NL
Contact:

Python plugin memory management

Post by heggink » Friday 16 November 2018 14:39

Every time a python plugin encounters an error, domoticz seems to lose a lot of memory. Since disabling the plugin isn't always an immediate option and since plugins are difficult to programme completely bug free, is there any way whereby the plugin framework can be made a bit more resilient in terms of memory?

H
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

User avatar
heggink
Posts: 622
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V4.9999
Location: NL
Contact:

Re: Python plugin memory management

Post by heggink » Wednesday 21 November 2018 14:54

@Dnpwwo?
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

User avatar
Dnpwwo
Posts: 767
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: Python plugin memory management

Post by Dnpwwo » Friday 23 November 2018 14:47

I haven't seen this.

Do you have a sample script that throws an error and loses memory?
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

User avatar
heggink
Posts: 622
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V4.9999
Location: NL
Contact:

Re: Python plugin memory management

Post by heggink » Friday 23 November 2018 17:36

The script that caused it last time was the life360 script. At that point, it still used the google maps api that google has just decommissioned. It got buggered up every time I would cross the atlantic (with google unable to find a route for me and the plugin raising an (unexpected) python error. Once I got to the other side of the pond, my memory was close to gone with Domoticz eating it all.

I still have the original script and can send it if you wish. Should be easy to fake a google error again.
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

User avatar
heggink
Posts: 622
Joined: Tuesday 08 September 2015 21:44
Target OS: Raspberry Pi
Domoticz version: V4.9999
Location: NL
Contact:

Re: Python plugin memory management

Post by heggink » Thursday 06 December 2018 12:11

@dnpwwo: just found another example: in the heartbeat of my own iCloud plugin, I did not check for any errors raised by the pyicloud library (PyiCloudFailedLoginException). Last 24 hours, it failed a bunch of times resulting in the domoticz process jumping from 20ish% mem usage to over 50% usage in less than a day. The below was the only error and domoticz memory had been stable for 5 days. I am on version 10171 (about a month old).

domoticz log:
Dec 5 19:07:27 domoticz domoticz: (XXXX) 'onHeartbeat' failed 'PyiCloudFailedLoginException'.
Dec 5 19:07:27 domoticz domoticz: (XXXX) ----> Line 261 in '/home/pi/domoticz/plugins/iCloud/plugin.py', function onHeartbeat
Dec 5 19:07:27 domoticz domoticz: (XXXX) ----> Line 141 in '/home/pi/domoticz/plugins/iCloud/plugin.py', function onHeartbeat
Dec 5 19:07:27 domoticz domoticz: (XXXX) ----> Line 192 in '/usr/lib/python3.5/pyicloud/base.py', function __init__
Dec 5 19:07:27 domoticz domoticz: (XXXX) ----> Line 215 in '/usr/lib/python3.5/pyicloud/base.py', function authenticate

there were 616 of these errors in total.
Pi3, latest beta
RFXCOM, z-wave (AEOTEC, switches, temhum, pir, contacts),
Plugwise2py, P1 'smart'meter & solar panel
Alexa, Wifi Cams motion detection
ESP: relays, PIR & Temp/TempHum
Geofence iCloud, Bluetooth & Wifi ping
Harmony hub, Nest

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest