Pass2PHP

Post Reply
User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Monday 19 December 2016 16:25

The only true thing about response time is the logfile.

For example: at ,922 the signal of the pir comes in, at ,962 the switch command is sent. Difference between them is the response time = 42 msec.
2016-12-19 15:22:34.922 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 15:22:34.964 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 15:22:34.964 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

For other stuff like telegram, kodi, whatever there are way more variables that can cause a delay.

Sneezydevil
Posts: 126
Joined: Monday 18 January 2016 10:56
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: LUA Pass2php

Post by Sneezydevil » Monday 19 December 2016 17:32

In my silly quest for more speed I found little difference between version 2.x (with Opcache) and 3.0 (with Opcache). Without opchache version 3 is even a microfaction slower.

I'm looking for a faster way to do the base64 encoding, thats the one thing holding me back at the moment.

Out of the 20msec it takes to trigger a switch, 7msec are just for loading the base64 file.

On the other hand you can't really notice 7msec can you :D.

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Monday 19 December 2016 17:42

Wich base64 and json lua files are you using?
I tried to remove all unnessescary stuff but as I don't know lua that's not so easy.
My versions are at https://github.com/Egregius/LUA-Pass2PH ... er/scripts

It is true that most of the time get lost doing that, but I can't see another way of pushing the lua tables to an external script.
Maybe of we could skip the json encode and use native lua things to make a string like array('switch1'=>'On','switch2'=>'On') etc
Or even better a multilevel array:
array('Switch1'=>array('Status'=>'On','idx'=>123,'time'=>1234567890),'Switch2'=>array('Status'=>'On','idx'=>124,'time'=>1234568890).
Then we would only need one base64 encode/decode to handle everything.

7msec in your case is almost half of the time...

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Monday 19 December 2016 19:04

pass2php 3.1 might come faster than I guessed...

2016-12-19 18:03:12.106 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 18:03:12.142 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 18:03:12.142 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

Found a possible way to avoid the json encode. Need some hours to see if everything still works...

Sneezydevil
Posts: 126
Joined: Monday 18 January 2016 10:56
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: LUA Pass2php

Post by Sneezydevil » Monday 19 December 2016 19:30

I am using the same base64 file and it is hard to tell if the json file is the same without the comments. I was at 40-45 msec with those, I then switched to cjson which removed 20msec.

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Monday 19 December 2016 19:56

I've read about cjson as being very fast but didn't search how to get it on my syno.

Nautilus
Posts: 760
Joined: Friday 02 October 2015 12:12
Target OS: Raspberry Pi
Domoticz version: beta
Location: Finland
Contact:

Re: LUA Pass2php

Post by Nautilus » Monday 19 December 2016 23:56

Egregius wrote: For example: at ,922 the signal of the pir comes in, at ,962 the switch command is sent. Difference between them is the response time = 42 msec.
2016-12-19 15:22:34.922 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 15:22:34.964 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 15:22:34.964 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)
As a comparison (using z-wave PIR to trigger MiLight spots with MiLight Wifi bridge v5):

Code: Select all

2016-12-19 23:37:41.592 (AotecZstick) Light/Switch (PIR Musiikkihuone)
-> 275ms
2016-12-19 23:37:41.867 EventSystem: Script event triggered: Door back
-> 34ms
2016-12-19 23:37:41.901 User: Admin initiated a switch command (129/Musiikkihuoneen spottivalot/On)
-> 314ms
2016-12-19 23:37:42.215 (MiLight2) Lighting Limitless/Applamp (Musiikkihuoneen spottivalot)
The delay between last two steps is something I hope RFLink will shorten, it is now there as "given", I assume. Also the time between LUA registering the event change and then executing the switch command is quite fast, 34ms (this is something that would take much longer if passing the "On" command via commandArray). But what I do not fully understand is the 275ms between PIR "On" and then event system firing. In your 42ms there is first the event system firing which would have to be considerably faster than this. I wonder why...? Maybe because my scripts are in the database and not in the scripts folder?

Also strange, in Domoticz log the first line is actually showing up as third so they are not in chronological order.

Edit: I moved this script to the scripts folder and result now is:

Code: Select all

2016-12-20 00:06:28.230 (AotecZstick) Light/Switch (PIR Musiikkihuone)
-> 55ms
2016-12-20 00:06:28.285 EventSystem: Script event triggered: /home/pi/domoticz/scripts/lua/script_device_PIRMusiikkihuone.lua
-> 38ms
2016-12-20 00:06:28.323 User: Admin initiated a switch command (129/Musiikkihuoneen spottivalot/On)
-> 395ms
2016-12-20 00:06:28.718 (MiLight2) Lighting Limitless/Applamp (Musiikkihuoneen spottivalot)
Quite a big difference, 275ms vs. 55ms. Still doesn't explain what takes the 55ms as it is a very simple script. Can it be this slow to do the logic operations with LUA?

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Tuesday 20 December 2016 0:25

File based scripts are faster than those in database.
Remember that EACH script is executed.
For the fun: place 1000 files names script_device_A0001.lua to script_device_A1000.lua and see how much slower it will go...
That's way I'm only using one.

Find another thing to make it even faster, but don't know yet if it's wurth all the extra effort.
2016-12-19 23:18:00.929 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 23:18:00.951 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 23:18:00.952 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

Storing all statusses, idx's and updates in a cache with a time script. Then with a device script only sent the updated device without base64 or json.
But, this would need all scripts to be changed...
First block is old system with $s array, second retrieving from cache

Code: Select all

<?php
if($s['pirkeuken']=="On")
{
    if($s['keuken']=='Off'&&$s['wasbak']=='Off'&&$s['werkblad']=='Off'&&$s['kookplaat']=='Off'&&$s['zon']<500)sw($i['wasbak'],'On','wasbak');
}

if(cget('spirkeuken')=="On")
{
    if(cget('skeuken')=='Off'&&cget('swasbak')=='Off'&&cget('swerkblad')=='Off'&&cget('skookplaat')=='Off'&&cget('szon')<500)sw(cget('iwasbak'),'On','wasbak');
}

Nautilus
Posts: 760
Joined: Friday 02 October 2015 12:12
Target OS: Raspberry Pi
Domoticz version: beta
Location: Finland
Contact:

Re: LUA Pass2php

Post by Nautilus » Tuesday 20 December 2016 0:30

Egregius wrote:File based scripts are faster than those in database.
Remember that EACH script is executed.
That's way I'm only using one.
Yes, that was something I learned way back, but never (until now I mean) went through the trouble to find out how much. And in this case, the device script in the scripts folder was the only one there so it should've been the first one to execute, still took 55ms :roll:

Edit: As Blockly is supposed to be the fastest native alternative, tried that as well, result:

Code: Select all

2016-12-20 00:39:18.812 (AotecZstick) Light/Switch (PIR Musiikkihuone)
2016-12-20 00:39:18.829 EventSystem: Event triggered: TeslBlock_1
Response to device change clearly faster, 17ms, but then the actual switch command seemed to take ages (maybe should have disabled the parallel LUA scripts first). Anyhow, seems that pass2php is something to dive into at least for the stuff that one would like to response fast :) Never really realized that most of the time is wasted waiting for the scripts to start running (at least after the first delay of waiting the commandArray to be returned is tackled with json...)

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Tuesday 20 December 2016 0:59

This looks also promising...

Code: Select all

c = ''
s = ''
i = ''
t = ''
for k,v in pairs(devicechanged) do c = c..k.."|"..v.."#" end
for k,v in pairs(otherdevices) do s = s..k.."|"..v.."#" end
for k,v in pairs(otherdevices_idx) do i = i..k.."|"..v.."#" end
for k,v in pairs(otherdevices_lastupdate) do t = t..k.."|"..v.."#" end
os.execute('curl -s --data "c='..c..'&s='..s..'&i='..i..'&t='..t..'" http://127.0.0.1/secure/pass2php.php &')
commandArray={}
return commandArray

Code: Select all

$c=ex($_REQUEST['c']);
$s=ex($_REQUEST['s']);
$i=ex($_REQUEST['i']);
$t=ex($_REQUEST['t']);
function ex($x){
    $return=array();
    $pieces=explode('#',$x);
    foreach($pieces as $piece){
        $keyval=explode('|',$piece);
        if(count($keyval)>1)$return[$keyval[0]]=$keyval[1];
        else $return[$keyval[0]]='';
    }
    return $return;
}
 
So, without base64 and json encode/decode. Response time 26msec :)

Sneezydevil
Posts: 126
Joined: Monday 18 January 2016 10:56
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: LUA Pass2php

Post by Sneezydevil » Tuesday 20 December 2016 14:58

Very nice, I think soon there is no more room for improvements, haha.

For now I am sticking with json and base64.

I modified the cjson lua module to do both the json and base64 encoding, and I don't think I can get this much faster.

Some results:

The old way with base64.lua and json.lua = 38msec

Code: Select all

2016-12-19 22:50:49.435 LUA: Encoding started
2016-12-19 22:50:49.473 LUA: Encoding finished - jf-json
With cjson and base64.lua = 16msec

Code: Select all

2016-12-19 22:50:50.466 LUA: Encoding started
2016-12-19 22:50:50.482 LUA: Encoding finished - cjson
With the modified module = 2msec

Code: Select all

2016-12-19 22:56:51.484 LUA: Encoding started
2016-12-19 22:56:51.486 LUA: Encoding finished - cb64json
This means pass2php gets called after only 2msec.

I will keep this running for a while to make sure its stable.

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Tuesday 20 December 2016 15:12

Wow! Can you post how you did that? Is that possible on Syno?
I'm at 25msec (total response time) now.
Maybe also good to mention how many devices? I have 109 used devices, the more the more time logically.

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Tuesday 20 December 2016 15:37

Now,... also added a print statement for the 'encoding'
Not much to improve there.

Code: Select all

2016-12-20 14:36:16.838  LUA: Encode Start
2016-12-20 14:36:16.840  LUA: encode ready
2016-12-20 14:36:16.847  LUA: curl ready

Sneezydevil
Posts: 126
Joined: Monday 18 January 2016 10:56
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: LUA Pass2php

Post by Sneezydevil » Tuesday 20 December 2016 15:50

But thats with your new way right? It was loading the files that was causing the delay for me, and you no longer do that anyway. Don't think it's worth the hassle for you.

I have about 70 devices in use, but that doesn't seem to have alot of impact.

If I encode just the changed device it takes 2msec.
If I encode all my devices it takes 2msec.
If I encode everything twice it takes 3msec

Don't know if it will work on a syno, but I guess it should be possible.

I attached the module I currently use, I extract it and place it in:

Code: Select all

/usr/local/lib/lua/5.2/
The top of the lua script looks like this:

Code: Select all

local cb64json = require "cb64json"
changed=cb64json.b64encode(cb64json.encode(devicechanged))
devices=cb64json.b64encode(cb64json.encode(otherdevices))
idx=cb64json.b64encode(cb64json.encode(otherdevices_idx))
lastupdate=cb64json.b64encode(cb64json.encode(otherdevices_lastupdate))
Attachments
cb64json.tar.gz
(13.31 KiB) Downloaded 29 times

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Tuesday 20 December 2016 16:01

Like I tought, doesn't work, other platform...

Code: Select all

2016-12-20 15:00:30.008  Error: EventSystem: in /usr/local/domoticz/var/scripts/lua/script_device_pass2php.lua: error loading module 'cb64json' from file '/usr/local/lib/lua/5.2/cb64json.so':
	/usr/local/lib/lua/5.2/cb64json.so: wrong ELF class: ELFCLASS32
Indeed, just a concatenate of all statusses etc:

Code: Select all

c = ''
s = ''
i = ''
t = ''
for k,v in pairs(devicechanged) do c = c..k.."|"..v.."#" end
for k,v in pairs(otherdevices) do s = s..k.."|"..v.."#" end
for k,v in pairs(otherdevices_idx) do i = i..k.."|"..v.."#" end
for k,v in pairs(otherdevices_lastupdate) do t = t..k.."|"..v.."#" end
os.execute('curl -s --data "c='..c..'&s='..s..'&i='..i..'&t='..t..'" http://127.0.0.1/secure/pass2php.php &')
commandArray={}
return commandArray

User avatar
Westcott
Posts: 548
Joined: Tuesday 09 December 2014 18:04
Target OS: Raspberry Pi
Domoticz version: Beta
Location: UK - Glos
Contact:

Re: LUA Pass2php

Post by Westcott » Tuesday 20 December 2016 19:27

Hi SneezyDevil,
WOW!
4ms to load your lib and do the 4 encodes on my Pi2.

I've now installed cjson as well.
Zwave - Sigma Z+ stick, Fibaro 2x FGSD002, FGMS001
Horstmann - 2x HRT4, 2x C-stat-11, 4x ASR-ZW, TKB 2x TZ56, 2x TZ68E
Max-EQ Cube, Valve, Neo 2x door
RFXtrx433E, RFlink - 1byone driveway, doorbell, Oregon and BangGood temp
Wifi - YeeLight RGB, 3x ESP8266

User avatar
sincze
Posts: 960
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi
Domoticz version: 4.9700
Location: Netherlands
Contact:

Re: RE: Re: LUA Pass2php

Post by sincze » Tuesday 20 December 2016 22:19

Westcott wrote:Hi SneezyDevil,
WOW!
4ms to load your lib and do the 4 encodes on my Pi2.

I've now installed cjson as well.
Do we have a wiki :) for my ARM based system.

Sent from my SM-G925F using Tapatalk
Using Pass2php since 2016-12
LAN: RFLink, P1-Port, OTGW, MySensors
USB: RFXCom, ZWave
WIFI: Mi-light Wifi-Bridge, Sonoff, ESP8266, Xiaomi Gateway
Solar: Omnik Inverter, PVOutput
Video: Kodi clients with Harmony HUB
Sensors: You name it I probably got 1.

User avatar
sincze
Posts: 960
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi
Domoticz version: 4.9700
Location: Netherlands
Contact:

Re: LUA Pass2php

Post by sincze » Wednesday 21 December 2016 19:13

Did the following now:

Extracted the so file from cb64json.tar.gz to

Code: Select all

/usr/local/lib/lua/5.2/
Modified the script_device_pass2php.lua to:

Code: Select all

local cb64json = require "cb64json"
changed=cb64json.b64encode(cb64json.encode(devicechanged))
devices=cb64json.b64encode(cb64json.encode(otherdevices))
idx=cb64json.b64encode(cb64json.encode(otherdevices_idx))
lastupdate=cb64json.b64encode(cb64json.encode(otherdevices_lastupdate))
c = ''
s = ''
i = ''
t = ''
for k,v in pairs(devicechanged) do c = c..k.."|"..v.."#" end
for k,v in pairs(otherdevices) do s = s..k.."|"..v.."#" end
for k,v in pairs(otherdevices_idx) do i = i..k.."|"..v.."#" end
for k,v in pairs(otherdevices_lastupdate) do t = t..k.."|"..v.."#" end
os.execute('curl -s --data "c='..c..'&s='..s..'&i='..i..'&t='..t..'" http://127.0.0.1/secure/pass2php.php &')
commandArray={}
return commandArray
And as a result of your guys work i get:

Code: Select all

2016-12-21 18:08:21.929 User: Admin initiated a switch command (193/Lamp (Keukentafel)/On)
2016-12-21 18:08:21.930 (RFXCom) Lighting 2 (Lamp (Keukentafel))
2016-12-21 18:08:23.027 (RFLink Gateway) Light/Switch (Lamp (Keukentafel) (RF))
Seems pretty okay to me right?

and:

Code: Select all

2016-12-21 18:13:38.180 User: Admin initiated a switch command (193/Lamp (Keukentafel)/Off)
2016-12-21 18:13:38.152 (RFLink Gateway) Temp + Humidity (TH7_Berging (RF))
2016-12-21 18:13:38.321 (RFXCom) Lighting 2 (Lamp (Keukentafel)) 
Using Pass2php since 2016-12
LAN: RFLink, P1-Port, OTGW, MySensors
USB: RFXCom, ZWave
WIFI: Mi-light Wifi-Bridge, Sonoff, ESP8266, Xiaomi Gateway
Solar: Omnik Inverter, PVOutput
Video: Kodi clients with Harmony HUB
Sensors: You name it I probably got 1.

User avatar
Egregius
Posts: 2713
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: Beta
Location: Beitem, BE
Contact:

Re: LUA Pass2php

Post by Egregius » Wednesday 21 December 2016 19:20

You're only using my part of it.
Don't need to double it.
Now your sending c s t and i
If you want to use vb64json that should be changed devices idx and lastupdate as in the example of Sneezydevil.
Keep in mind that when you base64/json encode you also need to enable the decode again in the php file.

User avatar
sincze
Posts: 960
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi
Domoticz version: 4.9700
Location: Netherlands
Contact:

Re: LUA Pass2php

Post by sincze » Wednesday 21 December 2016 20:34

Egregius wrote:You're only using my part of it.
Don't need to double it.
Now your sending c s t and i
If you want to use vb64json that should be changed devices idx and lastupdate as in the example of Sneezydevil.
Keep in mind that when you base64/json encode you also need to enable the decode again in the php file.
A yes.. came to that conclusion when I arrived home and not everything was working :D
Reverted back to old plain old pass2php 3.0

Code: Select all

2016-12-21 19:30:13.040 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:13.046 => SETLEVEL 40
2016-12-21 19:30:13.048 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:13.055 User: Admin initiated a switch command (193/Lamp (Keukentafel)/Set Level)
2016-12-21 19:30:13.596 (MySensors LAN) Lux (Light Level)
2016-12-21 19:30:13.897 RFLink: 2 level: 15
2016-12-21 19:30:13.865 (RFXCom) Lighting 2 (Lamp (Keukentafel))
2016-12-21 19:30:15.015 User: Admin initiated a switch command (226/Motion Keuken/On)
2016-12-21 19:30:15.016 (Virtual Device) Lighting 2 (Motion Keuken)
2016-12-21 19:30:15.286 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:15.288 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:15.293 => Motion Keuken, Lamp aangezet 40%
2016-12-21 19:30:15.295 => SETLEVEL 40
2016-12-21 19:30:15.298 => SETLEVEL 40
And everything was backup and running as it should. Have to read the Sneezydevil post more carefully i think.
Using Pass2php since 2016-12
LAN: RFLink, P1-Port, OTGW, MySensors
USB: RFXCom, ZWave
WIFI: Mi-light Wifi-Bridge, Sonoff, ESP8266, Xiaomi Gateway
Solar: Omnik Inverter, PVOutput
Video: Kodi clients with Harmony HUB
Sensors: You name it I probably got 1.

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests