Python plugin: Presence detection from wireless router

Python and python framework
User avatar
Innovator
Posts: 15
Joined: Tuesday 20 February 2018 13:33
Target OS: NAS (Synology & others)
Domoticz version: 8924
Location: Amsterdam
Contact:

Re: Python plugin: Presence detection from wireless router

Post by Innovator » Wednesday 13 June 2018 20:09

EscApe wrote:
Tuesday 12 June 2018 22:18
@escape
If setting up key based access for the user running Domoticz on the Synology (root??) is not an option you can change a small piece of code in the plugin.
it's root

Look for (will be around line 81 in the version i'm uploading in a few minutes):

Code: Select all

cmd =[self.sshbin, "-o", "ConnectTimeout=" + str(sshtimeout), user+"@"+host, routerscript]
And change it to (with whatever path you are using):

Code: Select all

cmd =[self.sshbin, "-i", "/volume1/path/to/sshfiles/privatekeyfile", "-o", "ConnectTimeout=" + str(sshtimeout), user+"@"+host, routerscript]
I have nothing changed use version 0.4.1
You can experiment from the synology command line to confirm it is working: ssh -i <keyfile-path> usernameonrouter@routerip
ssh keyfile is correct working

I have added the wlanconfig command and ath* interface names to the latest version of the plugin. Will upload it in a few minutes. If anything goes wrong please include some debug logging of the plugin starting up. The part you included only shows that the polling failed, but the reason is a failure while initializing the plugin.

At this moment the plugin create no devices. The domticz log shows only
2018-06-13 19:49:53.064 (iDetect Wifi presence detection) Initialized version 0.4.1, author 'ESCape'
2018-06-13 19:49:53.064 (iDetect Wifi presence detection) Entering work loop.
no errors, debug = on
Version 0.4.1. is unfortunately not working
Synology DS918+ /DS412+
ASUS BRT-AC828, RT-AC86U, RT-AC68U

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Wednesday 13 June 2018 20:29

There have been some changes that require extra attention when upgrading from an earlier version.

In this order:

Make sure the ioverride.zip file (new since 0.4) is in you plugin directory (besides plugin.py and the orther zip files)
Restart domoticz! (Otherwise settings page is not updated)
go to the hardware settings page for the plugin and set the new parameters (and save)

devros
Posts: 191
Joined: Saturday 29 October 2016 20:55
Target OS: -
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by devros » Friday 15 June 2018 10:20

Thanks now it all works (almost) without errors
sometimes i have in log timeout (few times in one log)
should i try modify script to make longer timeouts ?

Code: Select all

2018-06-15 10:03:57.215 Error: (presence) SSH subprocess failed with error: Command '['sshpass', '-p', 'XXXXX', 'ssh', '-o', 'StrictHostKeyChecking=no', '-o', 'ConnectTimeout=1', 'ubnt@XXXXX', "export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH;wlanconfig ath0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;wlanconfig eth0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;exit"]' timed out after 2 seconds
2018-06-15 10:03:57.216 Error: (presence) Failed to poll present phones from router
2018-06-15 10:03:57.216 Error: (presence) No list of connected WLAN devices from router

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Friday 15 June 2018 16:46

@devros

Glad it's working! :D

Since the plugin runs an external program (ssh) it will block the plugin system whilst ssh is running. From the wiki: "Plugins are single threaded so the whole plugin system will wait." That's why i kept the timeouts short. If the router, network or Domoticz system is very busy it might fail, but at least it will not block the plugin system for long.

If the errors are not that frequent you can just ignore them. It only means you miss one poll cycle and presence/absence will be detected again on the next poll.

The timeouts relevant are:
sshtimeout: time it may take ssh to establish the connection to the router (shorter than alltimeout because real connection issues are most likely to last longer, so you want to skip the poll as soon as possible)
alltimeout: time the entire ssh command may take on the host system (a failsafe in case of unforeseen problems that can not be detected as quick as a connection issue. An example would be a working ssh connection, but hanging command on the router)

You can experiment with the timeouts. If your poll interval is more than 30 second i see no harm in increasing the (alltimeout) of the subprocess to 3-5 seconds. Worst case it will block your plugin system for 10% or even 16,6% of the time, but only if the entire command takes that long for some reason (very busy Domoticz host or router?). The sshtimeout is a different story. If your router is somehow unreachable or very slow to respond (down, rebooting, disconnected) the plugin might wait the full amount of the timeout every single time it tries to poll. On the other hand if it only polls every 30 seconds it will probably do no real harm if you set the sshtimeout to 2-3 seconds.

I poll every 10 seconds, so short timeouts are crucial to prevent the plugin from blocking the plugin system for most of the time. It is all a question of balance. On my dedicated raspberry pi i get very very few errors. The ones that did occur were related to the domoticz host being very busy (compiling domoticz ;) ) or the router rebooting. If everything is functioning normally the whole ssh command will be processed within tenths of a second... at least on my setup.

devros
Posts: 191
Joined: Saturday 29 October 2016 20:55
Target OS: -
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by devros » Sunday 17 June 2018 20:30

EscApe wrote:
Friday 15 June 2018 16:46
@devros

Glad it's working! :D

Thanks for full info about timeouts :), but noticed that its not working (sorry) with new wlanconfig on my ubitiqui router...
wlanconfig eth0 list returns 'Invalid argument' (propablly disabled)

dunno why, previous (cat /proc/net/arp)
worked great and fast, can you add switch to use ARP for scaning please :)

Code: Select all

command: ['sshpass', '-p', 'xxx', 'ssh', '-o', 'StrictHostKeyChecking=no', '-o', 'ConnectTimeout=1', 'xxx@192.168.3.1', "export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH;wlanconfig ath0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;wlanconfig eth0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;exit"]
2018-06-17 20:19:49.615 (presence) Returncode ssh command: 0
2018-06-17 20:19:49.615 (presence) Output from router: b'80:2a:a8:a4:e8:11\n'
2018-06-17 20:19:49.615 (presence) Considered absent: xxx
2018-06-17 20:19:49.616 (presence) Considered absent: xxxx

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Monday 18 June 2018 21:21

Hi devros,

In your message you write the command isn’t working, yet you include a piece of the log that shows the plugin is returning a MAC address. Should it have returned more addresses? Are you sure the plugin isn’t working? Testing the command from the command line using eth0 might indeed throw an error. Looks like the interface on your router is called ath0 (a instead of e).

Edit:
In the plugin I used ‘wlanconfig ath0 list’ instead of ‘wlanconfig ath0 list sta‘. Could you try if there is a difference between the two commands (with and without sta)?

devros
Posts: 191
Joined: Saturday 29 October 2016 20:55
Target OS: -
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by devros » Monday 18 June 2018 22:01

EscApe wrote:
Monday 18 June 2018 21:21
Hi devros,

In your message you write the command isn’t working, yet you include a piece of the log that shows the plugin is returning a MAC address. Should it have returned more addresses? Are you sure the plugin isn’t working? Testing the command from the command line using eth0 might indeed throw an error. Looks like the interface on your router is called ath0 (a instead of e).

Edit:
In the plugin I used ‘wlanconfig ath0 list’ instead of ‘wlanconfig ath0 list sta‘. Could you try if there is a difference between the two commands (with and without sta)?

On my device is ath0 outside network(one ip adress), mine should be
eth0, but with this i get invalid argument...so you plugin works, but something wrong is with my router :-)

Arp scaning worked great

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Monday 18 June 2018 22:09

I would still be interested to see if we can get it working, but that will take some experimenting by someone who owns the router. Did you try the commands I suggested on ath0 and eth0, with and without ‘sta’?

You can (temporarily) ‘sabotage’ the plugin to not use wlanconfig by changing line 163
Change

Code: Select all

test=$(which wlanconfig > /dev/null 2>&1)
To

Code: Select all

test=$(which notwlanconfig > /dev/null 2>&1)
(Or any other non existing commandname)

devros
Posts: 191
Joined: Saturday 29 October 2016 20:55
Target OS: -
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by devros » Monday 18 June 2018 22:19

Here is my output, looks like sta make no diference

Code: Select all

XM.v6.1.7# wlanconfig ath0 list
ADDR               AID FREQ TxRATE RxRATE RSSI   IDLE  TXSEQ RXSEQ  CAPS ACAPS      ERP  STATE HTCAPS
80:2a:a8:a4:e8:11   19 5500   117M   130M   38      0   3299 38672   Es             0      80b AP     WME ATH

Code: Select all

XM.v6.1.7# wlanconfig ath0 list sta
ADDR               AID FREQ TxRATE RxRATE RSSI   IDLE  TXSEQ RXSEQ  CAPS ACAPS      ERP  STATE HTCAPS
80:2a:a8:a4:e8:11   19 5500   117M   130M   38      0   1447 38672   Es             0      80b AP     WME ATH

Code: Select all

XM.v6.1.7# wlanconfig eth0 list
Invalid argument
XM.v6.1.7# wlanconfig eth0 list sta
Invalid argument

Thanks :-)

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Monday 18 June 2018 22:26

Weird ... maybe the router has additional interface names for wireless interfaces?
You can find all interfaces using ifconfig and try them one by one with the same command. Maybe the MAC addresses show up on an interface that is now filtered out by the plugin. The plugin assumes eth* wlan* and ath* as acceptable names, but there might be more.

devros
Posts: 191
Joined: Saturday 29 October 2016 20:55
Target OS: -
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by devros » Monday 18 June 2018 23:06

EscApe wrote:
Monday 18 June 2018 22:26
Weird ... maybe the router has additional interface names for wireless interfaces?
You can find all interfaces using ifconfig and try them one by one with the same command. Maybe the MAC addresses show up on an interface that is now filtered out by the plugin. The plugin assumes eth* wlan* and ath* as acceptable names, but there might be more.
im using router on my local network only for ethernet conection

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Tuesday 19 June 2018 7:15

Hi devros,

That explains a lot.. The plugin and the command line tools used is intended for wireless devices, not for ethernet. What is your use case for monitoring the presence of wired (probably stationary?) devices?

There's nothing wrong with the plugin. It is just not intended for ethernet connections. I will keep the wlanconfig command for other ubiquiti owners. To force the plugin to use arp you can sabotage the other commands as described in my earlier post. I might add a 'force arp' option in the future, but that won't be soon. The plugin already uses all available configuration parameters offered by the Domoticz plugin framework.

devros
Posts: 191
Joined: Saturday 29 October 2016 20:55
Target OS: -
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by devros » Tuesday 19 June 2018 9:22

EscApe wrote:
Tuesday 19 June 2018 7:15
Hi devros,

That explains a lot.. The plugin and the command line tools used is intended for wireless devices, not for ethernet. What is your use case for monitoring the presence of wired (probably stationary?) devices?

There's nothing wrong with the plugin. It is just not intended for ethernet connections. I will keep the wlanconfig command for other ubiquiti owners. To force the plugin to use arp you can sabotage the other commands as described in my earlier post. I might add a 'force arp' option in the future, but that won't be soon. The plugin already uses all available configuration parameters offered by the Domoticz plugin framework.
but i think that wlanconfig monitoring is not working for all ubiquiti users
My ubitiqui works as lan DHCP router connected to tenda (witch works as wifi)

OK thanks for you time... with modified code ARP works great.
you can maybe join some parameters to save position for force ARP switch :)

kofec
Posts: 35
Joined: Friday 16 September 2016 14:16
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by kofec » Tuesday 19 June 2018 13:01

Maybe it is worth to add notification to DHCP server:
for ubiquity:
https://community.ubnt.com/t5/EdgeRoute ... -p/2069562
for Openwrt:
https://jpmens.net/2013/10/21/tracking- ... h-dnsmasq/

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Tuesday 19 June 2018 20:22

@kofec

I’m not sure what you mean. What would you like this type of integration to do?

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Tuesday 19 June 2018 20:29

@devros

Combining parameters has crossed my mind and I will consider it for a future version. Don’t want to over complicate things though.
The plugin is meant to talk to wireless routers directly (tenda in your case). Does the tenda offer ssh?

Just curious: how long does it take until absence is detected after a device on your network has gone offline? ARP is unusable on my Asus, because it can take forever for it to notice that a device has disconnected (person holding it has left the house).

drwurn
Posts: 42
Joined: Sunday 10 June 2018 16:59
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by drwurn » Tuesday 19 June 2018 20:54

Are the instructions on the first page still actual and working?

Sjaakios
Posts: 24
Joined: Saturday 04 June 2016 23:41
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by Sjaakios » Tuesday 19 June 2018 20:56

Hi thanks for this scripts! Have been looking for good presence detection, it's always a struggle...

Had some issues getting the ssh key configured, fixed it based on the above comments (linking directly to the keyfile..). However ended up with the following error: 'onStart' failed 'AttributeError':'module 'Domoticz' has no attribute 'Status''.

Don't know if its my version of domoticz (v3.8153), or because i'm running it on my synology. However, fixed it by replacing the Domoticz.Status -> Domoticz.Log.

Thanks again, let's see if it's going to work :-)

CHeers,
Daan

Code: Select all

 2018-06-19 20:38:57.046 (iDetect) Initialized version 0.4.1, author 'ESCape'
2018-06-19 20:38:57.101 (iDetect) Debug log level set to: 'true'.
2018-06-19 20:38:57.101 (iDetect) Not running on Windows
2018-06-19 20:38:57.141 (iDetect) Checking if [ssh -V] will run: OK
2018-06-19 20:38:57.142 (iDetect) Monitoring {'phone1': 'CC:25:EF:40:7B:F8'} for presence.
2018-06-19 20:38:57.142 (iDetect) Checking router capabilities and wireless interfaces
2018-06-19 20:38:57.142 (iDetect) Using ssh public key authentication (~/.ssh/id_rsa.pub) for OS user running domoticz.
2018-06-19 20:38:57.142 (iDetect) Fetching data from router using ssh
2018-06-19 20:38:57.142 (iDetect) command: ['ssh', '-i', '/root/.ssh/id_rsa', '-o', 'ConnectTimeout=2', 'xx@xxx', '#!/bin/sh\n\t\t\t\t\texport PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH\n\t\t\t\t\ttest=$(which wl > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "wl@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(wl -i $iface assoclist > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\ttest=$(which iwinfo > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "iwinfo@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\\|^ath\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(iwinfo $iface assoclist > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\ttest=$(which wlanconfig > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "wlanconfig@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\\|^ath\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(wlanconfig $iface list > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\ttest=$(which arp > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "arp"\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\tif [ -f /proc/net/arp ]; then\n\t\t\t\t\t\tprintf "procarp"\n\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\tprintf none']
2018-06-19 20:38:57.852 (iDetect) Returncode ssh command: 0
2018-06-19 20:38:57.852 (iDetect) Output from router: b'wl@eth1 eth2 '
2018-06-19 20:38:57.853 (iDetect) Parsed data from router: ['wl', 'eth1 eth2 ']
2018-06-19 20:38:57.853 Error: (iDetect) 'onStart' failed 'AttributeError':'module 'Domoticz' has no attribute 'Status''.
2018-06-19 20:38:57.853 Error: (iDetect) ----> Line 479 in /usr/local/domoticz/var/plugins/idetect/plugin.py, function onStart
2018-06-19 20:38:57.853 Error: (iDetect) ----> Line 311 in /usr/local/domoticz/var/plugins/idetect/plugin.py, function onStart
2018-06-19 20:38:57.853 Error: (iDetect) ----> Line 198 in /usr/local/domoticz/var/plugins/idetect/plugin.py, function routercommand 

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Tuesday 19 June 2018 21:02

@drwurn

The poll interval is now a pull down choice and an option for a override button has been added. Those changes have not yet been added to the instruction. The basic setup is still accurate.
Do you ask because something is not working?

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Tuesday 19 June 2018 21:06

@Sjaakios

The Domoticz.Status has been added to the Domoticz plugin framework relatively recent (don’t know the exact version). The error will be gone if you update to the latest domoticz beta. Or you can stick with your work around. Logging levels are I bit less granular but that’s no problem (that’s how it was before the framework change).

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest