Python plugin: Presence detection from wireless router

Python and python framework
mvzut
Posts: 446
Joined: Thursday 12 November 2015 11:55
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands
Contact:

Python plugin: Presence detection from wireless router

Post by mvzut » Friday 12 October 2018 8:22

hjzwiers wrote:With Asus AImesh putting in both IP addresses does not work, still only sees the main router.

=arp fails, =brctl works

Just giving the info, I'll stick to brctl thnx
Is it easy/possible to temporarily switch off this AImesh feature? I suspect this is causing your problem.
On my system I have two routers (Asus and Linksys) just using the same SSID (but another channel), so no AImesh or anything. The plugin perfectly finds the correct HW methods on both routers, I don't need to force anything.
Raspberry Pi 3 - RFXtrx433 - Aeon Z-Stick gen5 - Opentherm Gateway - P1 smart meter - Netatmo - Philips Hue - ELV Max! - ESP8266 DIY water meter - Xiaomi Gateway - Sonos - IP cameras - Wall mounted tablet + Imperihome - Alexa - MANY switches/sensors

vamp
Posts: 12
Joined: Saturday 06 October 2018 13:20
Target OS: Linux
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by vamp » Friday 12 October 2018 12:53

Hello there,

i would like to use this plugin to Vyos Router.

it show arp table to "show arp" command

If i try to add it to pre configured command i get this:

2018-10-12 12:51:05.898 Error: (teszt) Unsupported command (pre)configured for 192.168.31.1: show
2018-10-12 12:52:47.549 Error: (teszt) Failed 3 times to get capabilities for 192.168.31.1. Retry interval automatically reduced for this router

How to use your plugin it this router?

mikeoo
Posts: 159
Joined: Sunday 22 March 2015 8:35
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Holland
Contact:

Re: Python plugin: Presence detection from wireless router

Post by mikeoo » Friday 12 October 2018 15:01

vamp wrote:
Friday 12 October 2018 12:53
Hello there,

i would like to use this plugin to Vyos Router.

it show arp table to "show arp" command

If i try to add it to pre configured command i get this:

2018-10-12 12:51:05.898 Error: (teszt) Unsupported command (pre)configured for 192.168.31.1: show
2018-10-12 12:52:47.549 Error: (teszt) Failed 3 times to get capabilities for 192.168.31.1. Retry interval automatically reduced for this router

How to use your plugin it this router?
It is better when you give the info you use in the plug-in.
Is SSH working to your router?

Try to use it like
Wifi router IP Address: <ip router>:<port number>=arp

Different SSH port
example: 192.168.31.1:8325=arp

Default SSH port
example: 192.168.31.1=arp

You only need to add the port number when you use a different port number for SSH to the router.

When it is not working make a screenshot of the plugin settings en mask the username and password ;)

vamp
Posts: 12
Joined: Saturday 06 October 2018 13:20
Target OS: Linux
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by vamp » Friday 12 October 2018 15:19

I found a other way :)

My Vyos VM run in a host (proxmox, Debian based) so i SSH to Proxmox and check brtcl. It working, but need some changes in python script.

I replace this line

from:

Code: Select all

 generic['brctl']=";brctl showmacs br0 | grep '..:..:..:..:..:..' | awk '{print $ 2}'"
to:

Code: Select all

generic['brctl']=";brctl showmacs vmbr0 | grep '..:..:..:..:..:..' | awk '{print $ 2}'"
It need, because Proxmox rename the normal bridge interface.

vmbr0 is the "LAN" virtual switch on Proxmox.

AurelioF
Posts: 29
Joined: Wednesday 03 October 2018 18:01
Target OS: Raspberry Pi
Domoticz version:
Location: Spain
Contact:

Re: Python plugin: Presence detection from wireless router

Post by AurelioF » Monday 15 October 2018 10:26

mikeoo wrote:
Friday 12 October 2018 7:20
AurelioF wrote:
Thursday 11 October 2018 15:10
I confirm that it works with Unifi AP Pro :D
Thanks for sharing
Do you use ARP with the Unifi. I use also Unifi and interesting how you use it.
My installation has two unifi devices, with a single ssid but two IP addresses.
the plugin is working on raspberry. I'm only using this plugin, I do not perform other ARP queries.
I have to say that sometimes I experience repeated reconnections on an iphone for no apparent reason, I am still investigating.

mikeoo
Posts: 159
Joined: Sunday 22 March 2015 8:35
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Holland
Contact:

Re: Python plugin: Presence detection from wireless router

Post by mikeoo » Monday 15 October 2018 10:38

AurelioF wrote:
Monday 15 October 2018 10:26
mikeoo wrote:
Friday 12 October 2018 7:20
AurelioF wrote:
Thursday 11 October 2018 15:10
I confirm that it works with Unifi AP Pro :D
Thanks for sharing
Do you use ARP with the Unifi. I use also Unifi and interesting how you use it.
My installation has two unifi devices, with a single ssid but two IP addresses.
the plugin is working on raspberry. I'm only using this plugin, I do not perform other ARP queries.
I have to say that sometimes I experience repeated reconnections on an iphone for no apparent reason, I am still investigating.
I have an Edgerouter X and 3 Unifi AP but i can only use the Edgerouter for scanning the ip and app. Can you scan the AP directly for an ip ?

For 95% the plugin is working great. But sometimes a phone (android) is not at home but system still think it is, even after a few hours.
Reconnection i have also sometimes. Some day it is working great an then suddenly it disconnects for a few minutes and the comes back again.

Would Be nice if there was a better option then ARP with Unifi so we could kill this sort of problems.

AurelioF
Posts: 29
Joined: Wednesday 03 October 2018 18:01
Target OS: Raspberry Pi
Domoticz version:
Location: Spain
Contact:

Re: Python plugin: Presence detection from wireless router

Post by AurelioF » Monday 15 October 2018 12:16

mikeoo wrote:
Monday 15 October 2018 10:38
AurelioF wrote:
Monday 15 October 2018 10:26
mikeoo wrote:
Friday 12 October 2018 7:20


Do you use ARP with the Unifi. I use also Unifi and interesting how you use it.
My installation has two unifi devices, with a single ssid but two IP addresses.
the plugin is working on raspberry. I'm only using this plugin, I do not perform other ARP queries.
I have to say that sometimes I experience repeated reconnections on an iphone for no apparent reason, I am still investigating.
I have an Edgerouter X and 3 Unifi AP but i can only use the Edgerouter for scanning the ip and app. Can you scan the AP directly for an ip ?

For 95% the plugin is working great. But sometimes a phone (android) is not at home but system still think it is, even after a few hours.
Reconnection i have also sometimes. Some day it is working great an then suddenly it disconnects for a few minutes and the comes back again.

Would Be nice if there was a better option then ARP with Unifi so we could kill this sort of problems.
I access the unifi by IP and ssh using the plugin, I guess your unifi team must allow access by ssh, my unifi is "unifi ap pro"

mikeoo
Posts: 159
Joined: Sunday 22 March 2015 8:35
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Holland
Contact:

Re: Python plugin: Presence detection from wireless router

Post by mikeoo » Monday 15 October 2018 12:45

AurelioF wrote:
Monday 15 October 2018 12:16
mikeoo wrote:
Monday 15 October 2018 10:38
AurelioF wrote:
Monday 15 October 2018 10:26


My installation has two unifi devices, with a single ssid but two IP addresses.
the plugin is working on raspberry. I'm only using this plugin, I do not perform other ARP queries.
I have to say that sometimes I experience repeated reconnections on an iphone for no apparent reason, I am still investigating.
I have an Edgerouter X and 3 Unifi AP but i can only use the Edgerouter for scanning the ip and app. Can you scan the AP directly for an ip ?

For 95% the plugin is working great. But sometimes a phone (android) is not at home but system still think it is, even after a few hours.
Reconnection i have also sometimes. Some day it is working great an then suddenly it disconnects for a few minutes and the comes back again.

Would Be nice if there was a better option then ARP with Unifi so we could kill this sort of problems.
I access the unifi by IP and ssh using the plugin, I guess your unifi team must allow access by ssh, my unifi is "unifi ap pro"
I have one UniFi AP-AC-LR and two UniFi AP-AC-Lite but as fas as i know iu can only be managed with Unify Controller or with the Android App but nut directly. I can SSH to the Edgerouter with putty but i need to sort out how to create SSH key for it. That part i have no experience in so but difficult for the moment. And don't know if that part gowning to work and if it fix the problem.

So if you one a share some info for how you did it :mrgreen: But maby the Pro works different.

AurelioF
Posts: 29
Joined: Wednesday 03 October 2018 18:01
Target OS: Raspberry Pi
Domoticz version:
Location: Spain
Contact:

Re: Python plugin: Presence detection from wireless router

Post by AurelioF » Monday 15 October 2018 13:18

mikeoo wrote:
Monday 15 October 2018 12:45
AurelioF wrote:
Monday 15 October 2018 12:16
mikeoo wrote:
Monday 15 October 2018 10:38


I have an Edgerouter X and 3 Unifi AP but i can only use the Edgerouter for scanning the ip and app. Can you scan the AP directly for an ip ?

For 95% the plugin is working great. But sometimes a phone (android) is not at home but system still think it is, even after a few hours.
Reconnection i have also sometimes. Some day it is working great an then suddenly it disconnects for a few minutes and the comes back again.

Would Be nice if there was a better option then ARP with Unifi so we could kill this sort of problems.
I access the unifi by IP and ssh using the plugin, I guess your unifi team must allow access by ssh, my unifi is "unifi ap pro"
I have one UniFi AP-AC-LR and two UniFi AP-AC-Lite but as fas as i know iu can only be managed with Unify Controller or with the Android App but nut directly. I can SSH to the Edgerouter with putty but i need to sort out how to create SSH key for it. That part i have no experience in so but difficult for the moment. And don't know if that part gowning to work and if it fix the problem.

So if you one a share some info for how you did it :mrgreen: But maby the Pro works different.
unifi ap pro has native SSH access, it is only necessary to configure user and password, it seems that unifi AP-AC-Lite and AP-AC-LR do not have ssh access, I did not know this difference, this can be a problem for your solution.
I do not know how to manage Edgerouter, if possible create a plain text key in your Edgerouter, using certificates is safer, but it is much more difficult.

if you can use a plain text password, make sure you have installed sshpass (sudo apt-get install sshpass). in Domoticz

mikeoo
Posts: 159
Joined: Sunday 22 March 2015 8:35
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Holland
Contact:

Re: Python plugin: Presence detection from wireless router

Post by mikeoo » Monday 15 October 2018 14:27

AurelioF wrote:
Monday 15 October 2018 13:18
mikeoo wrote:
Monday 15 October 2018 12:45
AurelioF wrote:
Monday 15 October 2018 12:16


I access the unifi by IP and ssh using the plugin, I guess your unifi team must allow access by ssh, my unifi is "unifi ap pro"
I have one UniFi AP-AC-LR and two UniFi AP-AC-Lite but as fas as i know iu can only be managed with Unify Controller or with the Android App but nut directly. I can SSH to the Edgerouter with putty but i need to sort out how to create SSH key for it. That part i have no experience in so but difficult for the moment. And don't know if that part gowning to work and if it fix the problem.

So if you one a share some info for how you did it :mrgreen: But maby the Pro works different.
unifi ap pro has native SSH access, it is only necessary to configure user and password, it seems that unifi AP-AC-Lite and AP-AC-LR do not have ssh access, I did not know this difference, this can be a problem for your solution.
I do not know how to manage Edgerouter, if possible create a plain text key in your Edgerouter, using certificates is safer, but it is much more difficult.

if you can use a plain text password, make sure you have installed sshpass (sudo apt-get install sshpass). in Domoticz
tnx that is different indeed. For now i am working with just the password and sshpass indeed and that is working but with sometimes a problem. But wil search for hopefully other option witch works better in the future.

jackjhon
Posts: 2
Joined: Tuesday 16 October 2018 12:56
Target OS: Linux
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by jackjhon » Tuesday 16 October 2018 13:03

How do we use asp plugin to Optus router?
Optus Technical Support Number

jackjhon
Posts: 2
Joined: Tuesday 16 October 2018 12:56
Target OS: Linux
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by jackjhon » Tuesday 16 October 2018 13:37

how do we use asp plugging to my republic broadband?
Myrepublic internet support

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Tuesday 16 October 2018 17:22

@jackjhon

Ik you can supply a router make and model, then maybe someone can help you.
In general: if the router supports ssh just follow the instructions on GitHub. If it doesn't the plugin will not work with your router.

Freekers
Posts: 19
Joined: Friday 20 October 2017 17:36
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by Freekers » Saturday 27 October 2018 0:04

This plugin is exactly what I need! Sadly it's not compatible with my router :( My router supports SSH, but the commands your script is trying to execute are not found. Debug output is as follows:

Code: Select all

2018-10-26 22:15:06.843 (iDetect WiFi Presence Detection) Debug logging mask set to: PYTHON
2018-10-26 22:15:06.843 (iDetect WiFi Presence Detection) Not running on Windows
2018-10-26 22:15:06.889 (iDetect WiFi Presence Detection) The OS user profile running domoticz is:	root
2018-10-26 22:15:06.976 (iDetect WiFi Presence Detection) Checking if [ssh -V] will run: OK
2018-10-26 22:15:06.977 (iDetect WiFi Presence Detection) Monitoring {'Freek': {'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None, 'ignore': False}, 'Test': {'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None, 'ignore': False}} for presence.
2018-10-26 22:15:06.977 (iDetect WiFi Presence Detection) 2 of them will control the Anyone home switch
2018-10-26 22:15:06.977 (iDetect WiFi Presence Detection) Router configuration:192.168.88.1
2018-10-26 22:15:06.977 (iDetect WiFi Presence Detection) Fetching data from 192.168.88.1 using: ssh -i /var/services/homes/Freek/.ssh/id_rsa -o ConnectTimeout=3 -p22 domoticz@192.168.88.1
2018-10-26 22:15:06.977 #!/bin/sh
2018-10-26 22:15:06.977 export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH
2018-10-26 22:15:06.977 type wl > /dev/null 2>&1 && printf "~wl"
2018-10-26 22:15:06.977 type iwinfo > /dev/null 2>&1 && printf "~iwinfo"
2018-10-26 22:15:06.977 type wlanconfig > /dev/null 2>&1 && printf "~wlanconfig"
2018-10-26 22:15:06.977 type qcsapi_sockrpc > /dev/null 2>&1 && printf "~qcsapi_sockrpc"
2018-10-26 22:15:06.977 type brctl > /dev/null 2>&1 && printf "~brctl"
2018-10-26 22:15:06.977 type arp > /dev/null 2>&1 && printf "~arp"
2018-10-26 22:15:06.977 [ -f /proc/net/arp ] && printf "~procarp"
2018-10-26 22:15:06.977 exit
2018-10-26 22:15:06.977
2018-10-26 22:15:08.066 (iDetect WiFi Presence Detection) ssh command on 192.168.88.1 returned:b'expected end of command (line 1 column 8)\nexpected end of command (line 1 column 9)\nexpected end of command (line 1 column 13)\nexpected end of command (line 1 column 17)\nexpected end of command (line 1 column 21)\nexpected end of command (line 1 column 12)\nexpected end of command (line 1 column 10)\nbad command name -f (line 1 column 3)\n'
2018-10-26 22:15:08.066 (iDetect WiFi Presence Detection) SSH command on 192.168.88.1 took 89 milliseconds.
2018-10-26 22:15:08.066 (iDetect WiFi Presence Detection) Available commands on 192.168.88.1:['xpected end of command (line 1 column 8)\nexpected end of command (line 1 column 9)\nexpected end of command (line 1 column 13)\nexpected end of command (line 1 column 17)\nexpected end of command (line 1 column 21)\nexpected end of command (line 1 column 12)\nexpected end of command (line 1 column 10)\nbad command name -f (line 1 column 3)\n']
2018-10-26 22:15:08.066 (iDetect WiFi Presence Detection) Found suitable command (and interfaces) for router 192.168.88.1:{}
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) Could not construct router query command for 192.168.88.1
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) Router initialized as:{'192.168.88.1': {'port': 22, 'errorcount': 1, 'cmd': '', 'initialized': False, 'prospone': datetime.datetime(2018, 10, 26, 22, 15, 20, 67085), 'user': 'domoticz'}}
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) iDetect WiFi Presence Detection - Test is stil in use
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) iDetect WiFi Presence Detection - Freek is stil in use
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) Devicenames and their Domoticz (subdevice) index: {'Freek': 5, 'Test': 4}
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) MAC addresses to monitor: {'Freek': {'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None, 'ignore': False}, 'Test': {'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None, 'ignore': False}}
2018-10-26 22:15:08.067 (iDetect WiFi Presence Detection) Plugin initialization done.
2018-10-26 22:15:16.329 (iDetect WiFi Presence Detection) Prosponed connection to 192.168.88.1 for 0:00:03.737255
My router make and model is: Mikrotik RouterBoard RB962UiGS-5HacT2HnT hAP AC

Of the commands your script is trying to execute, only the ARP command is available but has the following syntax:

Code: Select all

ip arp print
Do you think it would be possible to make the script compatible with my router? How can I help?

Thank you!

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Saturday 27 October 2018 9:32

@Freekers

It seems the routeros on the MikroTik offers ssh for it's own configuration environment, but no real linux/busybox command line. If you can find a command that will produce a clean list of mac addresses then i can add that to the plugin as a preconfigured command.

After some searching on the MikroTik website i suspect that the router does not have the 'awk' command needed to filter only the mac-addresses from the arp table. Can you find another way to get a clean list of mac addresses from the routeros command line?

Maybe something like:

Code: Select all

ip arp print column MAC_ADDRESS
(taken some pointers from the MikroTik website, but i have no idea if this combination will work)

The router also seems to offer scripting capabilities, so maybe you can cook up a script for routeros that will produce a clean list of mac addresses(?)

Auto detection of the router type by the plugin will not work because the (current) router side script expects some linux/busbox command to be available. Not just a proprietary set of tools. That's not a huge problem though, since the plugin already offers an option to force a preconfigured command. I will simply add such a command for the MikroTik to the plugin. If you can come up with a safe way to detect routerOS as the router type (without the test crashing the detection script on other router types), then i can even have a look at auto detection.

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Saturday 27 October 2018 13:22

@Freekers

I have an idea to support custom commands in the plugin and not assume the output is always formatted correctly for the plugin. This way support for MikroTik and other obscure routers/accespoints can be easily added.

Will publish a new version today or at least this weekend (depending on the test results and effort ;-)

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Saturday 27 October 2018 17:33

**version 0.7.4**
- Changed: different approach to parse mac addresses from (potentially unformatted) router data
- Added: Custom command for MikroTik RouterOS (configure router address in the plugin as: <routerip>=routeros eg. 192.168.0.1=routeros).
Note: routeros uses the arp detection method but with a different syntax and the mac address parsing is handled by the plugin instead of the router side script.

dextm80
Posts: 137
Joined: Tuesday 24 October 2017 18:32
Target OS: Linux
Domoticz version: 4.10159
Contact:

Re: Python plugin: Presence detection from wireless router

Post by dextm80 » Sunday 28 October 2018 1:15

i guys, sorry but if i have fritzbox that not support telnet i can't use this plugin?

tnx
Domoticz on AsRock j3455-ITX 8gb ram - Aeotec ZWave Usb Stick - RFLink 433Mhz
1x Fibaro Wall Plug
1x Fibaro Motion Sensor
x NeoCoolcam Wall Plug
Netatmo Weather Station - Netatmo Thermostat
Philips Hue Bridge

Freekers
Posts: 19
Joined: Friday 20 October 2017 17:36
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: Python plugin: Presence detection from wireless router

Post by Freekers » Sunday 28 October 2018 1:49

EscApe wrote:
Saturday 27 October 2018 13:22
@Freekers

I have an idea to support custom commands in the plugin and not assume the output is always formatted correctly for the plugin. This way support for MikroTik and other obscure routers/accespoints can be easily added.

Will publish a new version today or at least this weekend (depending on the test results and effort ;-)
Wow, this is awesome! I've just tested the new version and I can confirm that it works :mrgreen: Great work, thank you very much for the quick fix!!

Do you still need more debug information to further optimize the script? Since it's based on the arp table, it takes a while for the MAC address to get removed from the arp table after the connection has been dropped/disconnected. Would it help to force a refresh of the arp table after every query, like so? https://forum.mikrotik.com/viewtopic.php?t=11030

EscApe
Posts: 230
Joined: Thursday 02 April 2015 8:46
Target OS: Raspberry Pi
Domoticz version: v3+beta
Location: The Netherlands
Contact:

Re: Python plugin: Presence detection from wireless router

Post by EscApe » Sunday 28 October 2018 9:54

Glad it's working :D

You might be able to tweak arp table management on the routerboard. Maybe set some time-outs(?) I don't know the platform, but it seems well documented and very tweakable. Refreshing the arp table on every polls seems a bit drastic. The plugin might miss mac addresses if the table isn't refreshed quick enough.

Querying the wireless interfaces will probably work much better than the generic arp command. You could investigate how to get the associated mac addresses from the wireless chipset. That is where the plugin is at is best.

Another question to really support RouterOS in the plugin is how to detect the routerboard as a routerboard with a command that will consistently identify the routerboard and that will consistently fail on other routers/firmwares. That way it can be included in the auto detection mechanism of the plugin.
Eg.
To check if a router has a broadcom chipset the plugin will try to run: 'type wl' on the router. type will return an error code if the wl command is not available. The plugin uses this info (and more checks like that) to identify the router. Since routeros offers no real linux-like shell the 'type'' test will not work.

Post Reply

Who is online

Users browsing this forum: moroen and 3 guests