Yet another presence detection method

Wob76
Posts: 65
Joined: Wednesday 19 April 2017 6:31
Target OS: Linux
Domoticz version:
Contact:

Re: Yet another presence detection method

Post by Wob76 » Monday 21 August 2017 8:30

Hi,

Thanks for your work, I have it in and running.

Just wondering how everyone is handling the Presence switches in Domoticz?

I wanted to try and simplify adding extra devices later for monitoring in the future, so I created a Presence script that toggles a "Occupied" switch if any of devices ending in "Phone" are on.

This occupied switch is then used in my heating\cooling scripts to enable\disable overnight monitoring, along with a selector switch with Off\On\Auto. So I can override the presence sensing.

I have another switch called "Guests" which is toggled by devices that end in "Phone (Guest)". I plan to add a few people who regularly stay in our guest room, this will then control the heating zone for that room.

So my new Presence script runs once a min and seems to be working as expected, but I have since had a thought if I should just be using Sub\Slave switches.

Regards,
Wob

Toyman
Posts: 35
Joined: Wednesday 31 August 2016 16:17
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Yet another presence detection method

Post by Toyman » Monday 18 September 2017 10:49

Thanks A LOT. The ONLY presence script that actually works with iPhones

DAVIZINHO
Posts: 66
Joined: Sunday 27 August 2017 18:00
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: Yet another presence detection method

Post by DAVIZINHO » Thursday 05 October 2017 22:35

edit, no problem. works great!!!

freakshock
Posts: 56
Joined: Friday 14 April 2017 13:39
Target OS: NAS (Synology & others)
Domoticz version:
Location: The Netherlands
Contact:

Re: Yet another presence detection method

Post by freakshock » Thursday 12 October 2017 20:40

This is a great method! Works great for my Android and iPhone devices.
I would also like to add some wired devices via this method.
Not as a presence detection function of course, but to see if these devices are online.

Is there any way to modify the code to also look for wired MAC adresses?
My skills are lacking here :lol:

WyldeBill
Posts: 4
Joined: Monday 01 December 2014 21:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Yet another presence detection method

Post by WyldeBill » Thursday 02 November 2017 1:25

Hi there, great script and I have got it working on the 2.5Ghz interface of my ASUS AC87U router. However the AC87U does not report the 5Ghz interface as eth2 and therefore the wl command fails. Initial investigation indicates that this is because the 5Ghz interface is not provided via the Broadcom chip (which supports the wl command).

Was wondering if anyone had managed to get this working on the ASUS AC87 router, it appears to need to use the qcsapi client.

Thanks in advance

WB

WyldeBill
Posts: 4
Joined: Monday 01 December 2014 21:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Yet another presence detection method

Post by WyldeBill » Sunday 05 November 2017 20:46

I have now had time to investigate detecting devices on the 5Ghz interface for the RT-AC87U router. I have an amended section of script to replace the 5Ghz section in the original script. This may not be the most elegant of scripts as Shell scripts are not my forte, however it works - please feel free to improve.

Hope this helps someone else

ATB

WB

Code: Select all

# look in 5GHz (wifi0) network for device

node_idx=0
interface=wifi0
nodemax=20
loopend=QCS
inc=1

#loops through upto 20 devices on the interface 
until [ $node_idx -gt $nodemax ] 

do
   
    #get_associated_device_mac_addr returns a mac adress for each node of the interface 
    for x in `/usr/sbin/qcsapi_sockrpc get_associated_device_mac_addr $interface $node_idx | awk '{print $1}'`; do

	    if [ $x = $mac ]; then
        	
		device_present=On

	    else
		
		#when the end of the wifi devices is reached QCS is returned so stop looping
	        if [ $x = $loopend ]; then

		   node_idx=$nodemax

		fi
	    
	    fi
	done

    #increment the node id
    node_idx=`expr $node_idx + $inc`

    done



rrozema
Posts: 10
Joined: Thursday 26 October 2017 13:37
Target OS: Raspberry Pi
Domoticz version: beta
Location: Delft
Contact:

Re: Yet another presence detection method

Post by rrozema » Friday 17 November 2017 13:47

I've been experimenting and found this detection method to work almost as I want it. I have made a few adjustments and need some help making some more.

Like many others I've got multiple wifi access points and thus detecting a phone on either the 2.4 or 5GHz interfaces doesn't work well, I'd have to run scripts on all 3 access points. My network however is the same over all access points (i.e. my devices are all in the same ip-range, getting an address from the same dhcp server while roaming the access points). To fix this issue I've replaced running wl against the wifi interfaces by looking up the mac address in the arp cache on bridge br0 in my main router. This way the mac address will be found when it is on either 2.4GHz, 5GHz or on the wired network (from another access point). In my RT-AC66U (merlin 380.68_4) arp entries time out after 2 minutes, which results in an acceptable delay for turning off my presence of 2 to 3 minutes. The switch will turn On at most a minute after my phone connects to wifi because of the 1 minute schedule, and Off within 2 to 3 minutes after my phone disconnects. I have no idea how this change affects "sleeping" iphone detection, but for me it works like a charm. I'll post my revised script below. Because arp returns the mac address somewhat differently formatted and I have little experience in scripting in these languages, you have to pass the MAC address formatted like AB:CD:EF:01:02, so using capitals and with : instead of - in between the hex pairs. My domoticz is accessible from my lan without user name and passw, so I don't have to specify this in the url's. If your domoticz is configured differently, adjust the script accordingly.

Code: Select all

#!/bin/sh

mac=$1
idx=$2
device_present=Off
domoticz_status=`curl -s "http://192.168.128.100:8080/json.htm?type=devices&rid=$idx" | grep Status | awk '{print $3}' | sed 's/[",]//g'`

# look in bridge (br0) network for device
for x in `arp -i br0 -na | awk '{print $4}'`; do
echo "x: $x, mac: $mac"
 if [ $x = $mac ]; then
        device_present=On
 fi
done

echo "Domoticz: $domoticz_status, router: $device_present"

# tell domoticz the new device status
if [ $domoticz_status != $device_present ]; then
 echo "New status $device_present"
 curl -s "http://192.168.128.100:8080/json.htm?type=command&param=switchlight&idx=$idx&switchcmd=$device_present" > /dev/null
 fi
fi
This is how far I've gotten and what I wanted to share with you. Now the part that I need some assistance on to make it work even better 8-) :

As I've got at least 6 devices I need to configure, plus the fact that the rate at which people change phones at my home is rather high, I would like to make the entire process a little bit more maintenance friendly by having to set up the script on the router only once and later on I only need to create more switches, remove switches or modify existing switches in Domoticz to change which devices are being monitored.

In order to do this I want to adjust the presence_detection script such that it first queries domoticz for all devices ( selects only those which name starts with 'Presence' and for each of these reads both the "idx" value plus the "Description" value for the devices' json returned by domoticz. "idx" of course holds the switches idx value, required to get and set the switches Status. "Description" I fill with the mac address of the device I need monitored. I did find a way to find and loop over the device names, but for retrieving the "idx" and "Description" values I realy need to parse the json structure returned by domoticz and I don't know how to do that using just shell commands. If anyone has a suggestion or maybe some example code on how to parse the both values from the json I'd be much obliged.

Code: Select all

#!/bin/sh

mac=$1
idx=$2
for device in `curl -s "http://192.168.128.100:8080/json.htm?type=devices" | grep '"Name".*"Presence' | awk -F'"' '{print $4}'`; do
   echo "Presence device: $device"
   # to do: read the idx and Description values, then query each switch and if needed set the switch accordingly.
done
And here's an example of a single device of which there are multiple returned by the "devices" query to domoticz. I need both the value for "idx" and "Description" for each of these:

Code: Select all

      {
         "AddjMulti" : 1.0,
         "AddjMulti2" : 1.0,
         "AddjValue" : 0.0,
         "AddjValue2" : 0.0,
         "BatteryLevel" : 255,
         "CustomImage" : 0,
         "Data" : "Off",
         "Description" : "DC:0B:34:82:AF:1B",
         "Favorite" : 0,
         "HardwareID" : 9,
         "HardwareName" : "Dummy",
         "HardwareType" : "Dummy (Does nothing, use for virtual switches only)",
         "HardwareTypeVal" : 15,
         "HaveDimmer" : true,
         "HaveGroupCmd" : true,
         "HaveTimeout" : false,
         "ID" : "000140A8",
         "Image" : "Light",
         "IsSubDevice" : false,
         "LastUpdate" : "2017-11-17 12:40:01",
         "Level" : 0,
         "LevelInt" : 0,
         "MaxDimLevel" : 100,
         "Name" : "Presence Richard",
         "Notifications" : "false",
         "PlanID" : "0",
         "PlanIDs" : [ 0 ],
         "Protected" : false,
         "ShowNotifications" : true,
         "SignalLevel" : "-",
         "Status" : "Off",
         "StrParam1" : "",
         "StrParam2" : "",
         "SubType" : "Switch",
         "SwitchType" : "On/Off",
         "SwitchTypeVal" : 0,
         "Timers" : "false",
         "Type" : "Light/Switch",
         "TypeImg" : "lightbulb",
         "Unit" : 1,
         "Used" : 1,
         "UsedByCamera" : false,
         "XOffset" : "0",
         "YOffset" : "0",
         "idx" : "88"
      }
EDIT: Although this script turns the swithces on and off, after some time of usage I don't think it is doing exactly what I want it to do: My phones sometimes do not go into the internet for periods longer than 2 minutes, resulting in their arp entries being cleared from the router's arp cache. Every time this happens, my presence switch is turned off, even though I'm still at home. i.e. this method is not reliable enough for me. I'll be looking for another method still.
Last edited by rrozema on Friday 01 December 2017 22:10, edited 1 time in total.

User avatar
galadril
Posts: 582
Joined: Monday 07 September 2015 10:32
Target OS: Raspberry Pi
Domoticz version: latest
Contact:

Re: Yet another presence detection method

Post by galadril » Friday 17 November 2017 21:37

rrozema wrote:
Friday 17 November 2017 13:47
I've been experimenting and found this detection method to work almost as I want it. I have made a few adjustments and need some help making some more.
Great work richard

rrozema wrote:
Friday 17 November 2017 13:47
And here's an example of a single device of which there are multiple returned by the "devices" query to domoticz. I need both the value for "idx" and "Description" for each of these:

There is jq for parsing json on the command line:
jq '.Body'

so for the values it would be:
jq '.result[0].idx'
jq '.result[0].Description'

Visit this for jq: https://stedolan.github.io/jq/
they even have an online tester:

https://jqplay.org/


hope that this assists you in your quest ;-)
see you on work!
Solar panels of Ginlong, Omnik-Solar, Transenergy or Solarman?? Try my Android app:
https://play.google.com/store/apps/deta ... ongmonitor

woody4165
Posts: 503
Joined: Monday 14 March 2016 14:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Yet another presence detection method

Post by woody4165 » Monday 27 November 2017 18:15

Sorry for my ignorance, this nice script requests two parameters, Mac address and idx
In the script I see also $3 and $4

What they refer to?

When I run it I get, as result of

Code: Select all

for x in `arp -i br0 -na | awk '{print $4}'`; do
echo "x: $x, mac: $mac"
I get

Code: Select all

x: entries, mac: xx:xx:xx:xx:xx:xx
What entries stand for?

Thanks

rrozema
Posts: 10
Joined: Thursday 26 October 2017 13:37
Target OS: Raspberry Pi
Domoticz version: beta
Location: Delft
Contact:

Re: Yet another presence detection method

Post by rrozema » Friday 01 December 2017 22:04

woody4165 wrote:
Monday 27 November 2017 18:15
Sorry for my ignorance, this nice script requests two parameters, Mac address and idx
In the script I see also $3 and $4

What they refer to?
Those $3 and $4 are not script parameters, those are parameters to the awk command. They represent the 3rd and 4th "word" from the input given to awk. What "arp -i br0 -na | awk '{print $4}'" does:
- execute arp -i br0 -na
- take the output and "pipe" it into the next command (using the "pipe" command or |-symbol)
- awk '{print $4}' then takes the 4th word from it's input and outputs that

Run the individual parts of the commands from your router's cli and you'll see what exactly is going on.
When I run it I get, as result of

Code: Select all

for x in `arp -i br0 -na | awk '{print $4}'`; do
echo "x: $x, mac: $mac"
I get

Code: Select all

x: entries, mac: xx:xx:xx:xx:xx:xx
What entries stand for?

Thanks
Apparently, in your case the word "entries" was the 4th word output by arp -i br0 -na. Can you run said command in your router and see what exactly the output is that makes it return this word? For me the output is multiple lines like these:

Code: Select all

? (192.168.128.20) at D0:66:78:79:68:52 [ether]  on br0
? (192.168.128.28) at 30:C5:B8:EE:6A:32 [ether]  on br0
? (192.168.128.22) at 4C:49:7D:AF:B7:C9 [ether]  on br0
I suspect you're getting some error message instead.

But the text output here isn't that important, this print statement is more like some debugging, it shows you what texts are found from the arp output. The next if-statement matches the word found with the mac address you've specified, and only if this matches the code is activated. All entries that don't match are simply ignored.

If it isn't an error message, it could be that you have a different firmware version in your router, resulting in different output from the arp command. My firmware version is ASUSWRT-Merlin RT-AC66U 380.68-4 Wed Oct 4 19:01:30 UTC 2017. What is yours? (It is shown when I log on using telnet/ssh)

woody4165
Posts: 503
Joined: Monday 14 March 2016 14:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Yet another presence detection method

Post by woody4165 » Saturday 02 December 2017 8:39

Thank for your reply.
When I run the arp command I get
arp: in 36 entries no match found

What can be my issue?

rrozema
Posts: 10
Joined: Thursday 26 October 2017 13:37
Target OS: Raspberry Pi
Domoticz version: beta
Location: Delft
Contact:

Re: Yet another presence detection method

Post by rrozema » Saturday 02 December 2017 11:11

woody4165 wrote:
Saturday 02 December 2017 8:39
Thank for your reply.
When I run the arp command I get
arp: in 36 entries no match found

What can be my issue?
Are you running the arp command in your router or in your raspberry? You should be running it in your router. Try running

Code: Select all

ifconfig
to list all available interfaces. In my router I have a bridge over all interfaces, called br0 (with a zero at the end). This one is particulary handy for this purpose as it includes both all wifi interfaces and the wired interfaces. i.e. wether the call came in via one of the wifi tranceivers or from another access point via the LAN connectors, all come through this bridge interface. The first column returned by ifconfig lists the interface names. This is the name that you can use with the -i parameter for arp.

woody4165
Posts: 503
Joined: Monday 14 March 2016 14:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Yet another presence detection method

Post by woody4165 » Saturday 02 December 2017 15:21

What do you mean by running arp in my router?
I have no access (telnet or ssh) to my Fritz router.

With ifconfig I get

Code: Select all

bond0     Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx 
          UP BROADCAST MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx   
          inet addr:192.168.xxx.xxx  Bcast:192.168.xxx.xxx  Mask:255.255.xxx.xxx
          inet6 addr: xx:xx:xx:xx:xx:xx /64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:932237 errors:0 dropped:259086 overruns:0 frame:0
          TX packets:954891 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:81454906 (77.6 MiB)  TX bytes:1121825048 (1.0 GiB)
          Interrupt:51 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:223401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:223401 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:27416526 (26.1 MiB)  TX bytes:27416526 (26.1 MiB)

wlan0     Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx   
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

rrozema
Posts: 10
Joined: Thursday 26 October 2017 13:37
Target OS: Raspberry Pi
Domoticz version: beta
Location: Delft
Contact:

Re: Yet another presence detection method

Post by rrozema » Saturday 02 December 2017 23:06

woody4165 wrote:
Saturday 02 December 2017 15:21
What do you mean by running arp in my router?
I have no access (telnet or ssh) to my Fritz router.
The script needs to be run on your router, via telnet or ssh. That's the idea of the methods presented in this thread: your router is in the path between your phones and the internet, so it can be used to monitor your phone's activities on the network and alert domoticz when a state changes. Various methods to monitor the presence of the phone have been presented, but they all only work correctly when run via telnet or ssh from your router. If you don't have access to your router's cli via telnet or ssh this method is of no use to you.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests