Connecting Plugwise-2-py via MQTT to Domoticz

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Wednesday 02 December 2015 14:17

I have been looking into setting the switch-state of a circle to domoticz, and switching it with domoticz using the on/off actions. As a starting point I took the Nov. 5th node-red scripts of bloemkool.

Where I stand:
- Virtual switches will be added automatically to the devices tab. Only enabled/use those who are actually used to switch. This is similar to automatically added energy-meters for the circles.
- Two way communication: Plugwise-2-web app can switch and update domoticz, and domoticz can switch the circle.
- This design causes infinite update loops, which I have stopped.
- Stopping the infinite loop can have a side effect in rare race-condition cases: When switched externally (e.g. through plugwise-w-web app), domoticz UI switches are not forwarded, but discarded for two seconds. This can result in domoticz being wrong in the actual state of the switch.

I do not have yet the killer idea to solve this. There is always a chance of the device getting an external update (due to external switching, plugwise schedule switching, a state-update heartbeat), and there is always the chance that at the same time the switch is operated form the domoticz UI(s). In the end, domoticz and plugwise need to agree on the actual switch state.

See also: viewtopic.php?f=11&t=2908&p=63910#p63910

SevenW

dorenberg
Posts: 62
Joined: Monday 22 June 2015 20:18
Target OS: Raspberry Pi
Domoticz version: 3.4984
Location: Veghel, The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby dorenberg » Thursday 03 December 2015 22:03

SevenW, good that there is still good progress in this topic. I think the best option is to try and add the source code into domoticz so it can function independently, but maybe another option is still better......

bloemkool
Posts: 10
Joined: Thursday 05 November 2015 12:21
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby bloemkool » Friday 04 December 2015 15:52

SevenW wrote:I have been looking into setting the switch-state of a circle to domoticz, and switching it with domoticz using the on/off actions. As a starting point I took the Nov. 5th node-red scripts of bloemkool.


Note... there is a config error in that/my nodered config.
In the power readout function there is a powerJSON.mac.substr(10,6), so 6 chars after the tenth char. In the switch/schedule function is a msg.payload.mac used together with a part of the mac adress ...What you fill in the script://-line in domoticz is a 7 digit part of the mac-address. So actually the power readout function should contain powerJSON.mac.substr(9,7).
This is because of a replaced circle and that replacement uses a "larger" mac address. So working with an complete MAC address is always favorable.

An excerpt from the pw-conf.json makes that very clear :

Code: Select all

{"mac": "000D6F000278F3A1", "category": "computer", "name": " ...
{"mac": "000D6F000038BE8A", "category": "EV", "name": "Volvo" ...
{"mac": "000D6F000038D3A4", "category": "witgoed", "name": "d ...
{"mac": "000D6F000038C3AF", "category": "witgoed", "name": "k ...

bloemkool
Posts: 10
Joined: Thursday 05 November 2015 12:21
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby bloemkool » Friday 04 December 2015 16:05

SevenW wrote:I do not have yet the killer idea to solve this. There is always a chance of the device getting an external update (due to external switching, plugwise schedule switching, a state-update heartbeat), and there is always the chance that at the same time the switch is operated form the domoticz UI(s). In the end, domoticz and plugwise need to agree on the actual switch state.
SevenW


It's a complicated situation.
The problem is not the switching itself, that is working.
The problem is the status update of the switch status in Domoticz. And we want to update the status of the switch in domoticz every second or 30 seconds. Not only once or when it is switched. No, every x times per second.

Can we alter the http:// into switch?mac=038BE8A&cmd=[switch|update]&val=on
And do some hocus pocus with "cmd" with the function that is attached to the [get] in NodeRed?
Domoticz runs the command always with cmd=switch, maybe the crontab/timer in NodeRed can do a cmd=update to differentiate.
There is also a possibility to do stuff with user variables in Domoticz via the API. Is that usable?

Thinking of it... This might include a third virtual switch. ( three units is better I think )
Why three?
1 virtual switch for the power reading
2 virtual switch for the actual switching
3 virtual switch for the status of the switch

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Monday 07 December 2015 21:47

I am making some interesting progress. Per circle I have two virtual switches. The first is the switch, and the second switches the schedule on or off. With the schedule switched on, the switch should follow the state of the plugwise circle. The following graph shows that the switch gets updated according to the schedule in the circle:
domoticz plugwise switch by schedule.png
domoticz plugwise switch by schedule.png (127.41 KiB) Viewed 1265 times


I have also added cumulative energy (kWh's) to the virtual sensors. I use the General,kWh instead of the Usage, electric sensors for this. After running this for a few days and validating the behavior I shall post my node-red stuff.

SevenW

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Monday 07 December 2015 23:49

... and I found a way to break the update loop. It is also a simpler setup. I do not use the On / Off actions anymore. I stopped listening to http calls to node-red from domoticz, but I subscribe to domticz/out MQTT messages. I filter them so that I get only MQTT switch messages. With only a minor trick / workaround I know when a switch is invoked by the plugwsise-2-py webapp, or when it is generated form domoticz (or any other http/json/api client).

This makes the Plugwise-2-py - Domoticz integration very workable.

As promised will publish node-red scripts soon.

SevenW

bloemkool
Posts: 10
Joined: Thursday 05 November 2015 12:21
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby bloemkool » Tuesday 08 December 2015 11:55

Ahh ... Can't wait!
Is it also possible to completely eliminate NodeRed and let plugwise2py work directly with the /domoticz/in/# /domoticz/out/# MQTT ? Or is that the next step?
Or do you still need NodeRed for the circle power readout?

I;d wish I had more time to contribute on this...

dorenberg
Posts: 62
Joined: Monday 22 June 2015 20:18
Target OS: Raspberry Pi
Domoticz version: 3.4984
Location: Veghel, The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby dorenberg » Tuesday 08 December 2015 12:14

Can't wait either for this. WOW! What a progress. Thanks in advance!

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Tuesday 08 December 2015 20:58

bloemkool wrote:Is it also possible to completely eliminate NodeRed and let plugwise2py work directly with the /domoticz/in/# /domoticz/out/# MQTT ? Or is that the next step?
Or do you still need NodeRed for the circle power readout?


I tested it a bit, and looked into the MQTT source code of domoticz, but currently it looks like the MQTT interface supports fewer commands, and I doubt I can automatically create devices, and I can not address them by did (plugwise ID), but only by idx. And then, I do not see a MQTT command to query all devices to map idx to did. A pure MQTT interface seems unfeasible, or at least impractical. Too bad!

Very interesting thought anyway.

SevenW

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Tuesday 08 December 2015 23:52

I have added the NodeRed scripts to my github, as well as some very rudimentary instructions, to be found in the domoticz folder:
https://github.com/SevenW/Plugwise-2-py/tree/master/domoticz

It is required that you update your PlugWise-2-py from github master if you have it already installed before today. In Plugwise-2-py I have added provision for cumulative energy and to break the update loop.

Key features:
  • Energy (savelog=yes) is using General-kWh sensor (cumulative kWh)
  • Power (monitor=yes) is using the Usage-Electric sensor (summing the 10 seconds values is not robust in a PW network).
  • When a schedule is used, the schedule name is displayed in a text-sensor.
  • When a circle is offline, it is also displayed in that text-sensor.
  • Separate Domoticz-switches to switch the circle on/off, and to switch the circle-schedule on/off.
  • Note: when using the circle-switch, the schedule will be turned off.
  • The infamous update/feedback loop of smart switches is fully prevented.
  • The Device IDs are last seven digits of MAC, prefixed with: A=status text, B=breaker/switch, C=clock/schedule, E=Energy(kWh), F=Power(W)

dorenberg
Posts: 62
Joined: Monday 22 June 2015 20:18
Target OS: Raspberry Pi
Domoticz version: 3.4984
Location: Veghel, The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby dorenberg » Wednesday 09 December 2015 12:19

Everything updated and so far so good. 6 months of circle data imported into domoticz as used today. But the hourly update works good! Switches and schedules and kwh meters and watt meters are being added to the unused section. I didn't have to make a dummy to get everything working.

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Wednesday 09 December 2015 21:34

so what hardware id did you assign then? Can it be arbitrary? Or did you use the MQTT hardware id?

SevenW

dorenberg
Posts: 62
Joined: Monday 22 June 2015 20:18
Target OS: Raspberry Pi
Domoticz version: 3.4984
Location: Veghel, The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby dorenberg » Wednesday 09 December 2015 22:32

In hardware number 10 is for dummy sensors (in my system). So both values are set at ten. That is the category. The idx will be auto generated in domoticz in the unused list. the circle ID with additonal letters in front as explained is the ID of the switch.

User avatar
proohu01
Posts: 160
Joined: Friday 17 October 2014 12:20
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby proohu01 » Monday 14 December 2015 11:16

So how is the MQTT hardware item configured? Should I keep the username and password empty (Domoticz Pi image is used for installation). How about the "Publish Topic"? I don't understand the choices. (flat/hierarchical/combined/none) :?:
Pi 2 B | Pi Camera | PiFace 2 | RFXtrx433E (KaKu + Oregon Scientific) | Plugwise (9 Circles) | Aeon Z-Wave stick (Fibaro - dimmers, sockets, RGB, screens/blinds) | RFXMeter (Kwh/water/gaz) | WOL | Onkyo Receiver | RTC | Hikvision | Netatmo | Satel Integra

dorenberg
Posts: 62
Joined: Monday 22 June 2015 20:18
Target OS: Raspberry Pi
Domoticz version: 3.4984
Location: Veghel, The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby dorenberg » Monday 14 December 2015 20:32

proohu01 wrote:So how is the MQTT hardware item configured? Should I keep the username and password empty (Domoticz Pi image is used for installation). How about the "Publish Topic"? I don't understand the choices. (flat/hierarchical/combined/none) :?:



The setup as described here uses MQTT translation via Node-red. If you have domoticz installed on port 8080 and excluded the localhost from the list to ask for a password, no adaptations of the script posted above are needed. I don't recognize the choices you present here.

bloemkool
Posts: 10
Joined: Thursday 05 November 2015 12:21
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby bloemkool » Tuesday 15 December 2015 15:25

Just had the time to git-pull plugwise2py and started to change the domoticz content. Was a bit of work but looks good right now!
I also saved the power data I already collected from previous config.
All we need is a change to the API: add descriptions to a device.

Great job SevenW!!

And I think I found a bug in domoticz, can anyone confirm?
-- When I add a group, name it christmas lights.
- Add a KAKU switch
- Add a Plugwise switch
-- Switch the group off/on
-- Domoticz says the Circle and the KAKU is off or on...
-- Only the KAKU switch actually switches off and on, the Plugwise switch does not respond.

When I directly switch the Circle it responds immediately.
I had this same thing with the previous http:// call I did. And hoped it would be solved with this config.
Last edited by bloemkool on Tuesday 15 December 2015 16:29, edited 1 time in total.

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Tuesday 15 December 2015 15:52

proohu01 wrote:So how is the MQTT hardware item configured? Should I keep the username and password empty (Domoticz Pi image is used for installation). How about the "Publish Topic"? I don't understand the choices. (flat/hierarchical/combined/none) :?:


I assume that you followed the instructions on the MQTT wiki. https://www.domoticz.com/wiki/MQTT

Did you install Mosquitto (may be standard in Raspberry pi image, I am not sure). Mosquittto MQTT server default install is using no username/password, and default port 1883 (at localhost or 127.0.0.1).

So leave user/password fields empty. Publish topic can be set to "Flat". This works. I did not test other settings, but they may work as well, expect for "none", which is just disabling MQTT

Let me know if it works.

Regards,
SevenW

User avatar
proohu01
Posts: 160
Joined: Friday 17 October 2014 12:20
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby proohu01 » Wednesday 16 December 2015 8:23

@SevenW. Yes I did follow the WiKi. There was no mention of a username and password in the wiki so I already left that empty.
Setting the publish topic to "flat" seems to have fixed it for me. I now see the new devices in Domoticz. Thank you.

The power I get from all of my Plugs are negative values. Why is that? Is that normal?
Pi 2 B | Pi Camera | PiFace 2 | RFXtrx433E (KaKu + Oregon Scientific) | Plugwise (9 Circles) | Aeon Z-Wave stick (Fibaro - dimmers, sockets, RGB, screens/blinds) | RFXMeter (Kwh/water/gaz) | WOL | Onkyo Receiver | RTC | Hikvision | Netatmo | Satel Integra

SevenW
Posts: 27
Joined: Sunday 16 August 2015 22:19
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby SevenW » Thursday 17 December 2015 0:48

proohu01 wrote:The power I get from all of my Plugs are negative values. Why is that? Is that normal?


That could be me causing this. When in plugwise-2-py, for a given circle production = yes, I negate that values in the nodered script, as plugwsie-2-py reports production as negative numbers, while Domoticz seems to dislike negative values (it does not draw a graph below zero). When you have a circle for a mixed production/consumption device, then my strategy is not applicable. I use it for PV-solare energy, and that circle sees only production or zero Watts.

If the negative numbers are just absolute values below 1W, than it is just noise in the circle measurement.

SevenW

User avatar
proohu01
Posts: 160
Joined: Friday 17 October 2014 12:20
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands
Contact:

Re: Connecting Plugwise-2-py via MQTT to Domoticz

Postby proohu01 » Thursday 17 December 2015 8:58

When you have a circle for a mixed production/consumption device....

My bad! I thought that the value "production=True/False" was to include or exclude a circle plug. As in production/test. :oops:

All of my Plugwise cicles are consumption devices so I changed it to production=false.
Pi 2 B | Pi Camera | PiFace 2 | RFXtrx433E (KaKu + Oregon Scientific) | Plugwise (9 Circles) | Aeon Z-Wave stick (Fibaro - dimmers, sockets, RGB, screens/blinds) | RFXMeter (Kwh/water/gaz) | WOL | Onkyo Receiver | RTC | Hikvision | Netatmo | Satel Integra


Return to “Meters”

Who is online

Users browsing this forum: No registered users and 1 guest