[90] MQTT Support

Use this forum to discuss possible implementation of a new feature before opening a ticket.
A developer shall edit the topic title with "[xxx]" where xxx is the id of the accompanying tracker id.
Duplicate posts about the same id. +1 posts are not allowed.
wildfire
Posts: 29
Joined: Friday 18 September 2015 11:04
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by wildfire » Saturday 23 January 2016 18:05

I still think the mqtt module should work without IDX aswell, programming nodes everytime you change someting in domoticz is a drag if there somewer build in to your house, in a top corner of a room somewhere.
so you should be able to make a virtual device listen to /raw/garage/temp or /raw/garage/humi.
this would make my life easyer at least
that way i can also skip using node red for rerouting every post and adding an IDX

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Monday 25 January 2016 2:41

Wrote a NodeJS module for interacting directly w/Domoticz, forked discussion regarding it into its own thread: viewtopic.php?f=21&t=10190
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

wildfire
Posts: 29
Joined: Friday 18 September 2015 11:04
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by wildfire » Monday 25 January 2016 8:57

this looks great, im gonne try this tonight!

thanks

kylegordon
Posts: 96
Joined: Friday 01 November 2013 0:58
Target OS: Linux
Domoticz version: Trunk
Contact:

Re: [90] MQTT Support

Post by kylegordon » Monday 25 January 2016 17:29

Looks good, thank you!

Annoyingly my continuing strace of domoticz is doing just that. It hasn't crashed at all now!
1 x HP DL380 & KVM
1 x RPI
1 x RFXtrx433 V78
11 x LWRF Switches
1 x LIFX
2 x Echo Dots
6 x Byron PIRs
2 x Nexa PIRs
2 x Kodi
2 x ESP8266 MQTT
1 x GPIO/MQTT PIR
1 x GPIO(PWM)/MQTT LEDs
1 x GPIO(SPI)/MQTT LPD6803 LEDs
Lots of Node-Red glue

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Tuesday 26 January 2016 2:10

I noticed when writing my module I sent in some data that crashed domoticz durring testing, I think more error checking needs to be done in the MQTT hardware.

but I added error checking to my node module to prevent Strings (or NaN) being pushed when expecting an integer and vice versa, I have not witnessed it happen recently.

I suspect your problem is data related, I would log both whats being sent and received and then focus on the last thing sent before everything goes pear shaped.

Another thing I found when writing my Node Module was that the MQTT API was missing the addlogmessage command that exists in the URL API... so I fixed that, hopefully it'll be merged soon: https://github.com/domoticz/domoticz/pull/382
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

kylegordon
Posts: 96
Joined: Friday 01 November 2013 0:58
Target OS: Linux
Domoticz version: Trunk
Contact:

Re: [90] MQTT Support

Post by kylegordon » Tuesday 26 January 2016 2:48

It is quite possibly data related, and I have quite a lot of it flying across the MQTT broker.

I've already raised the issue with the authors about the JS string 'NaN' crashing Domoticz, and I was told that I should ensure that such a string is never sent to Domoticz. So much for defensive programming. It's also destroyed the wife acceptance factor.

meh
1 x HP DL380 & KVM
1 x RPI
1 x RFXtrx433 V78
11 x LWRF Switches
1 x LIFX
2 x Echo Dots
6 x Byron PIRs
2 x Nexa PIRs
2 x Kodi
2 x ESP8266 MQTT
1 x GPIO/MQTT PIR
1 x GPIO(PWM)/MQTT LEDs
1 x GPIO(SPI)/MQTT LPD6803 LEDs
Lots of Node-Red glue

Toni
Posts: 86
Joined: Monday 20 July 2015 14:12
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by Toni » Tuesday 26 January 2016 9:28

I've been logging my domoticz/in topic into a file, and didn't notice any problems with the data though.

Toni
Posts: 86
Joined: Monday 20 July 2015 14:12
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by Toni » Saturday 30 January 2016 19:09

Today I also switched from feeding sensor data through MQTT to feeding in the data using the REST API. Let's see...

A very minor change in my automation, originally I was feeding the data through the API, and still had the old versions in version control. It's good to use Git also for your event scripts...

Toni
Posts: 86
Joined: Monday 20 July 2015 14:12
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by Toni » Sunday 31 January 2016 11:15

And it looks good so far, with MQTT it would already have crashed a number of times.

So, don't rely on MQTT, but use the REST API instead to feed in external data to Domoticz.

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Sunday 31 January 2016 11:24

Ive been pushing alot of data through MQTT for a while now without issue, at pretty high frequency to boot.. I think with a good library thats sanitizing inputs, such as my node-domoticz-js module, its plenty reliable..

I think if I tried to fallback to the URL API right now things would slow way down, when I turn my volume knob Ive got it throttled back to 1 command per 100ms and it still generates a ton of traffic for a few moments of movement (10 updates a second).. establishing http connections and closing them would suck up alot of that time.
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

Toni
Posts: 86
Joined: Monday 20 July 2015 14:12
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by Toni » Sunday 31 January 2016 18:33

I wish it would have been working for me. But it's not just about the awful reliability. My CPU use skyrocketed on Oct 14th 2015, and went back down yesterday when I switched from MQTT to the REST API. I think it was because of a Domoticz change, as I've been running the same MQTT scripts since last summer.
cpu.PNG
cpu.PNG (113.44 KiB) Viewed 1701 times

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Monday 01 February 2016 8:24

thats pretty dramatic, was that your script eating up cpu or domoticz?

ive seen no increase in cpu load since using the MQTT interface, quite the opposite actually.. my memory usage was pegged @ 100% and settled down once I started moving old scripts over to it.. but I also started cleaning up things and using a common library.. removed a ton of event scripts since I dont have to make calls to my software.

It just goes to show, it all comes down to your code.. its going to be hard to peg this on MQTT specifically, yeah the URL API is able to cope better.. that could be for lots of reasons.. MQTT is definitely a more raw interface, and less forgiving than the URL API.
Attachments
Screen Shot 2016-01-31 at 11.21.31 PM.png
Screen Shot 2016-01-31 at 11.21.31 PM.png (209.24 KiB) Viewed 1685 times
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

Toni
Posts: 86
Joined: Monday 20 July 2015 14:12
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by Toni » Monday 01 February 2016 9:21

I still sent the same data to Domoticz, and Domoticz still listens to domoticz/in. There's just no traffic in there.

So all I did was to change the publishing of my sensor values from mqtt to the rest api. I use a perl script, and the change was literally a one-liner, from mqtt publish to http get. So all the same functionality is still there, only the publishing method changed.

SchattenMann
Posts: 69
Joined: Friday 09 October 2015 19:14
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by SchattenMann » Thursday 11 February 2016 14:10

Is it possible to include the Hardware ID/Name/Type on default MQTT messages? This would be a great help!
right now we only have IDX, sType and dType...

on a different note is it possible to send the following command via MQTT?

Code: Select all

/json.htm?type=setused&idx=<idx>&setpoint=<setpoint>&mode=<mode>&used=true 
I've tried:

Code: Select all

{"type": "setused", "idx": "22" , "setpoint": "13.07", "mode": "TemporaryOverride", "used": "true"}
{"command": "setused", "idx": "22" , "setpoint": "13.07", "mode": "TemporaryOverride", "used": "true"}
without success so I take not all HTTP commands can be "translated" to MQTT?

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Thursday 11 February 2016 18:49

I get all that info out MQTT

Code: Select all

{"Battery":255,"RSSI":12,"dtype":"Light/Switch","id":"000140E0","idx":145,"name":"TV","nvalue":0,"stype":"Selector Switch","svalue1":"0","switchType":"Selector","unit":1}
and yeah not all HTTP commands are in MQTT, ive been trying to add the missing ones as I find them.. (addlogmessage for example)

you might see if you can use the udevice command to get what you want
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

SchattenMann
Posts: 69
Joined: Friday 09 October 2015 19:14
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by SchattenMann » Thursday 11 February 2016 19:44

nayr wrote:I get all that info out MQTT

Code: Select all

{"Battery":255,"RSSI":12,"dtype":"Light/Switch","id":"000140E0","idx":145,"name":"TV","nvalue":0,"stype":"Selector Switch","svalue1":"0","switchType":"Selector","unit":1}
So fo I but that doesn't give you the HardwareName for instance. It's really useful to know if it's a Z-Wave/Nest/Honeywell/Hue/Oregon upfront since I don't want to have the same treatment for a Nest Thermostat that I have for a Honeywell TRV for instance - makes sense?
For now I have an HTTP GET request to get a list of all devices and store the HardwareName (along side other things) in a JS Object to be later used by node-red but this step would be unnecessary if the MQTT Message could push this as well.
nayr wrote: and yeah not all HTTP commands are in MQTT, ive been trying to add the missing ones as I find them.. (addlogmessage for example)
Yeah i've found out the same thing now. It would be great to add Thermostart/Honeywell support :D
nayr wrote:you might see if you can use the udevice command to get what you want
for now I'm using /json.htm?type=devices& to get a full list of all devices on the startup code

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Thursday 11 February 2016 20:17

oic, yeah I think that would be really nice..

for example my Home Theatre controller uses alot of dummy switches.. right now you have to configure the idx of each switch manually, with you could create a dummy hardware device just for my software, and my software could detect new switches as they were created.

I'd also like to get the selector level name in output, for example svalue=20 = PlayStation3 in the UI but I cant get the level name via API like I can the switch name.

I also believe were missing the ability in MQTT to create devices, with all these exposed in the API I could likely make my software smart enough it sets its self up and creates all the hardware like a native device would.
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

SchattenMann
Posts: 69
Joined: Friday 09 October 2015 19:14
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by SchattenMann » Thursday 11 February 2016 20:33

yeah, it would be great to see Domoticz MQTT support to keep moving forward as I'm planning to ditch my Vera and use Domoticz + node-red for all my logic

jeanvdr
Posts: 1
Joined: Friday 12 February 2016 12:18
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: [90] MQTT Support

Post by jeanvdr » Friday 12 February 2016 12:29

Hi,

I try to connect a nodeMCU-esp12 to a RasberryPi Domoticz server through mqtt.
I use C to program the esp12 with Arduino IDE.
No problem to transmit some info to Domoticz from the nodeMCU (acting as a sensor node) using mqtt and publishing in domoticz/in.
On the contrary, no way to get any data when subscribing on domoticz/out ?
I get the data on my PC with python/paho so I know data are passing by.
When I use mosquitto_pub on my RaspberryPi to publish in domoticz/out, then my nodeMCU receive the data.
So for now, I use a lua-script in Domoticz to republish some data in domoticz/out to get it to my nodeMCU (I know I could use node-red)
I've tried all mqtt librairies existing on Arduino, same result

Any clue, anybody ?

Thanks a lot

Jean

User avatar
nayr
Posts: 430
Joined: Tuesday 11 November 2014 19:42
Target OS: Linux
Domoticz version: github
Location: Denver, CO - USA
Contact:

Re: [90] MQTT Support

Post by nayr » Monday 15 February 2016 22:30

@jeanvdr, Can you try using another client to see if domoticz is sending data to domoticz/out?

to anyone else concerned, I worked with @gaudryrc and he expanded the new Selector Switches output in MQTT: https://github.com/domoticz/domoticz/issues/459

Code: Select all

{
	"Battery":255,
	"LevelNames":"Off|TV|PS3|PS4|NVR|FM",
	"LevelOffHidden":"false",
	"RSSI":12,
	"SelectorStyle":"0",
	"dtype":"Light/Switch","
	id":"000140E0",
	"idx":145,
	"name":"TV",
	"nvalue":2,
	"stype":"Selector Switch",
	"svalue1":"10",
	"switchType":"Selector",
	"unit":1
}
Debian Jessie: CuBox-i4 (Primary) w/Static Routed IP and x509 / BeagleBone with OpenSprinkler / BeagleBone Planted Aquarium / 3x Raspbery Pi2b GPIO Slaves
Elemental Theme - node-domoticz-mqtt - Home Theatre Controller - AndroidTV Simple OSD Remote - x509 TLS Auth

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest