Python Plugin: Shelly MQTT

Python and python framework
Post Reply
lwolf
Posts: 5
Joined: Saturday 10 November 2018 19:29
Target OS: Raspberry Pi
Domoticz version:
Contact:

Python Plugin: Shelly MQTT

Post by lwolf » Saturday 10 November 2018 21:42

If anyone interested, I've made a Python plugin for Shelly MQTT integration into Domoticz. Currently I have only got a Shelly1 device to test, but more will come soon.
I would like to thank the zigbee2mqtt project for it's nice MQTT library that i reused.
https://github.com/enesbcs/Shelly_MQTT

minime
Posts: 2
Joined: Thursday 15 November 2018 16:41
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by minime » Thursday 15 November 2018 16:48

Great, thats something I'm looking. My shelly devices will arrive soon (hope so). So I will test this soon.
Because I want to use Shelly2 for shutter control, do you think theres a way to get them support soon, too?

Thanks alot in advance
Georg

lwolf
Posts: 5
Joined: Saturday 10 November 2018 19:29
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by lwolf » Wednesday 21 November 2018 20:38

minime wrote:
Thursday 15 November 2018 16:48
Great, thats something I'm looking. My shelly devices will arrive soon (hope so). So I will test this soon.
Because I want to use Shelly2 for shutter control, do you think theres a way to get them support soon, too?
My Shelly2 arrived yesterday, so i think roller shutter will be supported within a few days.

update: roller shutter is supported

minime
Posts: 2
Joined: Thursday 15 November 2018 16:41
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by minime » Monday 26 November 2018 11:49

Great job. Hopefully my devices will arrive this week. Can't wait for testing them.

Thanks alot for your work!

gianfrdp
Posts: 10
Joined: Sunday 10 January 2016 4:22
Target OS: Linux
Domoticz version: 3.8805
Contact:

Re: Python Plugin: Shelly MQTT

Post by gianfrdp » Monday 03 December 2018 13:18

Hello, I tried you plugin for a Shelly1.
I have Domoticz v3.8805 on linux amd_x64 with mosquitto on board and python version

Code: Select all

python --version
Python 2.7.9
I cloned repository into

Code: Select all

/opt/domoticz/plugins/Shelly_MQTT
Restarted domoticz.

I crated hw with mqtt server 127.0.0.1:1883.

Looking on domticz log I have

Code: Select all

2018-12-03 12:16:28.464  (Shelly) Calling message handler 'onHeartbeat'.
2018-12-03 12:16:28.464  (Shelly) Heartbeating...
2018-12-03 12:16:28.464  (Shelly) Reconnecting
2018-12-03 12:16:28.464  (Shelly) MqttClient::Open
2018-12-03 12:16:28.464  (Shelly) MqttClient::Close
2018-12-03 12:16:28.464  (Shelly) Protocol set to: 'MQTT'.
2018-12-03 12:16:28.464  (Shelly) Transport set to: 'TCP/IP', 127.0.0.1:1883.
2018-12-03 12:16:28.464  (Shelly) Connect directive received, action initiated successfully.
2018-12-03 12:16:28.514  (Shelly) Calling message handler 'onConnect'.
2018-12-03 12:16:28.514  (Shelly) MQTT connected successfully.
2018-12-03 12:16:28.514  (Shelly) MqttClient::Connect
2018-12-03 12:16:28.514  (Shelly) MQTT CONNECT ID: 'Domoticz_1543835788'
2018-12-03 12:16:28.515  Error: (ProcessOutbound) Send request Python object parameter was not of type Unicode or Byte, ignored.
2018-12-03 12:16:28.515  (Shelly) Sending 0 bytes of data:
2018-12-03 12:16:28.965  (Shelly) Disconnect directive received for '127.0.0.1:1883'.
2018-12-03 12:16:28.965  (Shelly): CPluginTransportTCP::handleDisconnect
2018-12-03 12:16:29.015  (Shelly) Calling message handler 'onDisconnect'.
2018-12-03 12:16:29.016  (Shelly) MqttClient::onDisonnect Disconnected from: 127.0.0.1:1883
2018-12-03 12:16:29.016  (Shelly) MqttClient::Close
2018-12-03 12:16:29.016  (Shelly) onMQTTDisconnected
2018-12-03 12:16:29.466  (Shelly) Disconnect directive received for '127.0.0.1:1883'.
2018-12-03 12:16:29.466  (Shelly): CPluginTransportTCP::handleDisconnect
2018-12-03 12:16:29.517  (Shelly) Calling message handler 'onDisconnect'.
2018-12-03 12:16:29.517  (Shelly) MqttClient::onDisonnect Disconnected from: 127.0.0.1:1883
2018-12-03 12:16:29.517  (Shelly) MqttClient::Close
2018-12-03 12:16:29.517  (Shelly) onMQTTDisconnected
2018-12-03 12:16:38.479  (Shelly) Calling message handler 'onHeartbeat'.
2018-12-03 12:16:38.479  (Shelly) Heartbeating...
2018-12-03 12:16:38.479  (Shelly) Reconnecting
2018-12-03 12:16:38.479  (Shelly) MqttClient::Open
2018-12-03 12:16:38.479  (Shelly) Protocol set to: 'MQTT'.
2018-12-03 12:16:38.479  (Shelly) Transport set to: 'TCP/IP', 127.0.0.1:1883.
2018-12-03 12:16:38.479  PluginSystem: Starting I/O service thread.
2018-12-03 12:16:38.479  (Shelly) Connect directive received, action initiated successfully.
2018-12-03 12:16:38.529  (Shelly) Calling message handler 'onConnect'.
2018-12-03 12:16:38.529  (Shelly) MQTT connected successfully.
2018-12-03 12:16:38.529  (Shelly) MqttClient::Connect
2018-12-03 12:16:38.529  (Shelly) MQTT CONNECT ID: 'Domoticz_1543835798'
2018-12-03 12:16:38.530  Error: (ProcessOutbound) Send request Python object parameter was not of type Unicode or Byte, ignored.
2018-12-03 12:16:38.530  (Shelly) Sending 0 bytes of data:
On mosquitto

Code: Select all

1543832868: New connection from 192.168.2.40 on port 1883.
1543832868: New client connected from 192.168.2.40 as shelly1-4A5936 (c1, k60).
How can I solve it?

Thanks

lwolf
Posts: 5
Joined: Saturday 10 November 2018 19:29
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by lwolf » Monday 10 December 2018 22:09

gianfrdp wrote:
Monday 03 December 2018 13:18
Hello, I tried you plugin for a Shelly1.
I have Domoticz v3.8805 on linux amd_x64 with mosquitto on board and python version

Code: Select all

python --version
Python 2.7.9
How can I solve it?

Thanks
Please install Python 3.4 or newer following the Domoticz guide:
https://www.domoticz.com/wiki/Using_Python_plugins

gianfrdp
Posts: 10
Joined: Sunday 10 January 2016 4:22
Target OS: Linux
Domoticz version: 3.8805
Contact:

Re: Python Plugin: Shelly MQTT

Post by gianfrdp » Tuesday 11 December 2018 16:52

Sorry, I checked on domoticz log: I have

Code: Select all

2018-12-11 15:43:32.794 PluginSystem: Started, Python version '3.4.2'.

Code: Select all

python3 --version
Python 3.4.2

lwolf
Posts: 5
Joined: Saturday 10 November 2018 19:29
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by lwolf » Tuesday 11 December 2018 21:57

gianfrdp wrote:
Tuesday 11 December 2018 16:52
Sorry, I checked on domoticz log: I have

Code: Select all

2018-12-11 15:43:32.794 PluginSystem: Started, Python version '3.4.2'.

Code: Select all

python3 --version
Python 3.4.2
I've downgraded my Python 3.6.6 to the oldest possible 3.4.9 version, but i am unable to reproduce this error.
"Error: (ProcessOutbound) Send request Python object parameter was not of type Unicode or Byte, ignored."

I assume you purged the previously installed Python 2.x to not interfere with v3.4?

My test environment is producing the logs below with Verbose debug output and works fine:

Code: Select all

2018-12-11 20:25:55.567 Status: Domoticz V4.9701 (c)2012-2018 GizMoCuz
2018-12-11 20:25:55.868 Status: PluginSystem: Started, Python version '3.4.9'.
2018-12-11 20:25:58.384 (shelly) Debug logging mask set to: PYTHON PLUGIN DEVICE CONNECTION MESSAGE
2018-12-11 20:25:58.384 (shelly) MqttClient::__init__
2018-12-11 20:25:58.384 (shelly) MqttClient::Open
2018-12-11 20:25:58.384 (shelly) Protocol set to: 'MQTT'.
2018-12-11 20:25:58.384 (shelly) Transport set to: 'TCP/IP', 192.168.2.100:1883.
2018-12-11 20:25:58.384 PluginSystem: Starting I/O service thread.
2018-12-11 20:25:58.384 (shelly) Connect directive received, action initiated successfully.
2018-12-11 20:25:58.434 (shelly) MQTT connected successfully.
2018-12-11 20:25:58.435 (shelly) MqttClient::Connect
2018-12-11 20:25:58.435 (shelly) MQTT CONNECT ID: 'Domoticz_1544556358'
2018-12-11 20:25:58.435 (shelly) Sending 33 bytes of data
2018-12-11 20:25:58.435 (shelly) 10 1f 00 04 4d 51 54 54 04 02 00 3c 00 13 44 6f 6d 6f 74 69 ....MQTT...<..Domoti
2018-12-11 20:25:58.435 (shelly) 63 7a 5f 31 35 34 34 35 35 36 33 35 38 .. .. .. .. .. .. .. cz_1544556358
2018-12-11 20:25:58.485 (shelly) Received 4 bytes of data
2018-12-11 20:25:58.485 (shelly) 20 02 00 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ....
2018-12-11 20:25:58.485 (shelly) MqttClient::Subscribe
2018-12-11 20:25:58.485 (shelly) Sending 17 bytes of data
2018-12-11 20:25:58.485 (shelly) 82 0f 00 01 00 0a 73 68 65 6c 6c 69 65 73 2f 23 00 .. .. .. �.....shellies/#.
2018-12-11 20:25:58.535 (shelly) Received 5 bytes of data
2018-12-11 20:25:58.536 (shelly) 90 03 00 01 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �....
2018-12-11 20:25:58.536 (shelly) onMQTTSubscribed
2018-12-11 20:25:58.342 Status: PluginSystem: Entering work loop.
2018-12-11 20:25:58.382 Status: (shelly) Initialized version 0.2.0
2018-12-11 20:25:58.384 Status: (shelly) Entering work loop.
2018-12-11 20:26:07.934 (shelly) Heartbeating...
2018-12-11 20:26:07.934 (shelly) MqttClient::Ping
2018-12-11 20:26:07.934 (shelly) Sending 2 bytes of data
2018-12-11 20:26:07.934 (shelly) c0 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �.
2018-12-11 20:26:07.985 (shelly) Received 2 bytes of data
2018-12-11 20:26:07.985 (shelly) d0 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �. 
My best guess is that somewhere between Domoticz v3.8805 (2017?) and current stable Domoticz v4.9701 (2018-06-23) the Domoticz API changed somewhat, that causes errors.
Domoticz.Connection() object has a Send() method and that is raising the error you sent.
However the mqtt.py that I've reused is in use by several other plugins, and i do not see any controversy that's code and the Domoticz Python documentation - which is not too long, neither talkative.
https://www.domoticz.com/wiki/Developin ... hon_plugin
Can you use any other MQTT or HTTP based Python plugin on your installation?

Do you have any specific reason for sticking old softwares, such as 4 year old python and Domoticz 3.8? I do not have this version, and do not know, how to downgrade to it, so the further debugging seems impossible to me.

You can try to change this code in mqtt.py from this:

Code: Select all

     else:
            ID = 'Domoticz_'+str(int(time.time()))
            Domoticz.Debug("MQTT CONNECT ID: '" + ID + "'")
            self.mqttConn.Send({'Verb': 'CONNECT', 'ID': ID})
To this:

Code: Select all

     else:
         try:
            ID = 'Domoticz_'+str(int(time.time()))
            Domoticz.Debug("MQTT CONNECT ID: '" + ID + "'")
            self.mqttConn.Send({'Verb': 'CONNECT', 'ID': ID})
         except Exception as e:
            Domoticz.Debug("Unknown connection error: "+str(e))
Then restart Domoticz and you will see more in the logs..

gianfrdp
Posts: 10
Joined: Sunday 10 January 2016 4:22
Target OS: Linux
Domoticz version: 3.8805
Contact:

Re: Python Plugin: Shelly MQTT

Post by gianfrdp » Wednesday 12 December 2018 4:39

Hello,
I have a Debian Jessie on a OpenMediaVault server, which provides

Code: Select all

python                                2.7.9-1
python3                               3.4.2-2
I installed a separate python3.5 on /usr/local, I have to understand how to force Domoticz to use it.

I will try also your suggested changes

I downgraded Domoticz to v3.8805 because last versions crash with signal 6.
I have no other python plugin.

Thanks

EDIT:

I tried another python plugin https://github.com/emontnemery/domoticz_mqtt_discovery with same error.
Searching into domoticz source files I found in https://github.com/domoticz/domoticz/bl ... tocols.cpp point where error message is printed

Code: Select all

        std::vector<byte> CPluginProtocol::ProcessOutbound(const WriteDirective* WriteMessage)
        {
                std::vector<byte>       retVal;

                // Handle Bytes objects
                if ((((PyObject*)WriteMessage->m_Object)->ob_type->tp_flags & (Py_TPFLAGS_BYTES_SUBCLASS)) != 0)
                {
                        const char*     pData = PyBytes_AsString(WriteMessage->m_Object);
                        int                     iSize = PyBytes_Size(WriteMessage->m_Object);
                        retVal.reserve((size_t)iSize);
                        retVal.assign(pData, pData + iSize);
                }
                // Handle ByteArray objects
                else if ((((PyObject*)WriteMessage->m_Object)->ob_type->tp_name == std::string("bytearray")))
                {
                        size_t  len = PyByteArray_Size(WriteMessage->m_Object);
                        char*   data = PyByteArray_AsString(WriteMessage->m_Object);
                        retVal.reserve(len);
                        retVal.assign((const byte*)data, (const byte*)data + len);
                }
                // Handle String objects
                else if ((((PyObject*)WriteMessage->m_Object)->ob_type->tp_flags & (Py_TPFLAGS_UNICODE_SUBCLASS)) != 0)
                {
                        std::string     sData = PyUnicode_AsUTF8(WriteMessage->m_Object);
                        retVal.reserve((size_t)sData.length());
                        retVal.assign((const byte*)sData.c_str(), (const byte*)sData.c_str() + sData.length());
                }
                else
                        _log.Log(LOG_ERROR, "(%s) Send request Python object parameter was not of type Unicode or Byte, ignored.", __func__);

                return retVal;
        }
so I think it should be a general problem with domoticz version.

I will try to upgade domoticz and see what appens.

Thanks

midomot
Posts: 18
Joined: Sunday 12 February 2017 14:27
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by midomot » Friday 14 December 2018 18:38

Hello
I want to thank you so very very much for this plugin! :)

I received my shelly2 yesterday, today I made a mock-up installation to test
I found your plugin, and it now all works, all the way to Domoticz's !

Good to know:

- if you already have a mqtt server running on port 1883, you need to set another port in both Shelly's interface and Domoticz's for this Shelly_MQTT
If you don't do that.. you get a bunch of errors !

- after above is done, don't worry if you see still errors going by in the log (I got worried...) - just follow the instructions from lwolf : you need to get the devices automatically created ! so authorize in Domoticz's settings, manipulate the switches, and voila: you'll have your switches created, and errors will be gone !
Now you can set it all up in Domoticz as you'll like

Finally,
- if like me you're using the Shelly2 behind a double switch (one switch per lamp), you will want to go into the individual button settings in Shelly's interface and change button type to "Edge Switch"

you may also want to update the "power on mode" - I need to check that one out myself.

lwolf
Posts: 5
Joined: Saturday 10 November 2018 19:29
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Python Plugin: Shelly MQTT

Post by lwolf » Friday 14 December 2018 22:50

midomot wrote:
Friday 14 December 2018 18:38
- if you already have a mqtt server running on port 1883, you need to set another port in both Shelly's interface and Domoticz's for this Shelly_MQTT
If you don't do that.. you get a bunch of errors !
You are welcome. Could you send the error messages, that you got? I am using the Shelly's on the same MQTT broker and port 1883 as my other Tasmota devices, and i did not saw any interesting yet.
Errors maybe pop up when it tries to create a new device, that just found, and automatic device creation is disabled?

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests