python plugin connection

Python and python framework
Post Reply
elgringo
Posts: 127
Joined: Thursday 18 May 2017 8:08
Target OS: Raspberry Pi
Domoticz version: Left
Contact:

python plugin connection

Post by elgringo » Thursday 08 June 2017 23:31

I am using the new pythyn plugin framework

When a connection failed (onconnect is called with status != 0) I get an error when reconnecting:

Code: Select all

if (Status == 0):
      Domoticz.Log("Connected successfully to: "+Connection.Address+":"+Connection.Port)
      self.sendNullValues()
    else:
      Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Connection.Address+":"+Connection.Port+" with error: "+Description)
      self.connection.Connect() # THIS FAILS
      
Error:
Error: CConnection_connect, connect request from 'Hosola' ignored. Transport is connected.

If I try to disconnect it first and reconnect in the onDisconnect call,I get another error:
Error: CConnection_disconnect, disconnection request from 'Hosola' ignored. Transport is not connected.

I am lost, how can i retry to establish a (the same) connection?

Used on version 3.7686

User avatar
Dnpwwo
Posts: 728
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: python plugin connection

Post by Dnpwwo » Friday 09 June 2017 5:31

@elgringo,

Looks strange to me :o

To answer you question: You should be able to connect and disconnect as you like using the same object.

Hard to say without seeing the rest of script, I would guess that

Code: Select all

self.connection != Connection
because both the Connect and Disconnect functions look at the same boolean prior to returning the errors you have reported (although there could be a bug in there somewhere). What happens if you do

Code: Select all

Connection.Connect()
If you post your whole script I'm happy to look at it or even try it on my dev setup.
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

elgringo
Posts: 127
Joined: Thursday 18 May 2017 8:08
Target OS: Raspberry Pi
Domoticz version: Left
Contact:

Re: python plugin connection

Post by elgringo » Friday 09 June 2017 5:59

The complete script can be found here:
https://github.com/ericstaal/domoticz/t ... gin/hosola

I tried a lot of things yesterday but could get it to work.

There is a variable with the connection. When the plugin is started the connect is called. When the inverter is offline (nighttime) it will take about 2 minutes to return with a timeout. The next heartbeat a check is done if the connection is connected if not retry. The retry causes the error.
I understand it is impossible to connect while a connection is beinig made. I don't know what the timeout is for a onconnect call of if it possible to set the timeout.

EDIT:
Does the connect() function return a value is succeed or not?
If so I can destroy the current connection and create a new one to retry.

Edit2:
I am going to try a version which a check function (only related functions are shown):

Code: Select all

def checkConnection(self):
    # Check connection and connect none
    isConnected = False
    if self.connection is None:
      self.connection = Domoticz.Connection(Name="Binary", Transport="TCP/IP", Protocol="None", Address=Parameters["Address"], Port=Parameters["Port"])
    
    if self.connection.Connected() == True:
      isConnected = True
    else:
      if self.busyConnecting:
        isConnected = False
      else:
        self.oustandingMessages = 0
        self.busyConnecting = True
        self.connection.Connect() # if failed (??) set self.busyConnecting back to false, create new conenction (??)
        isConnected = False
    return isConnected
    
def onConnect(self, Connection, Status, Description):
    self.busyConnecting = False
    if (Status == 0):
      Domoticz.Log("Connected successfully to: "+Connection.Address+":"+Connection.Port)
      self.sendNullValues()
    else:
      Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Connection.Address+":"+Connection.Port+" with error: "+Description)
      # destroy connection and create a new one
      self.connection = None
    return    
    
 def onHeartbeat(self):
    # send identifier
    if self.checkConnection(): # checks if connect if not retry
      if self.oustandingMessages > int(Parameters["Mode2"]):
        self.sendNullValues()
        self.connection.Disconnect()
      else:
        if self.inverterId is not None: # Only send message if inverter id is known
          self.oustandingMessages = self.oustandingMessages + 1
          if (len(self.readBytes) > 0):
            Domoticz.Error("Erased (send new request): "+createByteString(self.readBytes))
            self.readBytes = bytearray()  # clear all bytes read
          self.connection.Send(self.inverterId)
    
  def onDisconnect(self, Connection):
    Domoticz.Log("Disconnected from: "+Connection.Address+":"+Connection.Port)
    self.connection = None # reset connection
    return    

elgringo
Posts: 127
Joined: Thursday 18 May 2017 8:08
Target OS: Raspberry Pi
Domoticz version: Left
Contact:

Re: python plugin connection

Post by elgringo » Friday 09 June 2017 19:53

The part when resetting the connection to none worked. :)

However I updated my Domoticz to V3.7698 and now the binary connection does not work anymore:
2017-06-09 19:52:02.268 Error: CPlugin:CConnection_init, unable to find module for current interpreter.
2017-06-09 19:52:02.268 Error: CConnection_connect:, illegal operation, Plugin has not started yet.


I guess the protocol 'None' is removed. Is this a bug, or is there an alternatieve way to communicate without protocol? I just want to send raw binairy data

EDIT:
Found the cause: The connection was trying to be made before the IO system was up.

Fixed it by trying to connect in the first heartbeat

I have updated the source on github where all the bugs are solved

User avatar
Dnpwwo
Posts: 728
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: python plugin connection

Post by Dnpwwo » Saturday 10 June 2017 3:09

@elgringo,

As I think you've found, no protocols have been removed. None in particular is required for binary data.

The plugin framework is not usable until onStart because Python is just importing the plugin, making framework calls before then will cause a message similar to 'Plugin has not started yet' to be logged depending on what you call.

I create initial connections in onStart but you can use onHeartbeat as well.

I hadn't really thought about a connection timeout because it is asynchronous, I will see if I can add one in.

I could also add a 'Connecting()' method to show if a connection was in progress.
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

casimir
Posts: 6
Joined: Wednesday 18 April 2018 15:37
Target OS: Linux
Domoticz version:
Contact:

Re: python plugin connection

Post by casimir » Wednesday 18 April 2018 15:57

Hello,

One related question regarding your new MQTT Subscribe/Publish sample codes:

Code: Select all

    def onConnect(self, Connection, Status, Description):
        Domoticz.Debug("onConnect called ...")
        if self._shutdown is True: return
        if (Status == 0):
            Domoticz.Debug("MQTT connected successfully ...")
            sendData = { 'Verb' : 'CONNECT',
                         'ID' : "645364363" }
            Connection.Send(sendData)
            time.sleep(1)
            Domoticz.Debug("start subscription to "+ Parameters["Mode1"])
            #self.mqttConn.Send({'Verb' : 'SUBSCRIBE', 'PacketIdentifier': 1001, 'Topics': [{'Topic':Parameters["Mode1"], 'QoS': 0}]})
            self.mqttConn.Send({'Verb' : 'SUBSCRIBE', 'Topics': [{'Topic':Parameters["Mode1"], 'QoS': 0}]})
        else:
            Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Parameters["Address"]+":"+Parameters["Port"]+" with error: "+Description)
            Domoticz.Debug("retry to connect ...")
            time.sleep(1)
            self.mqttConn.Connect()
Could you tell me what is this ID in message 'Verb':'CONNECT' sent via Connection.Send(...) ?
... and why do you need to tell to 'Connection' that yes we're connected ??

Thanks
François

User avatar
Dnpwwo
Posts: 728
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: python plugin connection

Post by Dnpwwo » Friday 20 April 2018 7:58

@casimir,

If you want the details of the CONNECT command you can look here: http://docs.oasis-open.org/mqtt/mqtt/v3 ... c398718028

But the key part is that the ID maps to the ClientId:
The Client Identifier (ClientId) identifies the Client to the Server. Each Client connecting to the Server has a unique ClientId. The ClientId MUST be used by Clients and by Servers to identify state that they hold relating to this MQTT Session between the Client and the Server.

The Server MUST allow ClientIds which are between 1 and 23 UTF-8 encoded bytes in length, and that contain only the characters "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
basically it just needs to be any unique string
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

casimir
Posts: 6
Joined: Wednesday 18 April 2018 15:37
Target OS: Linux
Domoticz version:
Contact:

Re: python plugin connection

Post by casimir » Monday 23 April 2018 10:12

@Dnpwwo

Ok thanks for your answer. What was a bit strange for me is that we usually let the client set a random clientID.
However, this kind of <<double connect>> (i.e you connect via MQTT AND you use Connect.send ??)
I also noticed a 'testHey' ... probably for you to check that we've successfully subsribed to a MQTT topic ...

Code: Select all

[2018-04-23 10:00:54.102 (neOcampus_inventory) Pushing 'onMessageCallback' on to queue
2018-04-23 10:00:54.102 (neOcampus_inventory) Processing 'onMessageCallback' message
2018-04-23 10:00:54.102 (neOcampus_inventory) Calling message handler 'onMessage'.
2018-04-23 10:00:54.102 (neOcampus_inventory) onMessage called with: CONNACK
2018-04-23 10:00:54.102 (neOcampus_inventory) >'Description':'Connection Accepted'
2018-04-23 10:00:54.102 (neOcampus_inventory) >'Status': 0
2018-04-23 10:00:54.102 (neOcampus_inventory) >'Verb':'CONNACK'
2018-04-23 10:00:54.102 (neOcampus_inventory) connection ACK received :)
2018-04-23 10:00:54.111 (neOcampus_inventory) Pushing 'ReadEvent' on to queue
2018-04-23 10:00:54.113 (neOcampus_inventory) Pushing 'ReadEvent' on to queue
2018-04-23 10:00:54.152 (neOcampus_inventory) Processing 'ReadEvent' message
2018-04-23 10:00:54.152 (neOcampus_inventory) Received 5 bytes of data
2018-04-23 10:00:54.152 (neOcampus_inventory) 90 03 00 01 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �....
2018-04-23 10:00:54.152 (neOcampus_inventory) Pushing 'onMessageCallback' on to queue
2018-04-23 10:00:54.152 (neOcampus_inventory) Processing 'ReadEvent' message
2018-04-23 10:00:54.152 (neOcampus_inventory) Received 21 bytes of data
2018-04-23 10:00:54.152 (neOcampus_inventory) 31 13 00 0e 54 65 73 74 54 6f 70 69 63 2f 74 65 73 74 48 65 1...TestTopic/testHe
2018-04-23 10:00:54.152 (neOcampus_inventory) 79 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. y
2018-04-23 10:00:54.152 (neOcampus_inventory) Pushing 'onMessageCallback' on to queue
2018-04-23 10:00:54.152 (neOcampus_inventory) Processing 'onMessageCallback' message
2018-04-23 10:00:54.152 (neOcampus_inventory) Calling message handler 'onMessage'.
2018-04-23 10:00:54.152 (neOcampus_inventory) onMessage called with: SUBACK
2018-04-23 10:00:54.152 (neOcampus_inventory) > List 'Topics' (1):
2018-04-23 10:00:54.152 (neOcampus_inventory) ---> Dict (2):
2018-04-23 10:00:54.152 (neOcampus_inventory) ------>'Status': 0
2018-04-23 10:00:54.152 (neOcampus_inventory) ------>'Description':'Success - Maximum QoS 0'
2018-04-23 10:00:54.152 (neOcampus_inventory) >'PacketIdentifier': 1
2018-04-23 10:00:54.152 (neOcampus_inventory) >'Verb':'SUBACK'
2018-04-23 10:00:54.152 (neOcampus_inventory) subscription ACK received :)
2018-04-23 10:00:54.152 (neOcampus_inventory) Processing 'onMessageCallback' message
2018-04-23 10:00:54.152 (neOcampus_inventory) Calling message handler 'onMessage'.
2018-04-23 10:00:54.152 (neOcampus_inventory) onMessage called with: PUBLISH
2018-04-23 10:00:54.152 (neOcampus_inventory) >'Payload': b'Hey'
2018-04-23 10:00:54.152 (neOcampus_inventory) >'Retain': True
2018-04-23 10:00:54.152 (neOcampus_inventory) >'DUP': 0
2018-04-23 10:00:54.152 (neOcampus_inventory) >'QoS': 0
2018-04-23 10:00:54.152 (neOcampus_inventory) >'Verb':'PUBLISH'
2018-04-23 10:00:54.152 (neOcampus_inventory) >'Topic':'TestTopic/test'
2018-04-23 10:00:54.152 (neOcampus_inventory) message received :)
2018-04-23 10:01:02.499 (neOcampus_inventory) Pushing 'onHeartbeatCallback' on to queue
2018-04-23 10:01:02.519 (neOcampus_inventory) Processing 'onHeartbeatCallback' message
2018-04-23 10:01:02.519 (neOcampus_inventory) Calling message handler 'onHeartbeat'.
2018-04-23 10:01:02.519 (neOcampus_inventory) onHeartbeat called ...
/code]

In the end, what i'm trying to do is to automate sensors / actators creation upon received MQTT topics ... because we have hundreds of sensors and i'm tired to use nodered to translate our topics to Domoticz's idx ... thus this plugin ;)

Thanks
François

User avatar
Dnpwwo
Posts: 728
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: python plugin connection

Post by Dnpwwo » Monday 23 April 2018 13:15

@casimir,

Random IDs are okay unless you want to reconnect to a previous session, it seemed more flexible to let the plugin assign an ID so it could choose to reconnect or not (I'm all about giving people choice 8-) ).

Couldn't find a 'Hey' in the code and it doesn't need one, if you post your plugin I'm happy to run it locally and look for it. What version are you running?

The 2 satge connect is to keep things consistent in the plugin framework. onConnect is called for transport level connection (in your case TCP/IP). Protocols run on top of transports in the plugin framework and MQTT has it's own connect message, other protocols have different requirements.
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

casimir
Posts: 6
Joined: Wednesday 18 April 2018 15:37
Target OS: Linux
Domoticz version:
Contact:

Re: python plugin connection

Post by casimir » Monday 23 April 2018 16:43

@Dnpwwo

ah ok, thanks for your explanation ... anyway, i've not been able to find this 'TestHey' in your code ... and it occurs every time i subscribe to a topic (and it features a retain=True field!!). I'm running the latest git version in a docker.

Now if you look at the onConnect, this is where whenever you call Connection.send() --> something is published (or maybe some old forgotten code running in our servers :s)
(moreover, it will be an issue later because we also have topics upon you cannot publish (just listen))

Thanks for being so responsive :D
François

Code: Select all

#
# neOCampus
#
# This plug-in will try to automatically instantiate sensors / actuators according
#   to their types.
#
# Based on:
# Python Plugin MQTT Subscribe Example
# Author: Dnpwwo
#
# Thiebolt F.   Apr.18 initial release
#
"""
<plugin key="MQTTinventory" name="MQTT inventory for neOCapmpus" author="casimir" version="0.0.1" externallink="http://neocampus.univ-tlse3.fr/">
    <params>
        <param field="Address" label="IP Address" width="200px" required="true" default="neocampus.univ-tlse3.fr"/>
        <param field="Port" label="Connection" required="true" width="200px" default="1883"/>
        <param field="Username" label="Username" width="200px"/>
        <param field="Password" label="Password" width="200px"/>
        <param field="Mode1" label="Topic" width="125px" default="TestTopic/#"/>
        <param field="Mode6" label="Debug" width="75px">
            <options>
                <option label="True" value="Debug"/>
                <option label="False" value="Normal"  default="true" />
            </options>
        </param>
    </params>
</plugin>
"""

import Domoticz
import time

# Extend python's modules search path                                                                                                                                                                                           
import os                                                                                                                                                                                                                       
import sys                                                                                                                                                                                                                      
_pythonVersion="python" + str(sys.version_info.major) + "." + str(sys.version_info.minor)                                                                                                                                       
                                                                                                                                                                                                                                
_path2add=os.path.join("/usr/local/lib", _pythonVersion, "dist-packages")                                                                                                                                                       
if (os.path.exists(_path2add) and not os.path.abspath(_path2add) in sys.path):                                                                                                                                                  
    sys.path.append(os.path.abspath(_path2add))                                                                                                                                                                                 
                                                                                                                                                                                                                                
_path2add=os.path.join("/usr/local/lib", _pythonVersion, "site-packages")                                                                                                                                                       
if (os.path.exists(_path2add) and not os.path.abspath(_path2add) in sys.path):                                                                                                                                                  
    sys.path.append(os.path.abspath(_path2add))                                                                                                                                                                                 
                                                                                                                                                                                                                                
# JSON parser Cython written                                                                                                                                                                                                    
import msgpack                                                                                                                                                                                                                  
                                                                                                                                                                                                                                
                                                                                                                                                                                                                                
class BasePlugin:                                                                                                                                                                                                               
                                                                                                                                                                                                                                
    # Domoticz attributes                                                                                                                                                                                                       
    enabled = True                                                                                                                                                                                                              
    mqttConn = None                                                                                                                                                                                                             
                                                                                                                                                                                                                                
    # attributes
    DEFL_HEARTBEAT   = 30
    DEFL_MQTT_KEEPALIVE = 60

    unpacker = None
    _heartBeat = None
    _heartBeatCnt = 0
    _connack = False
    _suback = False
    _shutdown = False

    
    def __init__(self):
        self._heartBeat = __class__.DEFL_HEARTBEAT
        self.unpacker = msgpack.Unpacker(encoding='utf-8')
        return


    def onStart(self):
        Domoticz.Debug("onStart called ...")
        Domoticz.Debug("Devices count = %d" % len(Devices))
        if Parameters["Mode6"].lower() == "debug":
            Domoticz.Debugging(1)
        # change heartbeat
        if self._heartBeat is None or not isinstance(self._heartBeat,int):
            Domoticz.Debug("\tDomoticz HeartBeat as default ... (10s)")
        else:
            Domoticz.Heartbeat(self._heartBeat)
            Domoticz.Debug("\tHeartbeat set to %d" % self._heartBeat)
        self._shutdown = False
        self._connack = False
        self._suback = False
        self._heartBeatCnt = 0
        DumpConfigToLog()
        self.mqttConn = Domoticz.Connection(Name="MQTT Test", Transport="TCP/IP", Protocol="MQTT", Address=Parameters["Address"], Port=Parameters["Port"])
        self.mqttConn.Connect()


    def onStop(self):
        Domoticz.Debug("onStop called ...")
        self._shutdown = True
        if (self.mqttConn.Connected()):
            # unsubscribe
            Domoticz.Debug("\tunsubscribe ...")
            self.mqttConn.Send({'Verb' : 'UNSUBSCRIBE', 'Topics': [Parameters["Mode1"]]})
            # no need to wait for confirmation
            self._suback = False
            Domoticz.Debug("\tdisconnect ...")
            self.mqttConn.Send({ 'Verb' : 'DISCONNECT' })
            # no need to wait for confirmation
            self._connack = False


    def onConnect(self, Connection, Status, Description):
        Domoticz.Debug("onConnect called ...")
        if self._shutdown is True: return
        if (Status == 0):
            # i.e TCP connexion to MQTT broker is established 
            Domoticz.Debug("MQTT connection is on progress ...")
            # ... no we need to establish the MQTT link
            # generate our MQTT clientID
            _clientID = "_".join(["domoticz", Parameters['Key'], Parameters['Username']])
            sendData = { 'Verb' : 'CONNECT',  'ID' : _clientID }
            #sendData = { 'Verb' : 'CONNECT' } # pseudo-random clientID
            Connection.Send(sendData)
            time.sleep(1)
            # [apr.18] only support for ONE topic
            Domoticz.Debug("start subscription to "+ Parameters["Mode1"])
            #self.mqttConn.Send({'Verb' : 'SUBSCRIBE', 'PacketIdentifier': 1001, 'Topics': [{'Topic':Parameters["Mode1"], 'QoS': 0}]})
            self.mqttConn.Send({'Verb' : 'SUBSCRIBE', 'Topics': [{'Topic':Parameters["Mode1"], 'QoS': 0}]})
        else:
            Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Parameters["Address"]+":"+Parameters["Port"]+" with error: "+Description)
            Domoticz.Debug("retry to connect ...")
            time.sleep(1)
            self.mqttConn.Connect()            


    def onMessage(self, Connection, Data):
        Domoticz.Log("onMessage called with: "+Data["Verb"])
        DumpDictionaryToLog(Data)
        
        if Data.get('Verb','').lower() == "pingresp":
            Domoticz.Debug("ping response received :)")
            return

        if Data.get('Verb','').lower() == "connack":
            if Data.get('Status',42) == 0:
                Domoticz.Debug("connection ACK received :)")
                self._connack = True
                return
            else:
                Domoticz.Log("connection ACK with error code: " + str(Data.get('Status',42)))
                return

        if Data.get('Verb','').lower() == "suback":
            #Domoticz.Debug("suback payload = " + str(Data))
            # suback payload = {'Topics': [{'Status': 0, 'Description': 'Success - Maximum QoS 0'}], 'Verb': 'SUBACK', 'PacketIdentifier': 1}
            # [apr.18] only support for ONE topic
            if (Data['Topics'][0])['Status'] == 0:
                Domoticz.Debug("subscription ACK received :)")
                self._suback = True
                return
            else:
                Domoticz.Debug("subscribe ACK with errors: " + str(Data))
                return

        if Data.get('Verb','').lower() == "publish":
            Domoticz.Debug("message received :)")
            # check this is a relevant message
            #if Data['Topic'] == "/".join([Parameters['Mode1'],"test"]):
            if Data['Topic'].endswith("/test") and Data['Retain'] is True:
                # probably Domoticz test message ...
                Domoticz.Debug("domoticz test message received ... discard!")
                return
            # ok, start to process message
            _processMQTTmessage( topic=Data['Topic'], payload=Data.get('Payload') )
            return

        # unknown message ?!?!
        Domoticz.Debug("unhandled message: "+ str(Data))


    def onCommand(self, Unit, Command, Level, Hue):
        Domoticz.Log("onCommand called for Unit " + str(Unit) + ": Parameter '" + str(Command) + "', Level: " + str(Level))


    def onNotification(self, Name, Subject, Text, Status, Priority, Sound, ImageFile):
        Domoticz.Log("onNotification received: " + Name + "," + Subject + "," + Text + "," + Status + "," + str(Priority) + "," + Sound + "," + ImageFile)


    def onDisconnect(self, Connection):
        Domoticz.Log("onDisconnect called ...")
        
        if self._shutdown is True: return

        # TODO:trying to guess
        Domoticz.Debug("Connection: " + str(Connection))

        # [apr.18] there's no (simple) way to know if this is a regular disconnect (e.g user disable action)
        # or disconnect due to network outage for example.
        # Anyway, reconnect will silently fails is device is disabled ... or retry to connect otherwise

        self._connack = False
        self._suback = False
        Domoticz.Debug("\ttrying to reconnect ...")
        time.sleep(1)
        self.mqttConn.Connect()


    def onHeartbeat(self):
        Domoticz.Log("onHeartbeat called ...")
        if self._shutdown is True: return
        if (self.mqttConn.Connected()):
            # manage MQTT keep alive (if nothing has been sent)
            if (self._heartBeatCnt % (__class__.DEFL_MQTT_KEEPALIVE / self._heartBeat) ) == 0:
                Domoticz.Debug("... sending PINGREQ to MQTT broker ...")
                self.mqttConn.Send({ 'Verb' : 'PING' })

            self._heartBeatCnt += 1

'''
*** to update a device values:

def UpdateIcon(Unit, iconID):
    if Unit not in Devices: return
    d = Devices[Unit]
    if d.Image != iconID: d.Update(d.nValue, d.sValue, Image=iconID)

*** to process messages:
import msgpack
    def onMessage(self, Connection, Data):
        try:
            self.unpacker.feed(Data)
            for result in self.unpacker:

                Domoticz.Debug("Got: %s" % result)

                if 'exception' in result: return

                if result['cmd'] == 'status':

                    UpdateDevice(self.statusUnit,
                                 (1 if result['state_code'] in [5, 6, 11] else 0), # ON is Cleaning, Back to home, Spot cleaning
                                 self.states.get(result['state_code'], 'Undefined')
                                 )

                    UpdateDevice(self.batteryUnit, result['battery'], str(result['battery']), result['battery'],
                                 AlwaysUpdate=(self.heartBeatCnt % 100 == 0))

                    if Parameters['Mode5'] == 'dimmer':
                        UpdateDevice(self.fanDimmerUnit, 2, str(result['fan_level'])) # nValue=2 for show percentage, instead ON/OFF state
                    else:
                        level = {38: 10, 60: 20, 77: 30, 90: 40}.get(result['fan_level'], None)
                        if level: UpdateDevice(self.fanSelectorUnit, 1, str(level))

                elif result['cmd'] == 'consumable_status':

                    mainBrush = cPercent(result['main_brush'], 300)
                    sideBrush = cPercent(result['side_brush'], 200)
                    filter = cPercent(result['filter'], 150)
                    sensors = cPercent(result['sensor'], 30)

                    UpdateDevice(self.cMainBrushUnit, mainBrush, str(mainBrush), AlwaysUpdate=True)
                    UpdateDevice(self.cSideBrushUnit, sideBrush, str(sideBrush), AlwaysUpdate=True)
                    UpdateDevice(self.cFilterUnit, filter, str(filter), AlwaysUpdate=True)
                    UpdateDevice(self.cSensorsUnit, sensors, str(sensors), AlwaysUpdate=True)

        except msgpack.UnpackException as e:
            Domoticz.Error('Unpacker exception [%s]' % str(e))
'''

global _plugin
_plugin = BasePlugin()

def onStart():
    global _plugin
    _plugin.onStart()

def onStop():
    global _plugin
    _plugin.onStop()

def onConnect(Connection, Status, Description):
    global _plugin
    _plugin.onConnect(Connection, Status, Description)

def onMessage(Connection, Data):
    global _plugin
    _plugin.onMessage(Connection, Data)

def onCommand(Unit, Command, Level, Hue):
    global _plugin
    _plugin.onCommand(Unit, Command, Level, Hue)

def onNotification(Name, Subject, Text, Status, Priority, Sound, ImageFile):
    global _plugin
    _plugin.onNotification(Name, Subject, Text, Status, Priority, Sound, ImageFile)

def onDisconnect(Connection):
    global _plugin
    _plugin.onDisconnect(Connection)

def onHeartbeat():
    global _plugin
    _plugin.onHeartbeat()



#
# neOCampus functions
def _processMQTTmessage( topic=None, payload=None ):
    Domoticz.Debug("_processMQTTmessage: Topic = %s  Payload = %20s ..." % (topic,str(payload)) )
    #TODO!
    return



#
# Generic helper functions
def DumpConfigToLog():
    for x in Parameters:
        if Parameters[x] != "":
            Domoticz.Debug( "'" + x + "':'" + str(Parameters[x]) + "'")
    Domoticz.Debug("Device count: " + str(len(Devices)))
    for x in Devices:
        Domoticz.Debug("Device:           " + str(x) + " - " + str(Devices[x]))
        Domoticz.Debug("Device ID:       '" + str(Devices[x].ID) + "'")
        Domoticz.Debug("Device Name:     '" + Devices[x].Name + "'")
        Domoticz.Debug("Device nValue:    " + str(Devices[x].nValue))
        Domoticz.Debug("Device sValue:   '" + Devices[x].sValue + "'")
        Domoticz.Debug("Device LastLevel: " + str(Devices[x].LastLevel))
    return

def DumpDictionaryToLog(theDict, Depth=""):
    if isinstance(theDict, dict):
        for x in theDict:
            if isinstance(theDict[x], dict):
                Domoticz.Log(Depth+"> Dict '"+x+"' ("+str(len(theDict[x]))+"):")
                DumpDictionaryToLog(theDict[x], Depth+"---")
            elif isinstance(theDict[x], list):
                Domoticz.Log(Depth+"> List '"+x+"' ("+str(len(theDict[x]))+"):")
                DumpListToLog(theDict[x], Depth+"---")
            elif isinstance(theDict[x], str):
                Domoticz.Log(Depth+">'" + x + "':'" + str(theDict[x]) + "'")
            else:
                Domoticz.Log(Depth+">'" + x + "': " + str(theDict[x]))

def DumpListToLog(theList, Depth):
    if isinstance(theList, list):
        for x in theList:
            if isinstance(x, dict):
                Domoticz.Log(Depth+"> Dict ("+str(len(x))+"):")
                DumpDictionaryToLog(x, Depth+"---")
            elif isinstance(x, list):
                Domoticz.Log(Depth+"> List ("+str(len(theList))+"):")
                DumpListToLog(x, Depth+"---")
            elif isinstance(x, str):
                Domoticz.Log(Depth+">'" + x + "':'" + str(theList[x]) + "'")
            else:
                Domoticz.Log(Depth+">'" + x + "': " + str(theList[x]))

User avatar
Dnpwwo
Posts: 728
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: python plugin connection

Post by Dnpwwo » Sunday 29 April 2018 16:11

@casimir,

You plugin code looks fine and I still can't find a default 'subscription' being sent. From the log you have posted there would have been a WriteDirective message processed and the output buffer would have been dumped in hex and text if Domoticz had sent something.

Could it be a 'welcome' message from the MQTT server you are attaching to?? If you are recycling the ClientID and the original subscription was marked 'retain' will it still keep sending you updates?

Could of things about your plugin that might be useful:
  • in onConnect you send 2 messages and sleep the plugin to separate them. In a callback driven API that is not ideal because it freezes the plugins for a second. The Send function has a parameter to do this for you, try adding "delay=1" as the last parameter of the Send. Another way to do it would be to Send the SUBSCRIBE in onMessage when a 'connack' is processed.
  • You might want to have a look at https://github.com/emontnemery/domoticz ... /plugin.py. The author created a Python MQTT client class which might be useful
  • You can also now do finer grained debug messaging, have a look at https://github.com/dnpwwo/Domoticz-Kodi ... /plugin.py at line 56 to see a better 'Mode6' definition, you need to tweak onStart as well
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

casimir
Posts: 6
Joined: Wednesday 18 April 2018 15:37
Target OS: Linux
Domoticz version:
Contact:

Re: python plugin connection

Post by casimir » Monday 30 April 2018 0:56

@dnpwwo

... it was the 'Hey' in your code that lead me to think that this 'Test' message was yours ... while it was an old greetings no one remembered ;)
Thank you for the debug codes, the connack and suback ways to manage. The plugin is now working and automatically create sensors upon MQTT message arrival :)
I also noticed that when you Create a device ... the whole plugin reset if you attempt to update its values immediately.
One annoying point was related to dimmers: it took me some time to discover that if you want the dimmer to keep the value, you must update with nValue=2 (?!?!) and sValue containing the integer ... and i must confess this is a pain having to read code from others to discover this kind of subtlities (didn't found in wiki); Anyway, with this plugin, we don't anymore are in need for node-red to map topics <--> idx, so hurrah :)

Thanks for your time :)

Code: Select all

#
# neOCampus
#
# This plug-in will try to automatically instantiate sensors / actuators according
#   to their types.
# neOCampus MQTT conventions: <building>/<room>/<kind>
#   e.g u4/302/luminosity --> message features a unitID (kind of UUID) along with a subID (i2c addr)
#
# Based on:
# Python Plugin MQTT Subscribe Example
# Author: Dnpwwo
#
# Notes:
#  - Switch Subtype (e.g DIMMER=7) --> domoticz/main/RFXNames.h
#
# Dr Thiebolt F.    Apr.18 initial release
#
"""                                                                                                                                                                                                  
<plugin key="MQTTinventory" name="MQTT inventory for neOCapmpus" author="casimir" version="0.0.1" externallink="http://neocampus.univ-tlse3.fr/">                                                    
    <params>                                                                                                                                                                                         
        <param field="Address" label="IP Address" width="200px" required="true" default="neocampus.univ-tlse3.fr"/>                                                                                  
        <param field="Port" label="Connection" required="true" width="200px" default="1883"/>                                                                                                        
        <param field="Username" label="Username" width="200px"/>                                                                                                                                     
        <param field="Password" label="Password" width="200px"/>                                                                                                                                     
        <param field="Mode1" label="Topic" width="125px" default="TestTopic/#"/>                                                                                                                     
        <param field="Mode6" label="Debug" width="150px">                                                                                                                                            
            <options>                                                                                                                                                                                
                <option label="None" value="0"  default="true" />                                                                                                                                    
                <option label="Python Only" value="2"/>                                                                                                                                              
                <option label="Basic Debugging" value="62"/>                                                                                                                                         
                <option label="Basic+Messages" value="126"/>                                                                                                                                         
                <option label="Connections Only" value="16"/>                                                                                                                                        
                <option label="Connections+Queue" value="144"/>                                                                                                                                      
                <option label="All" value="-1"/>                                                                                                                                                     
            </options>                                                                                                                                                                               
        </param>                                                                                                                                                                                     
    </params>                                                                                                                                                                                        
</plugin>                                                                                                                                                                                            
"""


#
# Import zone
#
#
import Domoticz
import time
import json
from datetime import datetime
import threading

# Extend Domoticz's python modules search paths with sites packages
import os
import sys
import site
_path2add=site.getsitepackages()
for _p in _path2add:
    if (os.path.exists(_p) and not os.path.abspath(_p) in sys.path):
        sys.path.append(os.path.abspath(_p))


#
# Variables
#
#
NEOCAMPUS_SENSORS_TYPES     = [ "temperature", "co2", "luminosity", "hygro", "humidity", "energy", "digital", "noise" ]
NEOCAMPUS_ACTUATORS_TYPES   = [ "lighting", "shutter", "display" ]



#
# Plugin class
#
#
class BasePlugin:

    # Domoticz attributes
    enabled = True
    mqttConn = None
    
    # attributes
    DEFL_HEARTBEAT      = 20
    DEFL_MQTT_KEEPALIVE = 60
    DOMOTICZ_MAXUNITS   = 255           # Units range from 1 to 255
    NEOCAMPUS_TESTTOPIC = 'TestTopic'   # neOCampus sandbox
    NOISE_THRESHOLD_MAX = 1000          # maximum value for noise threshold (pulses counts over scan_window)

                                        # [apr.18] noise sensors emit messages whenever noise is
                                        # detected ... and stay quiet when ambiant is calm, thus
                                        # we need a kind of timer to cooldown the noise indicator
    heartBeatNoiseProcessing = None     # Noise management through heartBeat
    heartBeatNoiseProcessingLock = None

    _heartBeat = None
    _heartBeatCnt = 0
    _connack = False
    _suback = False
    _shutdown = False

    _devicesNamesDict   = None
    _availableUnits     = None
    

    def __init__(self):
        self._heartBeat = __class__.DEFL_HEARTBEAT
        self._availableUnits = list(range( 1, __class__.DOMOTICZ_MAXUNITS+1 ))
        self._devicesNamesDict = dict()
        self.heartBeatNoiseProcessing = list()
        self.heartBeatNoiseProcessingLock = threading.Lock()


    def onStart(self):
        Domoticz.Log("onStart called ...")

        # Debug mode ?
        if Parameters["Mode6"] != "0":
            try:
                Domoticz.Debugging(int(Parameters["Mode6"]))
            except Exception as ex:
                Domoticz.Debugging(1)

        # parse existing devices
        if len(Devices):
            Domoticz.Debug("Devices count = %d" % len(Devices))
            # parse existing devices
            self._parseDevices()
            Domoticz.Debug("Currently registered devices (Name:idx) :")
            DumpDictionaryToLog( self._devicesNamesDict )
            Domoticz.Debug("Currently available units numbers :" +str(self._availableUnits) )

        # change heartbeat
        if self._heartBeat is None or not isinstance(self._heartBeat,int):
            Domoticz.Debug("\tDomoticz HeartBeat as default ... (10s)")
        else:
            Domoticz.Heartbeat(self._heartBeat)
            Domoticz.Debug("\tHeartbeat set to %d" % self._heartBeat)

        self._shutdown = False
        self._connack = False
        self._suback = False
        self._heartBeatCnt = 0

        DumpConfigToLog()

        self.mqttConn = Domoticz.Connection(Name="MQTT neOCampus", Transport="TCP/IP", Protocol="MQTT", Address=Parameters["Address"], Port=Parameters["Port"])
        self.mqttConn.Connect()


    def onStop(self):
        Domoticz.Debug("onStop called ...")
        self._shutdown = True
        if (self.mqttConn.Connected()):
            # unsubscribe
            Domoticz.Debug("\tunsubscribe ...")
            self.mqttConn.Send({'Verb' : 'UNSUBSCRIBE', 'Topics': [Parameters["Mode1"]]})
            # no need to wait for confirmation
            self._suback = False
            Domoticz.Debug("\tdisconnect ...")
            self.mqttConn.Send({ 'Verb' : 'DISCONNECT' })
            # no need to wait for confirmation
            self._connack = False


    def onConnect(self, Connection, Status, Description):
        Domoticz.Debug("onConnect called ...")
        if self._shutdown is True: return
        if (Status == 0):
            # i.e TCP connexion to MQTT broker is established 
            Domoticz.Debug("MQTT connection is on progress ...")
            # ... now we need to establish the MQTT link
            # generate our MQTT clientID
            _clientID = "_".join(["domoticz", Parameters['Key'], Parameters['Username']])
            sendData = { 'Verb' : 'CONNECT',  'ID' : _clientID }
            #sendData = { 'Verb' : 'CONNECT' } # pseudo-random clientID
            Connection.Send(sendData)
            # ... and now wait for a connack to start the subscription
        else:
            Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Parameters["Address"]+":"+Parameters["Port"]+" with error: "+Description)
            Domoticz.Debug("retry to connect ...")
            time.sleep(1)   # to avoid too many attempts per second
            self.mqttConn.Connect()            


    def onMessage(self, Connection, Data):
        Domoticz.Log("onMessage called with: "+Data["Verb"])
        DumpDictionaryToLog(Data)
        
        if Data.get('Verb','').lower() == "pingresp":
            Domoticz.Debug("ping response received :)")
            return

        if Data.get('Verb','').lower() == "connack":
            if Data.get('Status',42) == 0:
                Domoticz.Debug("connection ACK received :)")
                self._connack = True
                # [apr.18] only support for ONE topic to subscribe to
                Domoticz.Debug("start subscription to "+ Parameters["Mode1"])
                self.mqttConn.Send({'Verb' : 'SUBSCRIBE', 'Topics': [{'Topic':Parameters["Mode1"], 'QoS': 0}]})
                return
            else:
                Domoticz.Log("connection ACK with error code: " + str(Data.get('Status',42)))
                # retry to establish MQTT connection ...
                time.sleep(1)   # to avoid too many attempts per second
                self.mqttConn.Connect()
                return

        if Data.get('Verb','').lower() == "suback":
            #Domoticz.Debug("suback payload = " + str(Data))
            # suback payload = {'Topics': [{'Status': 0, 'Description': 'Success - Maximum QoS 0'}], 'Verb': 'SUBACK', 'PacketIdentifier': 1}
            # [apr.18] only support for ONE topic
            if (Data['Topics'][0])['Status'] == 0:
                Domoticz.Debug("subscription ACK received :)")
                self._suback = True
                return
            else:
                Domoticz.Debug("subscribe ACK with errors: " + str(Data))
                return

        if Data.get('Verb','').lower() == "publish":
            Domoticz.Debug("message received :)")

            # Test message ?
            if Data['Topic'].endswith("/test") and Data['Retain'] is True:
                # probably our old test message ...
                Domoticz.Debug("old test message received ... discard!")
                return

            if Data['Topic'].endswith("/command"):
                # command message
                Domoticz.Debug("command message received ... discard!")
                return

            # ok, start to process message
            self._processMQTTmessage( topic=Data['Topic'], payload=json.loads(Data.get('Payload').decode('utf-8')) )
            return

        # unknown message ?!?!
        Domoticz.Debug("unhandled message: "+ str(Data))


    def onCommand(self, Unit, Command, Level, Hue):
        Domoticz.Log("onCommand called for Unit " + str(Unit) + ": Parameter '" + str(Command) + "', Level: " + str(Level))
        
        # checks ...
        if Unit not in Devices: return
        _dev = Devices[ Unit ]
        topic = _dev.Options.get('topic')
        if topic is None or 'order' not in _dev.Options: return
        if not topic.endswith('/command'): return
        if Command.lower() != "set level":
            Domoticz.Debug( "unknown Command = '%s'" % Command )
            return

        # determine kind of sensor / actuator
        kindID = topic.split('/')[-2]

        if kindID == 'temperature':
            return self._processTemperatureCommand( _dev, topic, order=_dev.Options.get('order'), value=Level )
        elif kindID == 'luminosity':
            return self._processLuminosityCommand( _dev, topic, order=_dev.Options.get('order'), value=Level )
        elif kindID == 'noise':
            return self._processNoiseCommand( _dev, topic, order=_dev.Options.get('order'), value=Level )
        else:
            Domoticz.Debug("unknwown kindID = %s" % kindID)


    def onNotification(self, Name, Subject, Text, Status, Priority, Sound, ImageFile):
        Domoticz.Log("onNotification received: " + Name + "," + Subject + "," + Text + "," + Status + "," + str(Priority) + "," + Sound + "," + ImageFile)


    def onDeviceModified(self, Unit):
        Domoticz.Log("onDeviceModified called for Unit " + str(Unit))


    def onDeviceRemoved(self, Unit):
        Domoticz.Log("onDeviceRemoved called for Unit " + str(Unit))


    def onDisconnect(self, Connection):
        Domoticz.Log("onDisconnect called ...")
        
        if self._shutdown is True: return

        # TODO:trying to guess
        Domoticz.Debug("Connection: " + str(Connection))

        # [apr.18] there's no (simple) way to know if this is a regular disconnect (e.g user disable action)
        # or disconnect due to network outage for example.
        # Anyway, reconnect will silently fails is device is disabled ... or retry to connect otherwise

        self._connack = False
        self._suback = False
        Domoticz.Debug("\ttrying to reconnect ...")
        time.sleep(1)   # to avoid too many attempts per second
        self.mqttConn.Connect()


    def onHeartbeat(self):
        Domoticz.Log("onHeartbeat called ...")
        if self._shutdown is True: return

        if (self.mqttConn.Connected()):
            # manage MQTT keep alive (if nothing has been sent)
            if (self._heartBeatCnt % (__class__.DEFL_MQTT_KEEPALIVE / self._heartBeat) ) == 0:
                Domoticz.Debug("... sending PINGREQ to MQTT broker ...")
                self.mqttConn.Send({ 'Verb' : 'PING' })

            self._heartBeatCnt += 1

        # manage Noise cooldown ...
        _now = datetime.now()
        with self.heartBeatNoiseProcessingLock:
            for devUnit in self.heartBeatNoiseProcessing:
                if devUnit not in Devices:
                    self.heartBeatNoiseProcessing.remove( devUnit )
                    continue
                _dev = Devices[ devUnit ]
                try:
                    _lastupdate = datetime.strptime( _dev.LastUpdate, "%Y-%m-%d %H:%M:%S" )
                except Exception as ex:
                    Domoticz.Debug("unable to convert lastUpdate time from '%s': " %   + str(ex) )
                    continue
                # lastupdate > 3/5 heartbeat
                if int((_now - _lastupdate).total_seconds()) < int((3*self._heartBeat)/5): continue

                # ok cooldown noise indicator
                nValue = _dev.nValue - 1
                sValue = ( "noise cooldown" if nValue > 1 else "calm :)" )
                UpdateDevice( Dev=_dev, nValue=nValue, sValue=sValue)

                # remove noise processing ?
                if nValue <= 1:
                    self.heartBeatNoiseProcessing.remove( devUnit )


    # Function to parse existing devices to create a reverse dictionary
    # to ease check, insertion of new devices
    def _parseDevices( self ):
        for unit in Devices:
            '''
            # [apr.18] Devices names are prefixed with the Parameters['Name'] 
            # ... we'll remove from "neOCampus-inventory - u4/302/xxx" ton only keep 'u4/302/xxx"
            _regexedDeviceName = Devices[unit].Name.replace(Parameters['Name'] + ' - ','',1).lstrip()
            if _regexedDeviceName not in self._devicesNamesDict:
                self._devicesNamesDict[ _regexedDeviceName ] = unit
                try:
                    self._availableUnits.remove( unit )
                except ValueError as ex:
                    Domoticz.Error("Unit %d has been already removed ?!?! ... continuing :s" % unit)
            '''
            # [apr.18] we don't anymore rely on 'Name' field of Devices but Options associated that contains 'name'
            name = Devices[unit].Options['name']
            if name not in self._devicesNamesDict:
                self._devicesNamesDict[ name ] = unit
                try:
                    self._availableUnits.remove( unit )
                except ValueError as ex:
                    Domoticz.Error("Unit %d has been already removed ?!?! ... continuing :s" % unit)
                

    #
    # neOCampus functions
    #   _processMQTTmessage
    #   _publishMQTTmessage
    #   _processTemperatureCommand
    #   _processLuminosityCommand
    #   _processNoiseCommand
    #

    # process neOCampus MQTT messages
    def _processMQTTmessage( self, topic=None, payload=None ):

        Domoticz.Debug("_processMQTTmessage: Topic = %s  Payload = %20s ..." % (topic,str(payload)) )

        # itemID        = u4/302/noise [auto_36FC:57]
        # itemID_ctrl   = u4/302/noise/threshold
        # itemID_ctrl   = u4/302/noise/sensitivity

        # determine type (i.e sensor or actuator) of item
        kindID = topic.split('/')[-1]
        if kindID == 'device': return
        if kindID not in NEOCAMPUS_SENSORS_TYPES and kindID not in NEOCAMPUS_ACTUATORS_TYPES:
            Domoticz.Log("unknown kind of topic '%s' ..." % topic)
            return

        # if not 'value' in message ... discard
        if 'value' not in payload: return

        # compute item & item_control identity
        if topic.startswith(__class__.NEOCAMPUS_TESTTOPIC):
            itemID=topic.lstrip(__class__.NEOCAMPUS_TESTTOPIC+'/')
        else:
            itemID=topic
        itemID_ctrlBase = itemID
        if payload.get('subID') is not None:
            itemID+=" [%s:%d]" % (payload.get('unitID'),payload.get('subID'))
        else:
            itemID+=" [%s]" % payload.get('unitID','??')
        Domoticz.Debug("ItemID = " + itemID)


        # let's start to create/update sensors'n actuators
        _device = None
        _deviceUnit = None
        _options = { 'topic' : topic, 'name' : itemID }   # we save itemID in Options in case user change names in web UI

        if kindID == "temperature":

            if itemID not in self._devicesNamesDict and "value" in payload:
                # create sensor
                _device = None
                _deviceUnit = self._availableUnits[0]
                try:
                    _device = Domoticz.Device(Name=itemID,  Unit=_deviceUnit, TypeName="Temperature", Options=_options, Description="name:"+itemID, Used=1).Create()
                except Exception as ex:
                    Domoticz.Error("unable to CREATE sensor Name = '%s': " % (itemID) + str(ex))
                    return
                if _device is None: return
                # delete used unit number and add new device to reverse dict
                self._devicesNamesDict[ itemID ] = _deviceUnit
                self._availableUnits.pop(0)

                # sensor control
                #TODO: create a switch to send commands to sensor (e.g change acquisition frequency)
            elif itemID in self._devicesNamesDict:
                # Domoticz device already exists
                try:
                    _deviceUnit = self._devicesNamesDict[ itemID ]
                    _device = Devices[ _deviceUnit ]
                except Exception as ex:
                    Domoticz.Error("unable to FIND sensor Name = '%s': " % (itemID) + str(ex))
                    return
            else:
                return

            if "value" not in payload: return
            UpdateDevice( Dev=_device, sValue=str(payload['value']) )


        elif kindID == "luminosity":

            if itemID not in self._devicesNamesDict and "value" in payload:
                # create sensor
                _device = None
                _deviceUnit = self._availableUnits[0]
                try:
                    _device = Domoticz.Device(Name=itemID,  Unit=_deviceUnit, TypeName="Illumination", Options=_options, Description="name:"+itemID, Used=1).Create()
                except Exception as ex:
                    Domoticz.Error("unable to CREATE sensor Name = '%s': " % (itemID) + str(ex))
                    return 
                if _device is None: return
                # delete used unit number and add new device to reverse dict
                self._devicesNamesDict[ itemID ] = _deviceUnit
                self._availableUnits.pop(0)

                # sensor control
                #TODO: create a switch to send commands to sensor (e.g change acquisition frequency)
            elif itemID in self._devicesNamesDict:
                try:
                    _deviceUnit = self._devicesNamesDict[ itemID ]
                    _device = Devices[ _deviceUnit ]
                except Exception as ex:
                    Domoticz.Error("unable to FIND sensor Name = '%s': " % (itemID) + str(ex))
                    return
            else:
                return

            if "value" not in payload: return
            UpdateDevice( Dev=_device, nValue=int(payload['value']) , sValue=str(payload['value']) )


        elif kindID == "noise":

            # noise control: threshold
            # one for all noise sensors in same location
            itemID_ctrl = itemID_ctrlBase + "/threshold"
            if itemID_ctrl not in self._devicesNamesDict:
                _options_ctrl = { 'topic':topic+'/command', 'name':itemID_ctrl, 'dest':'all', 'order':'threshold' }
                _device = None
                _deviceUnit = self._availableUnits[-1]
                try:
                    _device = Domoticz.Device(Name=itemID_ctrl,  Unit=_deviceUnit, TypeName="Switch", Switchtype=7, Options=_options_ctrl, Description="name:"+itemID_ctrl, Used=1).Create()
                except Exception as ex:
                    Domoticz.Error("unable to CREATE sensor control Name = '%s': " % (itemID_ctrl) + str(ex))
                    return
                if _device is None: return
                # delete used unit number and add new device to reverse dict
                self._devicesNamesDict[ itemID_ctrl ] = self._availableUnits.pop(-1)

            # noise control: sensitivity
            # one for all noise sensors in same location
            itemID_ctrl = itemID_ctrlBase + "/sensitivity"
            if itemID_ctrl not in self._devicesNamesDict:
                _options_ctrl = { 'topic':topic+'/command', 'name':itemID_ctrl, 'dest':'all', 'order':'sensitivity' }
                _device = None
                _deviceUnit = self._availableUnits[-1]
                try:
                    _device = Domoticz.Device(Name=itemID_ctrl,  Unit=_deviceUnit, TypeName="Switch", Switchtype=7, Options=_options_ctrl, Description="name:"+itemID_ctrl, Used=1).Create()
                except Exception as ex:
                    Domoticz.Error("unable to CREATE sensor control Name = '%s': " % (itemID_ctrl) + str(ex))
                    return
                if _device is None: return
                # delete used unit number and add new device to reverse dict
                self._devicesNamesDict[ itemID_ctrl ] = self._availableUnits.pop(-1)

            # noise sensor
            if itemID not in self._devicesNamesDict and "value" in payload:

                _device = None
                _deviceUnit = self._availableUnits[0]   # TODO: create function that allocates an id
                try:
                    _device = Domoticz.Device(Name=itemID,  Unit=_deviceUnit, TypeName="Alert", Options=_options, Description="name:"+itemID, Used=1).Create()
                except Exception as ex:
                    Domoticz.Error("unable to CREATE sensor Name = '%s': " % (itemID) + str(ex))
                    return
                if _device is None: return
                # delete used unit number and add new device to reverse dict
                self._devicesNamesDict[ itemID ] = self._availableUnits.pop(0)

            elif itemID in self._devicesNamesDict:
                try:
                    _deviceUnit = self._devicesNamesDict[ itemID ]
                    _device = Devices[ _deviceUnit ]
                except Exception as ex:
                    Domoticz.Error("unable to FIND sensor Name = '%s': " % (itemID) + str(ex))
                    return
            else:
                return

            if "value" not in payload: return
            # a message is received whenever there is noise ...
            UpdateDevice( Dev=_device, nValue=4, sValue='[%d%s] NOISE NOISE!!' % (payload['value'],payload.get('value_units','')) )

            # [apr.18] we'll switch back to normal condition upon HeartBeat processing
            with self.heartBeatNoiseProcessingLock:
                if _deviceUnit not in self.heartBeatNoiseProcessing:
                    self.heartBeatNoiseProcessing.append( _deviceUnit )

    # publish a MQTT message over a topic
    def _publishMQTTmessage( self, topic, msg ):
        if not self.mqttConn.Connected():
            Domoticz.Debug("unable to publish while not Connected :|")
            return

        # publish message
        Domoticz.Debug( "MQTT publish '%s' <-- %s" % (topic,str(msg)) )
        return self.mqttConn.Send( {'Verb' : 'PUBLISH', 'QoS': 0, 'Topic': topic, 'Payload': json.dumps(msg)} )


    # process Temperature commands
    def _processTemperatureCommand( self, dev, topic, order, value ):
        if order!="frequency": return
            
        # Not yet implemented!
        return


    # process Luminosity commands
    def _processLuminosityCommand( self, dev, topic, order, value ):
        if order!="frequency": return
            
        # Not yet implemented!
        return


    # process Noise commands
    def _processNoiseCommand( self, dev, topic, order, value ):
        msg = dict()
        msg['dest'] = 'all'
        msg['order'] = order
        msg['value'] = value
        if order=="sensitivity":
            self._publishMQTTmessage( topic, msg )
            #UpdateDevice(Dev=dev, nValue=value, sValue="sensitivity=%d" % value)
            UpdateDevice( Dev=dev, nValue=2, sValue=str(value) )
        elif order=="threshold":
            _value = (value * __class__.NOISE_THRESHOLD_MAX)/100
            msg['value'] = _value
            self._publishMQTTmessage( topic, msg )
            # perequation over 100% --> NOISE_THRESHOLD_MAX
            #UpdateDevice(Dev=dev, nValue=value, sValue="threshold=%d" % _value )
            UpdateDevice( Dev=dev, nValue=2, sValue=str(value) )
        else:
            # Not yet implemented!
            Domoticz.Debug("Noise command order '%s' not yet implemented" % str(order) )
            return


'''
*** to update a device values:

def UpdateIcon(Unit, iconID):
    if Unit not in Devices: return
    d = Devices[Unit]
    if d.Image != iconID: d.Update(d.nValue, d.sValue, Image=iconID)

*** to process messages:
import msgpack
    def onMessage(self, Connection, Data):
        try:
            self.unpacker.feed(Data)
            for result in self.unpacker:

                Domoticz.Debug("Got: %s" % result)

                if 'exception' in result: return

                if result['cmd'] == 'status':

                    UpdateDevice(self.statusUnit,
                                 (1 if result['state_code'] in [5, 6, 11] else 0), # ON is Cleaning, Back to home, Spot cleaning
                                 self.states.get(result['state_code'], 'Undefined')
                                 )

                    UpdateDevice(self.batteryUnit, result['battery'], str(result['battery']), result['battery'],
                                 AlwaysUpdate=(self.heartBeatCnt % 100 == 0))

                    if Parameters['Mode5'] == 'dimmer':
                        UpdateDevice(self.fanDimmerUnit, 2, str(result['fan_level'])) # nValue=2 for show percentage, instead ON/OFF state
                    else:
                        level = {38: 10, 60: 20, 77: 30, 90: 40}.get(result['fan_level'], None)
                        if level: UpdateDevice(self.fanSelectorUnit, 1, str(level))

                elif result['cmd'] == 'consumable_status':

                    mainBrush = cPercent(result['main_brush'], 300)
                    sideBrush = cPercent(result['side_brush'], 200)
                    filter = cPercent(result['filter'], 150)
                    sensors = cPercent(result['sensor'], 30)

                    UpdateDevice(self.cMainBrushUnit, mainBrush, str(mainBrush), AlwaysUpdate=True)
                    UpdateDevice(self.cSideBrushUnit, sideBrush, str(sideBrush), AlwaysUpdate=True)
                    UpdateDevice(self.cFilterUnit, filter, str(filter), AlwaysUpdate=True)
                    UpdateDevice(self.cSensorsUnit, sensors, str(sensors), AlwaysUpdate=True)

        except msgpack.UnpackException as e:
            Domoticz.Error('Unpacker exception [%s]' % str(e))
'''

global _plugin
_plugin = BasePlugin()

def onStart():
    global _plugin
    _plugin.onStart()

def onStop():
    global _plugin
    _plugin.onStop()

def onConnect(Connection, Status, Description):
    global _plugin
    _plugin.onConnect(Connection, Status, Description)

def onMessage(Connection, Data):
    global _plugin
    _plugin.onMessage(Connection, Data)

def onCommand(Unit, Command, Level, Hue):
    global _plugin
    _plugin.onCommand(Unit, Command, Level, Hue)

def onNotification(Name, Subject, Text, Status, Priority, Sound, ImageFile):
    global _plugin
    _plugin.onNotification(Name, Subject, Text, Status, Priority, Sound, ImageFile)

def onDeviceModified(Unit):
    global _plugin
    _plugin.onDeviceModified(Unit)

def onDeviceRemoved(Unit):
    global _plugin
    _plugin.onDeviceRemoved(Unit)

def onDisconnect(Connection):
    global _plugin
    _plugin.onDisconnect(Connection)

def onHeartbeat():
    global _plugin
    _plugin.onHeartbeat()



#
# Helpers functions
#
#

# Update Device into DB
def UpdateDevice( Unit=None, Dev=None, nValue=None, sValue=None, Image=None):
    # Make sure that the Domoticz device still exists (they can be deleted) before updating it
    if (Unit is not None) and (Unit in Devices):
        Dev = Devices.get('Unit')
    if Dev is None:
        Domoticz.Error("Error while trying to update values, Dev is None ?!?!")
        return

    if nValue is None:
        nValue = Dev.nValue

    if sValue is None:
        sValue = Dev.sValue

    if Image is None:
        Image = Dev.Image

    Dev.Update(nValue=nValue, sValue=str(sValue), Image=Image)
    Domoticz.Log("Update "+str(nValue)+":'"+str(sValue)+"' ("+Dev.Name+")")
    return

def DumpConfigToLog():
    for x in Parameters:
        if Parameters[x] != "":
            Domoticz.Debug( "'" + x + "':'" + str(Parameters[x]) + "'")
    Domoticz.Debug("Device count: " + str(len(Devices)))
    for x in Devices:
        Domoticz.Debug("Device:           " + str(x) + " - " + str(Devices[x]))
        Domoticz.Debug("Device ID:       '" + str(Devices[x].ID) + "'")
        Domoticz.Debug("Device Name:     '" + Devices[x].Name + "'")
        Domoticz.Debug("Device nValue:    " + str(Devices[x].nValue))
        Domoticz.Debug("Device sValue:   '" + Devices[x].sValue + "'")
        Domoticz.Debug("Device LastLevel: " + str(Devices[x].LastLevel))
        Domoticz.Debug("Device Options: " + str(Devices[x].Options))
    return

def DumpDictionaryToLog(theDict, Depth="-"):
    if isinstance(theDict, dict):
        for x in theDict:
            if isinstance(theDict[x], dict):
                Domoticz.Log(Depth+"> Dict '"+x+"' ("+str(len(theDict[x]))+"):")
                DumpDictionaryToLog(theDict[x], Depth+"---")
            elif isinstance(theDict[x], list):
                Domoticz.Log(Depth+"> List '"+x+"' ("+str(len(theDict[x]))+"):")
                DumpListToLog(theDict[x], Depth+"---")
            elif isinstance(theDict[x], str):
                Domoticz.Log(Depth+">'" + x + "':'" + str(theDict[x]) + "'")
            else:
                Domoticz.Log(Depth+">'" + x + "': " + str(theDict[x]))

def DumpListToLog(theList, Depth="-"):
    if isinstance(theList, list):
        for x in theList:
            if isinstance(x, dict):
                Domoticz.Log(Depth+"> Dict ("+str(len(x))+"):")
                DumpDictionaryToLog(x, Depth+"---")
            elif isinstance(x, list):
                Domoticz.Log(Depth+"> List ("+str(len(theList))+"):")
                DumpListToLog(x, Depth+"---")
            elif isinstance(x, str):
                Domoticz.Log(Depth+">'" + x + "':'" + str(theList[x]) + "'")
            else:
                Domoticz.Log(Depth+">'" + str(x) + "': " + str(theList[x]))

casimir
Posts: 6
Joined: Wednesday 18 April 2018 15:37
Target OS: Linux
Domoticz version:
Contact:

Re: python plugin connection

Post by casimir » Wednesday 23 May 2018 13:22

@dnpww0

Hello, i noticed that whenever i create sensors through my plugin (see previous post), these sensors won't ever turn in red if data does not arrive during the specified amount of time (as in settings). On the other hand, it works perfectly for sensors that have been added via others ways ?!?!

Thanks
François

User avatar
Dnpwwo
Posts: 728
Joined: Sunday 23 March 2014 10:00
Target OS: Raspberry Pi
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: python plugin connection

Post by Dnpwwo » Wednesday 23 May 2018 14:11

@casimir,

I'm guessing by 'Settings' you are refering to the 'Data Timeout:' setting for the hardware? If so then that does not control devices showing red, that triggers a restart of the hardware (in your case plugin) if no data is seen for a period.

To make devices go 'red' in the GUI you need to use the 'TimedOut' parameter on the Device.Update. 0 is not timed out, other values show it is. I use something like:

Code: Select all

def UpdateDevice(Unit, nValue, sValue, TimedOut):
    # Make sure that the Domoticz device still exists (they can be deleted) before updating it 
    if (Unit in Devices):
        if (Devices[Unit].nValue != nValue) or (Devices[Unit].sValue != sValue) or (Devices[Unit].TimedOut != TimedOut):
            Devices[Unit].Update(nValue=nValue, sValue=str(sValue), TimedOut=TimedOut)
            Domoticz.Debug("Update "+str(nValue)+":'"+str(sValue)+"' ("+Devices[Unit].Name+")")
    return
BTW: I see your plugin has a Password field, more recent versions of Domoticz support 'password="true"' for <param> tags and will not show the password on screen.
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw

casimir
Posts: 6
Joined: Wednesday 18 April 2018 15:37
Target OS: Linux
Domoticz version:
Contact:

Re: python plugin connection

Post by casimir » Sunday 03 June 2018 8:43

@dnpwwo

Thanks, i now understand much more things: the timeout flag is just a way to set the red banner to the sensor and it is my plugin responsability to set this value according to a specific timeout ... is there a way (from my plugin) to access to the value of the Domoticz's data timeout (used to be 20mn or 60mn as default) ?

Thanks for the good job :)
François

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests