Python - Read-out of DDS238 kWh-meter and upload to Domoticz and PVOutput

From Domoticz
Jump to: navigation, search

Introduction

This wiki-section can be divided in three main parts:

  1. a domoticz configuration part where you will be creating the appropriate virtual sensors.
  2. a shell script and cronjob which control a python script that extracts information from the DDS238-meter and will be posting values into Domoticz and to PVOutput.
  3. a description of variations and their impact on the 2 earlier parts.

The interface-routines for DDS238 in part 2 are specific as well as part 3, but for part 1 and for main parts of part 2 I have gratefully borrowed useful elements from the Wiki for the Omnik Solar Inverter.

The main reasons I made this script:

  1. my setup with Involar inverters is so small that application of Involar's eGate would be a giant overkill, not even considering that the eGate does not connect to Domoticz, PVOutput etc., but only to the 'own' portal of Involar.
  2. still I want registration of reasonably quality as well as application of the data by Domoticz and by PVOutput.
  3. a script for reading a kWh-meter with RS485-interface does not exist as 'open source'

The description and script in this Wiki assumes a single kWh-meter type DDS238-1ZN with default settings:

  1. Slave-address = 01
  2. Baudrate = 01 = 9600 Baud

For any other configuration or setting, see section 3 of this Wiki.

My current setup

As perspective for this Wiki-description, below a short description of the situation at my home.

Solar Inverter & Server

Hardware Configuration

For a small subsection of my PV-system 2* Involar-inverter type MAC250 connect to 2* PV-panel type ESE215. The 2 inverters coupled in series feed a common 230V-grid-interface through a kWh-meter type DDS238-1ZN. This kWh-meter has 2 types of data-interface:

  1. S0-interface with 3200pulses/kWh, feeding a counter, of which after calculation the output is the accumulated energy (Wh), and (after backward-calculation) the actual power (W).
  2. RS485-interface provides directly measured data for LifeEnergy (Wh), Power (W), Voltage (V), Current (A), Frequency (Hz) and several other aspects. This interface applies the ModBus RTU protocol, reading by calls to 'Modbus' registers 03Hex and 04Hex, and by writing to 'Modbus' register 10Hex (=21Dec).

In my configuration, both interfaces are connected to a Raspberry acting as server running Domoticz and Python:

  1. the frontend for the S0-interface is an S0PulseCountingModule [as described elsewhere in this wiki]
  2. the frontend for the RS485-interface is an RS485-shield by Linksprite

Because of the direct measurement at the inverter-output and due to the data-catalogue, the RS485-interface is more versatile and superior to the S0-interface, and topic of this Wiki-description.

Software

For the RS485-interface of the DDS238-meter no ready interface exists as open-source material.

A Python-script pulls out the data from my DDS238-meter (applying the Python-module minimalmodbus), stores it in Domoticz using JSON, and uploads info to PVOutput.

For this script the front-end interfacing to the DDS238-meter's RS485-interface is 'own design', the idea to determine the time-gate (using pyephem) has been borrowed and converted to an adapted script, while the interface towards Domoticz and PVOutput has been borrowed & adapted from the Wiki-section for the Omnik_Solar_Inverter: ;-) re-inventing the wheel should be minimized ......

The proposed scripts are all assuming application for a DDS238-meter numbered 1 (straight out of the box): for meters with other number you have to adapt (at least) that number.

Before you can use the script, you have to adapt the settings for IDXes for your virtual sensors and for the API-key and SID applicable for upload to YOUR account at PV-Output.org

Prepare Domoticz

To create a virtual sensor, perform following actions.

  1. Go to "Hardware"
  2. Fill in the name field with a desired name (like "Virtual")
  3. Choose for type the "Dummy, does nothing, use for virtual switches only"
  4. Click on "Add"
  5. The added hardware is now added to the hardware list. In the same row there is a "Create virtual sensor" button: click it.

Now you are ready to create virtual devices.

  1. Go to "Hardware", and look for the section you created a moment ago
  2. Press "Create virtual sensor".
  3. Fill in the name-field with a desired name

To cover the most interesting elements provided at the RS485-interface of the DDS238-meter, we need ad least 3 virtual sensors:

# DDS238 Power / Energy (W / kWh)
# DDS238 AC Voltage (V)
# DDS238 AC Current (A)

Therefore repeat the above steps 3 times, after which:

  1. Go to "Devices"
  2. Click "Not-Used"
  3. Search for the 3 virtual devices created a moment ago (and click on the green arrow behind it).
  4. Give the sensors the following name:
 # "DDS238 Production1",  will contain 2 values for Current Power-production, respectively accumulated Energy in (Watt / kWh) for meter#1
 # "DDS238 Voltage1", will contain a value of inverter's output voltage (V) for meter#1
 # "DDS238 Current1", will contain a value of inverter's output current (A) for meter#1

Creating the Shell script used in cron

Now create for meter#1 a shell script DDS238_01.sh and put it in the "home/pi/domoticz/scripts/" folder.

#!/bin/bash
sudo python /home/pi/domoticz/scripts/DDS238Export01.py


Make this Shell-script executable: "sudo chmod +x /home/pi/domoticz/scripts/DDS238_01.sh"

Collect required Inputs for Settings

For the Python-script you require the following information (in the sequence in whicht they appear in the script).

For upload to PVOutput.org

  1. your api-key
  2. the destination SID

For setting of Ephem (to get data for sunrise and sunset)

  1. your home.lat(itude)
  2. your home.long(itude)

For upload to Domoticz

  1. the IDXes of the virtual devices

Create DDS238Export01.py

#!/usr/bin/python
# -*- coding = utf-8 to enable reading by simple editors -*-
# --------------------------------------------------
# Line004 = PREPARATION & SETTINGs
# --------------------------------------------------
 
# Imports for script-operation
 
import serial                   # required for handling of the serial port
import minimalmodbus            # required for handling modbus
import datetime                 # used for timestamps & timecheck
import time                     # used for timestamps & delays
 
# Line014 = Provide here the settings for access to PVOutput
 
slave = 1             # decimal slave-address of the DDS238 kWh-meter
retry = 5             # number of retries to get power-info from DDS238
delay = 2             # time (seconds) between retries
pvout_enabled   = True
pvout_apikey    = "<YOUR api-key>"
pvout_sysid     = <YOUR SID>
pvout_cumflag   = 1   # flag is 1 if you apply lifetime-energy as reported by DDS238
                      # flag must be 0 if you upload a calculated Wh_today
 
# --------------------------------------------------
# Line026 = READING THE RS485-INTERFACE OF DDS238
# --------------------------------------------------
# the "print"-lines only added for local read-out e.g. during script_tuning
 
now = datetime.datetime.now()
print
print 'Present Date & Time = ', now
print
instrument = minimalmodbus.Instrument('/dev/ttyAMA0',slave) # port name, slave address
instrument.serial.baudrate = 9600
instrument.serial.timeout = 0.5
# instrument.debug = True
 
TotalEnergy1 = instrument.read_registers(0,2)
LifeEnergy_High = TotalEnergy1[0]
LifeEnergy_Low = TotalEnergy1[1]
LifeEnergy = 0.01 * ((LifeEnergy_High * 65535) + LifeEnergy_Low)
LIFENERGY = LifeEnergy * 1000
TotalEnergy2 = instrument.read_registers(8,2)
RevEnergy_High = TotalEnergy2[0]
RevEnergy_Low = TotalEnergy2[1]
RevEnergy = 0.01 * ((RevEnergy_High * 65535) + RevEnergy_Low)
REVENERGY = RevEnergy * 1000
TotalEnergy3 = instrument.read_registers(10,2)
FwdEnergy_High = TotalEnergy3[0]
FwdEnergy_Low = TotalEnergy3[1]
FwdEnergy = 0.01 * ((FwdEnergy_High * 65535) + FwdEnergy_Low)
FWDENERGY = FwdEnergy * 1000
Voltage = instrument.read_register(12,1)
VOLTAGE = Voltage
Current = instrument.read_register(13,2)
CURRENT = Current
ActivPower = instrument.read_register(14,0)
if (LifeEnergy > 0) and (ActivPower >= 0):
   Missing = False
   i = 1
else:              # try to get at least power-info
   Missing = True
   i = 1
   time.sleep(delay)   # delay till conditional repeat
   while Missing and (i < retry):   
     Voltage = instrument.read_register(12,1)
     VOLTAGE = Voltage
     ActivPower = instrument.read_register(14,0)
     Missing = (ActivPower == '')
     time.sleep(delay) # delay till next cycle    
     i = i+1
 
if ActivPower < 65535/2:
   PRODUCTION = ActivPower
   CONSUMPTION = 0
else:
   ActivPower = ActivPower - 65534
   PRODUCTION = 0
   CONSUMPTION = - ActivPower
 
#ReactPower = instrument.read_register(15,0)
#PF = instrument.read_register(16,3)
#Freq = instrument.read_register(17,2)
#Status = instrument.read_register(21,0)
#Adres = Status/256
#print 'DDS238_Status, Address  = ', Adres
#Rate = Status%256
#print 'DDS238_Status, Baudrate = ', Rate
print
print 'Run ', i
print '= RS485-Values for Upload ='
print 'Life_Net_Energy  = ', LIFENERGY, ' Wh'
print 'Forward_Energy   = ', FWDENERGY, ' Wh'
print 'Reverse_Energy   = ', REVENERGY, ' Wh'
print 'Voltage       = ', VOLTAGE, ' V'
print 'Current       = ', CURRENT, ' A'
print 'ActivePower   = ', ActivPower, ' VA'
print '=> Production   = ', PRODUCTION, ' VA'
print '=> Consumption  = ', CONSUMPTION, ' VA'
 
ENERGY = LIFENERGY
POWER = PRODUCTION
# --------------------------------------------------
# END READING RS485-INTERFACE OF DDS238
# --------------------------------------------------
print
# ---------------------------------------------------
# EXTRACT SUNRISE & SUNSET
# ---------------------------------------------------
 
# Imports for script-operation
 
import ephem
#import datetime    # already imported in Part1
 
# Presetting of parameters
 
sun = ephem.Sun()
home = ephem.Observer()
home.lat, home.lon = '52.2962643', '6.8033612' #your lat long
# lat/long-info must be inserted as decimal degrees
sun.compute(home)
sunrise, sunset = (home.next_rising(sun),
                   home.next_setting(sun))
print '= Sunrise & Sunset expressed in UTC! ='
print 'Sunrise = ',sunrise
print 'Sunset  = ',sunset
a = sunrise.datetime()
b = sunset.datetime()
Starthour = a.hour
if a.minute > 30:
  Starthour = Starthour - 1  # shift 1 hour before sunrise-UTC
if Starthour < 5:
  Starthour = 5
Stophour = b.hour + 1        # shift 1 hour after sunset-UTC
if b.minute < 30:
  Stophour = Stophour + 1    # shift 1 more hour
if Stophour > 21:
  Stophour = 21
print 'Upload-window opens when start-hour =',Starthour
print 'Upload-window closes after stop-hour=',Stophour
# --------------------------------------------------
# END READING SUNRISE & SUNSET
# --------------------------------------------------
print
# --------------------------------------------------
# START OF UPLOAD TO DOMOTICZ
# --------------------------------------------------
# Imports for script-operation
 
#import json
import urllib
from datetime import date
 
# Presetting of IDX for YOUR virtual sensors
 
domoticz_P1 = "209"    # IDX for virtual P1-meter
domoticz_V = "211"     # IDX for virtual sensor for voltage
domoticz_PE = "202"    # IDX for virtual sensor for power & energy
domoticz_C = "210"     # IDX for virtual sensor for current
 
# Setting & Linking of parameters
 
weekday = date.isoweekday(now)
hightime = (now.hour > 6) and (now.hour < 23) and (weekday < 6)
print 'Weekday = ',weekday
# (If desired) further code to be inserted to include holidays in the tariff-switching and to store at switching time the energy-values from the previous period
USAGE1 = REVENERGY
USAGE2 = 0
RETURN1 = FWDENERGY
RETURN2 = 0
CONS = CONSUMPTION
PROD = PRODUCTION
CONS = CONSUMPTION
PROD = PRODUCTION
 
httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_P1) + "&nvalue=0&svalue=" + str(float(USAGE1)) + ";" + str(float(USAGE2)) + ";" + str(float(RETURN1)) + ";" + str(float(RETURN2)) + ";" + str(float(CONS)) + ";" + str(float(PROD)) )
print 'Uploaded P1'
 
if POWER >= 0:
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_V) + "&nvalue=0&svalue=" + str(float(VOLTAGE)) )
   print 'Uploaded P&E'
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_PE) + "&nvalue=0&svalue=" + str(float(POWER)) + ";" + str(float(ENERGY)) )
   print 'Uploaded V'
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_C) + "&nvalue=0&svalue=" + str(float(CURRENT)) )
   print 'Uploaded C'
else:
   print 'No Power = No Upload'
 
# --------------------------------------------------
# END OF UPLOAD TO DOMOTICZ
# --------------------------------------------------
 
# --------------------------------------------------
# START OF UPLOAD TO PVOUTPUT
# --------------------------------------------------
 
# Imports for script-operation
 
import subprocess
from time import strftime
#import time       # already imported in Part1
 
# Presetting of parameters
 
SYSTEMID = pvout_sysid
APIKEY = pvout_apikey
t_date = format(strftime('%Y%m%d'))
t_time = format(strftime('%H:%M'))
pv_volts=0.0
pv_power=0.0
Wh_life=ENERGY
Wh_today=0.0
 
# space for addition of a calculation for current_temp
 
uploadtime = (now.hour >= Starthour) and (now.hour <= Stophour)
if uploadtime:
   print 'inside upload-window'
else:
   print 'outside upload-window'
 
if pvout_enabled and uploadtime:
   print 'uploading'
   pv_volts=VOLTAGE
   pv_power=POWER
   Wh_life=ENERGY
   cum=pvout_cumflag
   cmd=('curl -d "d=%s" -d "t=%s" -d "v1=%s" -d "v2=%s" -d "v6=%s" -d "v10=%s" -d "c1=%s" -H \
   "X-Pvoutput-Apikey: %s" -H \
   "X-Pvoutput-SystemId: %s" \
   http://pvoutput.org/service/r2/addstatus.jsp'\
   %(t_date, t_time, Wh_life, pv_power, pv_volts, Wh_life, cum, \
   APIKEY, SYSTEMID))
   ret = subprocess.call(cmd, shell=True)
 
# space for addition of a calculation for Wh_today
 
#time.sleep(20)            # delay to create space till next upload
 
#if Wh_today>0:            # if you also apply a calculated Wh_today
#  cmd=('curl -d "data=%s,%s,,,,,,," -H \
#  "X-Pvoutput-Apikey: %s" -H \
#  "X-Pvoutput-SystemId: %s" http://pvoutput.org/service/r2/addoutput.jsp' \
#  %(t_date, Wh_today,\
#  APIKEY,SYSTEMID))
#  ret = subprocess.call(cmd, shell=True)
 
print
print '= Info for upload to PVOutput ='
print 't_date %s' %t_date
print 't_time %s' %t_time
print 'pv_volts %s' %pv_volts
print 'pv_power %s' %pv_power
print 'Wh_life %s' %Wh_life
# --------------------------------------------------
# END OF UPLOAD TO PVOUTPUT
# --------------------------------------------------

Put the above python script in the "home/pi/domoticz/scripts/python" folder.

Install the Python-modules minimalmodbus and pyephem, required to operate the above Python-script.

Create the Cronjob

Now all ingredients are available to read the data from the inverter and insert them into Domoticz.

  1. execute python DDS238Export01.py
  2. Domoticz Inverter devices should now be filled with data.
  3. execute "crontab -e"
  4. add the following line "*/5 * * * * /home/pi/domoticz/scripts/DDS238_01.sh 2>&1 >> /dev/null"
  5. exit crontab. Now everything should be working for you.

More than 1 DDS238-meter or other kWh-meters in the configuration

Multiple DDS238-meters

If you want to apply more than 1 DDS238 in your configuration, please be aware of the following aspect:

  1. The DDS238-1ZN meter is delivered with default Slave address 01 for it's RS485-interface.

Change of that address is by write-command over the RS485-interface to the meter at Slave address 01.

  1. Therefore initially only one meter with that address 01 should be present at the RS485-network.
  2. Before fitting an additional meter to the RS485-network, therefore you must shift the meter(s) already connected to another address.

As a consequence, if you want to have more than 1 DDS238-1ZN meter at your RS485-network (direct out of the box), then

1) additional software must first be added & run which - for the first device at Slave-address 01 shifts the adress towards another free Slave-address to make space for the second device.

2) After that action you can connect the next DDS238-meter to the RS485-network, and run the same software another time to make space at Slave-address 01 for the third device, etc.

3) Repeat this sequence until you have fitted all DDS238-meters to your network

4) To get the information to Domoticz and to PVOutput for each additional DDS238-meter X, you now must replicate and must 'tune' the whole sequence of 'Prepare Domoticz', 'Create Shell-script', 'Create DDS238Export0X.py' and 'create the Cronjob', as described in the previous sections!

< the required DDS238-scripts for (address-check + address-shift) will be added here after cosmetic clean-up and further testing >

Other types of kWh-meters with RS485-interface

The DDS238-meter applies a specific protocol at it's RS485-interface, usually under the Modbus-protocol.

Although calling the Modbus' registers 03H, 04H and 10H seems a common feature for such kWh-meters, other types of meters most probably will have a different protocol-layout, with other default network-address, other register-addresses, different functional calls and different output information.

Therefore for such 'other' meter with different protocol a specific 'own' interface-routine must be developed, replacing the interface-routine of this Python script.

Obviously also a dedicated shell-script and dedicated cronjob must be made, and probably for multiple meters you have to replicate this effort.

Development status

Especially for the addition of extra DDS238-devices to an RS485-network, more work has to be done for robust software.

History and latest developments can be seen in this thread: http://www.domoticz.com/forum/viewtopic.php?f=14&t=5808