Weather Underground Upload script

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script [AWEKAS HTX-Upload]

Post by Toulon7559 » Wednesday 19 July 2017 18:01

For easy upload of meteo-information AWEKAS advices to apply Weatherlink-layout--> Variation 4
Which means:
- generate a html-file in specific layout
- put that file at location accessible by AWEKAS
- AWEKAS will grab it for further processing

The software setup described in this message performs the job, but as usual 'it is best effort DIY, as-is, for personal use / own risc, to be adapted for YOUR configuration'.

The lua-script below realises the following functions:
- execute the script at the rate set as set for interval (now 15 minutes)
- take the data as inputs from Domoticz as defined [you have to adapt that definitions for your own configuration]
Probably most of you will not have the 2 measurements of Light by ESP8266s. Take out or de-activate/comment the applicable input-definitions if not needed, and accordingly adapt the applicable lines further down in the script
- application of metric units. For that case the script only applies the conversion-function for luxtoWattsm2.
If you want to apply imperial units, then you have to change the unit-definitions, you need all conversion-functions, and you have to exchange the related formatting script lines for those applying the functions incl. call to the conversion-functions
- application of local time instead of UTC
- for barometer and rain facilities to insert an offset relative to the available input data
- conversion the data in the formats required for the html-file
- generation of the html-file by filling each line of the html-file, either by dynamic info prepared by the proceeding formatting script-lines, or by 'fixed' text.
- at the end of the script change the permissions of the html-file to allow transfer & application
- put the html-file in folder /home/pi/

Code: Select all

-- AWEKAS PWS upload script version 1c for Domoticz@Raspberry3 with RFLinkGateway
-- (C)2013 GizMoCuz for WeatherUnderground_Upload, adapted by Toulon7559 for periodic upload to AWEKAS of WS7000-info received via RFLink, extended by SolarRadiation info from calculation, Si1145 and TSL2561
-- This script produces a html-file in accordance with upload-format for AWEKAS Weatherlink HTX-template

commandArray = {}

-- Line 07, Define inputs from Domoticz
Outside_Temp_Hum = 'WS7000A_Temp_RV_Hoog'
Outside_Temp_Soil = 'WS7000B_Temp_RV_Bodem'
Barometer = 'WS7000_Temp_RV_Baro'
RainMeter = 'WS7000_Neerslag'
WindMeter = 'WS7000_Wind'
UVMeter = 'UVN800'
LuxMeter = 'Lux_Calc' -- calculated Lux-value (placeholder/fallback by the script from jmleglise)
SolarRad1 = 'ESP8266C_Licht1' -- measured lux-value1 (experimental)
SolarRad2 = 'ESP8266C_Licht2' -- measured lux-value2 (experimental)

-- Line 18, Define Settings & Units & Offsets
-- If Alternative units are used, then align the readouts by activation of the related dashed lines!
Interval = 15       -- Interval foor execution of script
Line10text ="Unknown" -- 5 Legal texts "Rising Rapidly" "Rising Slowly" "Steady" "Falling Slowly" "Falling Rapidly"
-- For Line10text any other string than the 5 legal texts is illegal and not shown
Line23text ="°C"     -- Alternative is "°F"
Line24text ="%"
Line25text ="m/s"   -- Alternative is "mph"
Line26text ="hPa"   -- Alternative is "in"
Line27text ="mm"    -- Alternative is "in"
Line28text ="W/m²"
Line29text ="mm/hr" -- Alternative is "in/hr"
Line30text ="index"
-- Setting of an offset is only meaningful after extended comparison of the sensor information against calibrated information such as from an official meteorological station
a = 1 -- relative correction for rain_day, default is 1, range from 0.9 to 1.1 for max. 10% offset
b = 1 -- relative correction for rain_lasthour, default is 1, range from 0.9 to 1.1 for max. 10% offset
-- absolute correction for rain_day and for rain_lasthour is 0, because otherwise strange bias each day at 00:00
c = 1 -- relative correction for air pressure in hPa, default is 1, range from 0.9 to 1.1 for max. 10% offset
d = 0 -- absolute correction for air pressure in hPa, default is 0, practical range from -10 to +10

-- Line 38, Define Functions
local function CelciusToFarenheit(C)
   return (C * (9/5)) + 32
end

local function hPatoInches(hpa)
   return hpa * 0.0295301
end

local function mmtoInches(mm)
   return mm * 0.039370
end

local function luxtoWattsm2(Lx)
--   if Lx == nil then return 0 end
   return Lx * 0.0079
end

-- Line 56= Get Current date & time from os as date.year, date.month, date.day, date.hour, date.min, date.sec
date = os.date("*t")
year = date.year
month = date.month
if month < 10 then month = "0" .. month
end
day = date.day
if day < 10 then day = "0" .. day
end
hour = date.hour
hourvalue = hour
if hour < 10 then hour = "0" .. hour
end
minutes = date.min
if minutes < 10 then minutes = "0" .. minutes
end
seconds = date.sec
if seconds < 10 then seconds = "0" .. seconds
end

-- Line 76, Read-out & Compilation of Lux-value
-- Data Extraction1 from svalues for Solarradiation-info
sLux = tonumber(otherdevices_svalues[LuxMeter])
--sLux = 1000 -- Placeholder for testing
LuxLvl = tonumber(sLux);
print ('Lux-level   = '.. LuxLvl)
-- Line 82, Experimental readout & processing of measured Lux-values 
-- Data Extraction2 from svalues for Solarradiation-info
sLux1 = tonumber(otherdevices_svalues[SolarRad1])
-- sLux1 = 500 -- Placeholder for testing
sLux2 = tonumber(otherdevices_svalues[SolarRad2])
-- sLux2 = 1000 -- Placeholder for testing
LuxDif = sLux1 - sLux2
if sLux1 > sLux2 then
  LuxMax = tonumber(sLux1)
else
  LuxMax = tonumber(sLux2)
end
print ('Lux1        = '.. sLux1)
print ('Lux2        = '.. sLux2)
print ('Lux_Max     = '.. LuxMax)
print ('Lux1-Lux2   = '.. LuxDif)

-- Line 99, Readout Temperature & Humidity
if Outside_Temp_Hum ~= '' then
   Line02text= string.format("%3.1f", (otherdevices_temperature[Outside_Temp_Hum]))
-- Line02text= string.format("%3.1f", CelciusToFarenheit(otherdevices_temperature[Outside_Temp_Hum]))
   Line03text= string.format("%.0f", (otherdevices_humidity[Outside_Temp_Hum]))
end

-- Line 106, Readout Barometer
if Barometer ~= '' then
    Line04text = string.format("%2.2f", ((otherdevices_barometer[Barometer])*c + d))
--  Line04text = string.format("%2.2f", hPatoInches((otherdevices_barometer[Barometer])*c + d))
end

-- Line 112, Readout of Rainvalues
if RainMeter ~= '' then
   Line05text= string.format("%2.2f", ((otherdevices_rain[RainMeter])*a + b))
-- Line05text= string.format("%2.2f", mmtoInches((otherdevices_rain[RainMeter])*a + 0)
   Line21text= string.format("%2.2f", ((otherdevices_rain_lasthour[RainMeter])*b + 0))
-- Line21text= string.format("%2.2f", mmtoInches((otherdevices_rain_lasthour[RainMeter])*b + 0)
end

-- Line 123, Readout wind direction & speed
if WindMeter ~= '' then
   Line06text= string.format("%.0f", (otherdevices_windspeed[WindMeter]))
-- Line06text= string.format("%.0f", (otherdevices_windspeed[WindMeter]/0.1)*0.223693629205)
   Line07text= string.format("%.0f", (otherdevices_winddir[WindMeter]))
-- Line07text= string.format("%.0f", (otherdevices_winddir[WindMeter]))
   Line18text= string.format("%.0f", (otherdevices_windgust[WindMeter]))
-- Line18text= string.format("%.0f", (otherdevices_windgust[WindMeter]/0.1)*0.223693629205)
end

-- Line 133, Readout Light, calculated
if LuxMeter ~= '' then
   Line19text= string.format("%.0f", luxtoWattsm2(LuxLvl))
end
-- Line 137, Readout SolarRad1 = Light, measured
-- if (SolarRad1 ~= '') OR (SolarRad2 ~= '') then
--   Line19text=  string.format("%.1f", luxtoWattsm2(LuxMax))
-- end

-- Line 142, Readout UVI
if UVMeter ~= '' then
    Line20text= string.format("%.1f", (otherdevices_uv[UVMeter]))
 end

-- Line 147, Readout Soil Temperature
if Outside_Temp_Soil ~= '' then
   Line22text= string.format("%3.1f", (otherdevices_temperature[Outside_Temp_Soil]))
-- Line22text= string.format("%3.1f", CelciusToFarenheit(otherdevices_temperature[Outside_Temp_Soil]))
end

-- Line 153, Check interval-time and make htm-file
date = os.date("*t")
if (date.min % Interval == 0) then
   file = io.open("/home/pi/awekas_wl.html", "w+")
   -- Opens a file named awekas_wl.html (stored under the designated sub-folder of Domoticz)
   -- in append mode
   -- write 31 lines to opened file
    file:write("AWEKAS_Template_start<br>\n")
    file:write(Line02text .. "<br>\n")
    file:write(Line03text .. "<br>\n")
    file:write(Line04text .. "<br>\n")
    file:write(Line05text .. "<br>\n")
    file:write(Line06text .. "<br>\n")
    file:write(Line07text .. "<br>\n")
    file:write(hour .. ":" .. minutes .. "<br>\n")
    file:write(day .. "." .. month .. "." .. year .. "<br>\n")
    file:write(Line10text .. "<br>\n")
    file:write("-<br>\n")
    file:write("-<br>\n")
    file:write("-<br>\n")
    file:write("-<br>\n")
    file:write("-<br>\n")
    file:write("-<br>\n")
    file:write("-<br>\n")
    file:write(Line18text .. "<br>\n")
    file:write(Line19text .. "<br>\n")
-- If no Light, then replace above line by file:write("-<br>\n")
    file:write(Line20text .. "<br>\n")
-- If no UV, then replace above line by file:write("-<br>\n")
    file:write(Line21text .. "<br>\n")
    file:write(Line22text .. "<br>\n")
-- If no soiltemp, then replace above line by file:write("-<br>\n")
    file:write(Line23text .. "<br>\n")
    file:write(Line24text .. "<br>\n")
    file:write(Line25text .. "<br>\n")
    file:write(Line26text .. "<br>\n")
    file:write(Line27text .. "<br>\n")
    file:write(Line28text .. "<br>\n")
    file:write(Line29text .. "<br>\n")
    file:write(Line30text .. "<br>\n")
    file:write("Template_V1.5<br>\n")
    file:close() -- closes the open file
   end
-- next line is required to set permissions for open&copy&shift of the file 
os.execute("chmod a+rw /home/pi/awekas_wl.html")

return commandArray
AWEKAS requires that the html-file is put at a location where AWEKAS can grab/download the file.
For upload to a remote server unfortunately no lua-scriptlines known to me, but a short python-script, to be periodically run as part of a cronjob.

Code: Select all

#!/usr/bin/python
# -*- coding = utf-8 to enable reading by simple editors -*-
# (c)2017 script compiled by Toulon7559 from various material from forums, version 0.2 for upload of awekas_wl.html
# --------------------------------------------------
# Line005 = PREPARATION & SETTING
# --------------------------------------------------
# Assure that ftplib has been installed, and make this script part of cronjob for periodic execution
# This script assumes the html-file to be saved to folder /home/pi/
# This script uploads the html-file to the root-folder of the remote ftp-server
# Lines 32 and 33 to be filled with YOUR applicable account-info for the ftp-account
# Line 34, 35, 37 and 38 inserted for change and reset of working directory for upload. Line-numbers and Print-lines are inserted for testing.
print
print ('Start of script')

# --------------------------------------------------
# Line016 = Function for FTP_UPLOAD to Server
# --------------------------------------------------
# Imports for script-operation
import ftplib
import os
# Definition of Upload_function
def upload(ftp, file):
    ext = os.path.splitext(file)[1]
    if ext in (".txt", ".htm", ".html"):
        ftp.storlines("STOR " + file, open(file))
    else:
        ftp.storbinary("STOR " + file, open(file, "rb"), 1024)

# --------------------------------------------------
# Line030 = Actual FTP-Login & -Upload
# --------------------------------------------------
ftp = ftplib.FTP("<ftp_servername>")
ftp.login("<ftp_username>", "<ftp_password>")
# ftp.cwd('AWEKAS') # scriptline to select remote directory AWEKAS
# ftp.pwd() # return/check actual path
upload(ftp, "awekas_wl.html")
# ftp.cwd('/') # scriptline to reset to remote root-directory
# ftp.pwd() # return/check actual path
print ('End of script')
Last edited by Toulon7559 on Wednesday 19 September 2018 9:36, edited 3 times in total.
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Thursday 27 July 2017 15:06

In the previous message one scriptline reads

Code: Select all

Line10text ="Unknown" -- 5 Legal texts "Rising Rapidly" "Rising Slowly" "Steady" "Falling Slowly" "Falling Rapidly"
In my opinion the required indication can be generated by comparison of the actual barometer-reading against a barometer-reading which according to general definitions is 3 hours earlier. Sometimes the value at 1 hour is read.
Looking around I found that
- Steady = variation of < 0.5 hPa
- Slowly = variation of 0.5 till 3 hPa
- Rapidly = variation of 3 till 6 hPa

For the required comparison a data table is needed, which contains a list of values earlier than the actual reading.
That list has to be periodically updated.

A 'clumsy, but probably effective' solution would be to make a kind of shiftregister of uservariables, and e.g. every 15minutes shift the values, inserting the latest/actual value at the start of the shift-register.
But probably a 'more clever' method exists: anybody with such knowledge, example or hint?
Last edited by Toulon7559 on Tuesday 22 August 2017 23:09, edited 2 times in total.
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Thursday 27 July 2017 15:44

The AWEKAS_push_API-protocol has some ;-) 'challenging' aspects, as extracted below.
Not needed for a 'basic setup', but nice-to-have for a 'luxury edition'

Code: Select all

Pos11: weather condition (see Table below) (int)
Pos12: warning text ( if you have selected weather warning in weather condition - this warning text will displayed) (int)
Pos13: snow high (in cm) (int) if no snow leave blank
For the technically-minded meteo-amateur Pos13 invites to make a device to measure depth of snow layer, coupled to Domoticz.
I remember to have seen something in the past, but lost the link: any hints?
Because in the Netherlands we hardly have any snowfall of significance, the simpler solution is to make a manual insertion for this information.
Pos11 and Pos12 are mutually linked and they certainly require a rather elaborate, manual selection function.
Considering the list below (as extracted from the protocol-description) a nice challenge for ManMachineInterface for Domoticz!
Anybody with a solution how to make an 'easy' MMI, which provides an output in/to Domoticz for Pos11, Pos12 and Pos13?

Code: Select all

Weather conditions: (if available --> only set by human / not calculated)

1=clear 
2=sunny sky 
3=partly cloudy 
4=cloudy 
5=heavy cloundy 
6=overcast sky 
7=fog 
8=rain showers 
9=heavy rain showers 
10=light rain 
11=rain 
12=heavy rain 
13=light snow 
14=snow 
15=light snow showers 
16=snow showers 
17=sleet 
18=hail 
19=thunderstorm 
20=storm 
21=freezing rain 
22=warning 
23=drizzle 
24=heavy snow 
25=heavy snow showers 
0=clear warning

warning conditions --> (integer value for Pos.12 - only active if Pos.11 (weather condition) = 22 (warning))
1= storm
2= heavy rain
3= heavy fog
4= extreme heat >35°C
5= extreme cold <-20°C
6= glaze
7= heavy snowfall
8= snowdrift
9= squalls
10= hurricane/ tornado
11= constant rain
12= flooding
13= sleet
14= hail
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Friday 28 July 2017 18:35

;-) Digging deeper and deeper in the handling of sensor-data, you stumble on unexpected aspects.

For handling of data in quite a lot of scripts you see scriptlines like

Code: Select all

if Outside_Temp_Hum ~= '' then
That is a check that a value is available.
However the check does not look whether the data is meaningful, and it does not tell you how 'fresh' or how 'old' the data is.
The widgets, Setup/Devices and Setup/More Options/Event/Current States show the time of latest refresh.
And in the datatables attached to the logs you can see the successive registrations with a time-stamp.
The extract below from such datatable shows that between approx. 00:40 and 06:10 no update took place, out of sequence related to the 'normal' refresh every 5 minutes.
Knipsel_Tempdatatable.JPG
Registration_table
Knipsel_Tempdatatable.JPG (20.61 KiB) Viewed 2579 times
To check that a registration is continuous, you need a time-out check, which detects that new data is 'intime' or 'overdue'.
For that the validity-time or time-stamp for the data is needed.
Is a script-call available which extracts the validity-time from the table with svalues?
Or better ;-), anybody knowing/having an example script for a time-outcheck in Domoticz' lua-script?
Last edited by Toulon7559 on Tuesday 15 August 2017 15:17, edited 1 time in total.
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Wednesday 02 August 2017 9:41

Question has been asked before, but a recent crash & reinstall of my second Domoticz-Raspberry give motivation to ask again, because of problems with reinstall of the crontab.

A lua-script generates the data which needs to be periodically uploaded to a remote server.
For that upload I apply a Python-script.
Periodically running Python-scripts (etc.) calls for a cronjob & crontab.
:-( Crontab is not the easiest software to debug .......

Would be nicer to have the uploadfunction as part of the lua-script.
Not only because then integrated with the data-source (and therefore 'in view' in one package), but then also with own aligned schedule etc., independent of the cronjob/crontab.
Somebody having/knowing a working example of applicable lua-scriptlines?


Or as alternative:
is there an easy&robust way to backup & restore a crontab?
Last edited by Toulon7559 on Wednesday 02 August 2017 20:42, edited 2 times in total.
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

R0yk3
Posts: 46
Joined: Sunday 24 July 2016 21:51
Target OS: Raspberry Pi
Domoticz version: beta
Location: the Netherlands
Contact:

Re: Weather Underground Upload script

Post by R0yk3 » Wednesday 02 August 2017 9:49

backing up crontab is easy.
crontab -e
select all copy and paste it into notepad or any text editor, save it localy.
after crash restore scripts
copy paste the textfile back into crontab

even more easy is regulary make a image of your sd card.
:!: :!: but i think this is a bit off topic!! :!: :!:
Raspberry PI 3, raspbian, ZwaveMe, RFLink

Haringstad
Posts: 27
Joined: Tuesday 22 November 2016 10:54
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by Haringstad » Wednesday 02 August 2017 10:36

Man, wouldn't it be great if the original script would be put into a dzVents version????

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Wednesday 02 August 2017 20:41

@R0yk3
Did it your way, but :-( as described earlier, some problems to get the crontab running again, which controls the python-upload script.

As alternative, development &test of a lua-script for upload proves to be more difficult = time-consuming than expected:
too many examples, too many adaptations required, and unclear error-reports during testing, makes progess slow .........
Therefore my question in previous message stays valid.

Pragmatic solution to keep the uploads running:
- installed vsftpd on Raspberry-with-Domoticz as a starter for ftp-upload
- the sync-software at my 'central' server (which runs anyway for other jobs of upload&file-alignment&backup) reads the file from the Raspberry and periodically uploads to the remote server as backup for the Python-uploadscript

Certainly not a beauty and not extremely reliable, because the upload&sync-chain depends on yet another computer, but it usually works, until a better, fully integrated, simple&robust solution becomes operational .........
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

R0yk3
Posts: 46
Joined: Sunday 24 July 2016 21:51
Target OS: Raspberry Pi
Domoticz version: beta
Location: the Netherlands
Contact:

Re: Weather Underground Upload script

Post by R0yk3 » Thursday 21 September 2017 10:09

I have a strange error,

sometimes the script stops uploading ad i get this error:
Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_weatherunderground.lua: .../domoticz/scripts/lua/script_time_weatherunderground.lua:29: attempt to perform arithmetic on local 'mm' (a nil value)

If i do nothing, most of the times it starts uploading data again?
The stop is at midnight. Than the error starts.
My script is not altered in any way, the sensor just provides data?

See the itches
https://www.wunderground.com/personal-w ... 920/mmonth

What do i miss here??? :?:
Raspberry PI 3, raspbian, ZwaveMe, RFLink

R0yk3
Posts: 46
Joined: Sunday 24 July 2016 21:51
Target OS: Raspberry Pi
Domoticz version: beta
Location: the Netherlands
Contact:

Re: Weather Underground Upload script

Post by R0yk3 » Friday 22 September 2017 12:35

So nobody??
Nothing changed, i am at work and it came back online?
https://www.wunderground.com/personal-w ... IROSMALE19
Not really a steady connection.
Raspberry PI 3, raspbian, ZwaveMe, RFLink

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Tuesday 27 February 2018 2:29

westd001 wrote:
Friday 11 September 2015 17:22
Toulon7559,

The timer value is OK, though, I deleted my script and copied yours (with the BMP180 and 55 min. timer), changed the timer to 2 minutes and that works !! I changed the code a bit because it gives an error every minute because the URL is null. Don't know the difference between the two versions but it works. See below, thanks for your reply !

Code: Select all

-- Weatherunderground PWS upload script
-- (C)2013 GizMoCuz, adapted by Toulon7559 for periodic upload of info received via RFXCom, supplemented by pressure-info from a BMP180 sensor

date = os.date("*t")
if (date.min % 55 == 0) then
--start script every 55 minutes

-- Line 05, Defintion of inputs
Outside_Temp_Hum = 'Temperatuur buiten'
Barometer = 'Barometer'
RainMeter = 'Regenmeter'
WindMeter = 'Windmeter'
-- UVMeter = ' '

-- Line 12, WU Settings
baseurl = "http://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?"
ID = <ID>
PASSWORD = <password>

-- Line 18, Local functions
local function CelciusToFarenheit(C)
   return (C * (9/5)) + 32
end

local function hPatoInches(hpa)
   return hpa * 0.0295301
end

local function mmtoInches(mm)
   return mm * 0.039370
end

-- Line 31, Extraction of required calendar info
utc_dtime = os.date("!%m-%d-%y %H:%M:%S",os.time())
month = string.sub(utc_dtime, 1, 2)
day = string.sub(utc_dtime, 4, 5)
year = "20" .. string.sub(utc_dtime, 7, 8)
hour = string.sub(utc_dtime, 10, 11)
minutes = string.sub(utc_dtime, 13, 14)
seconds = string.sub(utc_dtime, 16, 17)

timestring = year .. "-" .. month .. "-" .. day .. "+" .. hour .. "%3A" .. minutes .. "%3A" .. seconds

SoftwareType="Domoticz"

-- Line 44, Current date as date.year, date.month, date.day, date.hour, date.min, date.sec
WU_URL= baseurl .. "ID=" .. ID .. "&PASSWORD=" .. PASSWORD .. "&dateutc=" .. timestring
--&winddir=230
--&windspeedmph=12
--&windgustmph=12

-- Line 53, Extraction of data from BMP180
sBMP180T, sBMP180B = otherdevices_svalues[Barometer]:match("([^;]+);([^;]+)")
-- sBMP180T = tonumber(sBMP180T);
-- print ('BMP180_temp = '.. sBMP180T) 
sBMP180B = tonumber(sBMP180B);
-- print ('BMP180_druk = '.. sBMP180B)

if Outside_Temp_Hum ~= '' then
   WU_URL = WU_URL .. "&tempf=" .. string.format("%3.1f", CelciusToFarenheit(otherdevices_temperature[Outside_Temp_Hum]))
   WU_URL = WU_URL .. "&humidity=" .. otherdevices_humidity[Outside_Temp_Hum]
   WU_URL = WU_URL .. "&dewptf=" .. string.format("%3.1f", CelciusToFarenheit(otherdevices_dewpoint[Outside_Temp_Hum]))
end

if Barometer ~= '' then
   WU_URL = WU_URL .. "&baromin=" .. string.format("%2.2f", hPatoInches(sBMP180B))
end

if RainMeter ~= '' then
   WU_URL = WU_URL .. "&dailyrainin=" .. string.format("%2.2f", mmtoInches(otherdevices_rain[RainMeter]))
   WU_URL = WU_URL .. "&rainin=" .. string.format("%2.2f", mmtoInches(otherdevices_rain_lasthour[RainMeter]))
end

if WindMeter ~= '' then
   WU_URL = WU_URL .. "&winddir=" .. string.format("%.0f", otherdevices_winddir[WindMeter])
   WU_URL = WU_URL .. "&windspeedmph=" .. string.format("%.0f", (otherdevices_windspeed[WindMeter]/0.1)*0.223693629205)
   WU_URL = WU_URL .. "&windgustmph=" .. string.format("%.0f", (otherdevices_windgust[WindMeter]/0.1)*0.223693629205)
end

-- if UVMeter ~= '' then
--    WU_URL = WU_URL .. "&UV=" .. string.format("%.1f", mmtoInches(otherdevices_uv[UVMeter]))
-- end

--&weather=
--&clouds=

WU_URL = WU_URL .. "&softwaretype=" .. SoftwareType .. "&action=updateraw"

--print (WU_URL)

commandArray = {}

--remove -- before the line below to actually upload
commandArray['OpenURL']=WU_URL

return commandArray

-- end of script

else
  commandArray = {}
  return commandArray
end
Hello everyone,

I'm starting with Domoticz and so far I love the experience.
I'm having some problems with the pressure in the WU, I use a netammo and the values appear to me as 65 instead of 1006.8 hPa does anyone know why I say?
Otherwise everything seems to be working fine with this script.

Thanks for sharing!

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Tuesday 27 February 2018 15:26

Looking at the question in the last section (having 1006.8 hPa and seeing 65), the only answer coming close (because I recognize digits in a value) is the conversion of 1006.8 hPa into 6.62353696696941 kg/in^2, but that is still off by a multiplier factor of 10.
Raises the question: do you have the unit-settings correct for the pressure at the Netatmo?

On the other hand, it is non-standard to apply kg/in^2 as unit for meteo pressure-reading.
The more usual alternative for hPa is the 'Imperial' unit inch mercury(Hg) for which 1006.8 hPa would convert to 29.7307840298324 in
Scriptline 24 is performing that conversion, because WUnderground is expecting 'imperial units'.
If you activate line 15 from the bottom of the script, then you can see in Domotiz' log which pressure-output (expressed in inch(Hg) ) the script sends to WUnderground:
now

Code: Select all

--print (WU_URL)
for active printing in Domoticz' log make it

Code: Select all

print (WU_URL)
Hint: If you want to check on conversion-factors, then Steamwire Quicktool is handy.
Last edited by Toulon7559 on Thursday 01 March 2018 0:09, edited 1 time in total.
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Tuesday 27 February 2018 16:14

Toulon7559 wrote:
Tuesday 27 February 2018 15:26
Looking at the question in the last section (having 1006.8 hPa and seeing 65), the only answer coming close (because I recognize digits in a value) is the conversion of 1006.8 hPa into 6.62353696696941 kg/in^2, but that is still off by a multiplier factor of 10.
Raises the question: do you have the unit-settings correct for the pressure at the Netammo?

On the other hand, it is non-standard to apply kg/in^2 as unit for meteo pressure-reading.
The more usual alternative for hPa is the 'Imperial' unit inch mercury(Hg) for which 1006.8 hPa would convert to 29.7307840298324 in
Scriptline 24 is performing that conversion, because WUnderground is expecting 'imperial units'.
If you activate line 15 from the bottom of the script, then you can see in Domotiz' log which pressure-output (expressed in inch(Hg) ) the script sends to WUnderground:
now

Code: Select all

--print (WU_URL)
for active printing in Domoticz' log make it

Code: Select all

print (WU_URL)
Hint: If you want to check on conversion-factors, then Steamwire Quicktool is handy.
Thanks for your help. In the panel of the netatmo is like mb, this implies something?
At the moment it stopped showing graphs for atmospheric pressure, but when I tested it the first time, it gave these weird values, in the log appeared the baromin register if I am not mistaken as 1.92, I do not know because later it turns 65 hpa.

I basically wanted the pressure to appear in hPa, when I get home I'll test what it suggested.
The station is this:https://www.wunderground.com/personal-w ... 12#history

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Tuesday 27 February 2018 17:27

To be clear:
the display at WUnderground is something different than the upload to WUnderground!!!

The upload to WUnderground must be in 'imperial' units, and if your PWS is feeding 'metric information' that is the reason for the conversion function starting at line 18 in the script.

At your WU-webpage you have a choice how to display.
Go to the small 'wheel' in the upper right corner in the black bar of the webpage.
If clicked, you get the choice F or C = Fahrenheit respectively Celsius = 'imperial units' respectively 'metric units'
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Tuesday 27 February 2018 19:12

Toulon7559 wrote:
Tuesday 27 February 2018 17:27
To be clear:
the display at WUnderground is something different than the upload to WUnderground!!!

The upload to WUnderground must be in 'imperial' units, and if your PWS is feeding 'metric information' that is the reason for the conversion function starting at line 18 in the script.

At your WU-webpage you have a choice how to display.
Go to the small 'wheel' in the upper right corner in the black bar of the webpage.
If clicked, you get the choice F or C = Fahrenheit respectively Celsius = 'imperial units' respectively 'metric units'
Yes I know this, except that the first time I tested the atmospheric pressure it appeared with wrong values and now it stopped appearing without touching the script. Since all values were presented well, the atmospheric pressure should not appear correct as well?

What happened when I tested the first time was to display erroneous values on the pressure graph and appear - hpa in the station information. The graph does not now appear and continues to appear - hPa
Later, when you get home, I'll try this better!

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Wednesday 28 February 2018 0:48

From time to reappear again the pressure chart, but it appears as 74.2 hPa
appearing in the station information --- hPa I do not understand

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Wednesday 28 February 2018 15:36

Here's what happens:
Image

Toulon7559
Posts: 465
Joined: Sunday 23 February 2014 18:56
Target OS: Raspberry Pi
Domoticz version: latest
Location: Hengelo(Ov)/NL
Contact:

Re: Weather Underground Upload script

Post by Toulon7559 » Thursday 01 March 2018 0:18

If the WUnderground-page shows -- hPa, it means that WUnderground does'nt understand the upload (or something equivalent to that).
On the other hand, in the screenshot in the Graph I see that some times WUnderground is accepting the value for pressure (because otherwise it would not draw the black line).

Upload can only be eligible if the source-data is present & valid.

Check the aspect of consistency of data for upload:
if you check in Domoticz for pressure the graph in the Log, is the graph-line continuous without big jumps in time?
if you see a big jump in time in that graph, can you at WUnderground recognize in the graph a 'failure' at the same place (or reverse)?

(double)check the script:
- Did you put the correct name for the Domoticz' pressure in line 07 in the script, linking to your Netatmo barometer?
[must be, because you sometimes get a graphline in WUnderground]
- Looking at the script you quoted, I am wondering whether lines 53 ~58 in that script are applicable for your Netatmo, because that script-version was written to get pressure from a BMP180-sensor, which is separate from the PWS, and also provides temperature.
For a more straightforward solution perhaps you should take out those lines 53~58 and replace the lines 66~68, which now read

Code: Select all

if Barometer ~= '' then
   WU_URL = WU_URL .. "&baromin=" .. string.format("%2.2f", hPatoInches(sBMP180B))
end
by these 3 lines from the original script of Gizmocuz

Code: Select all

if Barometer ~= '' then
	WU_URL = WU_URL .. "&baromin=" .. string.format("%2.2f", hPatoInches(otherdevices_barometer[Barometer]))
end
Set1 = RPI-B+RFXCom433+S0PCM+Linksprite-shield for BMP180/DS18B20/RS485+DDS238-1ZN
Set2 = RPI-3+RFLinkGTW+ESP8266s+PWS_WS7000
Common = 3*PVLogger+PWS_TFA_Nexus+KAKUs
=> Energy & Data Management based on Time and on PV&Consumption&Meteo

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Thursday 01 March 2018 2:20

Good night, this really solved my problem!
Thank you very much! Any questions you might have, I ask for help.

remember
Posts: 15
Joined: Tuesday 27 February 2018 2:16
Target OS: Windows
Domoticz version:
Contact:

Re: Weather Underground Upload script

Post by remember » Thursday 08 March 2018 16:22

Good afternoon, since one day I'm having some difficulties in sending data to the WU, and the script is always working well, from one moment to the next, started to repeat data in the WU leaving the web server 127.0.0.1 with some red instead of green.

Does netatmo have readings in the mynetatmo portal every 5 minutes, and I in the hardware part inserted 5 minutes will be the most correct?

I do not know how from moment to moment it begins to give these problems, being a week to work in perfect conditions.

Thank you for your help.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest