Script LUA to read value on a webpage ?

Post Reply
cycloneseb
Posts: 6
Joined: Wednesday 14 December 2016 18:29
Target OS: OS X
Domoticz version:
Contact:

Script LUA to read value on a webpage ?

Post by cycloneseb » Thursday 18 October 2018 15:29

Hello all Domoticz's Experts !

I'm back to the Domoticz Forum to explain my dream :lol:

As I'm living near a lake in Italy, I was a few years ago almost to be flooded ...

As the value of the lake level can be monitored through internet, I'm wondering if it's possible to extract the data from the webpage using by example an LUA script, and then insert it into a dummy device in Domoticz ?

Webpage : http://www.girovaghi.it/METEO/livelli_i ... ticino.php
Data te be removed : after "quota invaso", by example today : 192,81

Is someone already did it, would it be possible to have some tips to extract this data ? or better, a tutorial !

many thanks !

Regards

User avatar
jvdz
Posts: 1446
Joined: Tuesday 30 December 2014 20:25
Target OS: Raspberry Pi
Domoticz version: Stable
Location: Westland/Netherlands
Contact:

Re: Script LUA to read value on a webpage ?

Post by jvdz » Thursday 18 October 2018 18:03

It is actually pretty strait forward to do this.
This script requires LUA socket.http to be installed:

Code: Select all

DEVIDX=123
weburl='http://www.laghi.net/homepage.aspx?tab=3&subtab=2&idlago=3'
domurl='http://192.168.0.xx:8080/json.htm?type=command&param=udevice&idx=' .. DEVIDX .. '&nvalue=0&svalue='
-- retrieve web info
http = require "socket.http";
-- Get info from website
response, status = http.request(weburl)
-- Get level from info
response=response:match('Quota invaso.-<td class="col_right">(.-)</td>')
print(response)
-- update text device IDX
response, status = http.request(domurl..response)
print(status,response)
I leave it to you to either schedule it outside of domoticz ( which I would do) or incorporate it in an event script and only do the update x times per day and add the required error checking.
Let me know when you have questions.

Jos

cycloneseb
Posts: 6
Joined: Wednesday 14 December 2016 18:29
Target OS: OS X
Domoticz version:
Contact:

Re: Script LUA to read value on a webpage ?

Post by cycloneseb » Saturday 27 October 2018 18:39

Hello Jos;

many thanks for your answer !

I'm back home and trying your code, but up to now, I got some errors due to the wrong location of the LUA socket.

Thanks again !

Seb.

User avatar
jvdz
Posts: 1446
Joined: Tuesday 30 December 2014 20:25
Target OS: Raspberry Pi
Domoticz version: Stable
Location: Westland/Netherlands
Contact:

Re: Script LUA to read value on a webpage ?

Post by jvdz » Saturday 27 October 2018 18:51

Look at DTGBOT installation in the wiki, which has a download for it and tells you where it needs to be located. :)

Edit: this is the forum post with links: http://domoticz.com/forum/viewtopic.php ... lit=Dtgbot
Wiki link: https://www.domoticz.com/wiki/Remote_Co ... legram_Bot
Jos

cycloneseb
Posts: 6
Joined: Wednesday 14 December 2016 18:29
Target OS: OS X
Domoticz version:
Contact:

Re: Script LUA to read value on a webpage ?

Post by cycloneseb » Sunday 28 October 2018 19:39

Hi Jos;

many thanks for your help, I succeeded to install the sockett and now receive the right information.

but, if I can again abuse of your knowledge....

When I'm trying to send the information to update the device using the LUA script, (http.request ('http://192.168.....), no update is done on the device, but surprisingly, inserting the direct adress into my webbrowser, the update is done without any problem !!

do you've an idea of the trouble ? and how to solve it ?

Many thanks again

Seb

User avatar
jvdz
Posts: 1446
Joined: Tuesday 30 December 2014 20:25
Target OS: Raspberry Pi
Domoticz version: Stable
Location: Westland/Netherlands
Contact:

Re: Script LUA to read value on a webpage ?

Post by jvdz » Sunday 28 October 2018 19:47

Abuse huh? :)

You really will have to provide the script with the code you tried to be able to see what could be wrong.
Also tell me what type of dummy device you try to update? (text?)

Jos

cycloneseb
Posts: 6
Joined: Wednesday 14 December 2016 18:29
Target OS: OS X
Domoticz version:
Contact:

Re: Script LUA to read value on a webpage ?

Post by cycloneseb » Sunday 28 October 2018 20:02

not abuse, let say .. benefit too much from your time :lol:

please find the code from the LUA script

Code: Select all

DEVIDX=7180
weburl='http://www.laghi.net/homepage.aspx?tab=3&subtab=2&idlago=3'

domurl='http://192.168.1.102:8080/json.htm?type=command&param=udevice&idx=' .. DEVIDX .. '&nvalue=0&svalue='

-- retrieve web info

local http = require "socket.http";

-- Get info from website

commandArray={}

domurl='http://192.168.1.102:8080/json.htm?type=command&param=udevice&idx=7180&nvalue=0&svalue='

response, status = http.request(weburl)

-- Get level from info

response=response:match('Quota invaso.-<td class="col_right">(.-)</td>')
print(response)

-- update text device IDX

-- Trial to insert a predefined value(500) :

response, status = http.request('http://192.168.1.102:8080/json.htm?type=command&param=udevice&idx=7180&nvalue=0&svalue=500')

print(status,response)


return commandArray

from the log, I have this comment :
LUA: <html><head><title>Unauthorized</title></head><body><h1>401 Unauthorized</h1></body></html
but, when I' typing directly into my web browser "http://192.168.1.102:8080/json.htm?type ... svalue=500", the right value (500) is displayed on the device !

The device to be updated is a custom sensor.

Many thanks for your help

Seb

User avatar
jvdz
Posts: 1446
Joined: Tuesday 30 December 2014 20:25
Target OS: Raspberry Pi
Domoticz version: Stable
Location: Westland/Netherlands
Contact:

Re: Script LUA to read value on a webpage ?

Post by jvdz » Sunday 28 October 2018 20:51

Are you using Userid/Password security for Domoticz?
When the script is running on the same server you could make the url:

Code: Select all

response, status = http.request('http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=7180&nvalue=0&svalue=500')
And make sure the 127.0.0.1 is in the Setup/Settings page defined in
Local Networks (no username/password): Networks:

Jos

cycloneseb
Posts: 6
Joined: Wednesday 14 December 2016 18:29
Target OS: OS X
Domoticz version:
Contact:

Re: Script LUA to read value on a webpage ?

Post by cycloneseb » Tuesday 30 October 2018 23:53

Hi Jos;

I just come back to you to thank you for your precious help, now the lua script is working fine .. and I can manage the level of the lake every 15' (still increasing those days).

Many many thanks for your time and for sharing your knowledge !

for interested people, I join the code :

Code: Select all

-- Website
weburl='http://www.laghi.net/homepage.aspx?tab=3&subtab=2&idlago=3'

-- domurl='http://192.168.1.102:8080/json.htm?type=command&param=udevice&idx=' .. DEVIDX .. '&nvalue=0&svalue='

-- LUA socket http
local http = require "socket.http"

-- Current time
t1=os.time()

-- Last upadte of the domoticz display
s = otherdevices_lastupdate['Niveau Lac Majeur']
year = string.sub(s, 1, 4)
month = string.sub(s, 6, 7)
day = string.sub(s, 9, 10)
hour = string.sub(s, 12, 13)
minutes = string.sub(s, 15, 16)
seconds = string.sub(s, 18, 19)


commandArray={}

print('Script LUA - Lake level')

-- 1° time difference

t2 = os.time{year=year, month=month, day=day, hour=hour, min=minutes, sec=seconds}
difference = (os.difftime (t1, t2))

if (difference > 900 and difference < 3600) then

        -- 2° website information
        response, status = http.request(weburl)

        response=response:match('Quota invaso.-<td class="col_right">(.-)</td>')

        -- send info to the virtual device
        commandArray['UpdateDevice']='7180|0|'..response
end

return commandArray

Post Reply

Who is online

Users browsing this forum: ArnieO and 2 guests