Nested HTTPResponse

Easy to use, 100% Lua-based event scripting framework.
Post Reply
elmortero
Posts: 233
Joined: Sunday 29 November 2015 21:46
Target OS: Raspberry Pi
Domoticz version: 3.9639
Location: Spain
Contact:

Nested HTTPResponse

Post by elmortero » Sunday 06 May 2018 12:41

Hi all,

Our national Weather Service has a rather complicated way of fetching info.
FIrst you access the API with a URL constructed of several parameters (type of data you want, stationID and API key):
https://opendata.aemet.es/opendata/api/ ... key=XXXXXX
That results in a JSON response like:

Code: Select all

{
descripcion: "éxito",
estado: 200,
datos: "https://opendata.aemet.es/opendata/sh/ff3de32d",
metadatos: "https://opendata.aemet.es/opendata/sh/8ca2e7e3",
}
Where a temporary URL is generated, based on you request. It is valid for 5 minutes. The URL that leads to the actual info is in the field called "Datos".

I have tried several approaches to fetch the data I want but so far all fail..
One method I used was running 2 separate scripts: 1 to fetch the "Datos" field and write it to a variable (both dzvents persisten and domoticz user variable) and then 2 minutes later a second one that calls the generated URL. With that one I always get nil data or invalid JSON result. When the URL has been fetched and I open it by copy/past from the variable I get a valid JSON with the data available.

Second method I have been trying is combining the URLs in one script (like below) but that does not give the results either.
It does get the "datos" field, which I know by both printing it to the log and writing it to the variable.
But I do not get it to trigger the part where the final data are fetched.
There are no errors shown, it just does not trigger.

Hope the many eyes here spot the (probably) silly mistake I am making.
The current script I have is

Code: Select all

return {
	on = {
		timer = { 'at *:26', 'at *:52' },
		httpResponses = { 'aemeturl', 'aemetInfo' } -- matches callback string below
	},
	
	execute = function(domoticz, triggerItem)

		if (triggerItem.isTimer) then
			domoticz.openURL({
				url = 'https://opendata.aemet.es/opendata/api/prediccion/especifica/playa/0301408/?api_key=MYKEY',
				method = 'GET',
				callback = 'aemeturl'
			})
			
		elseif (triggerItem.isHTTPResponse) then

	local response = triggerItem
		if (response.ok and response.isJSON and response.trigger == 'aemeturl') then
		 local Aemetlink = tostring(response.json.datos)
			--processing
			print('Aemetlink = '..Aemetlink)
			domoticz.variables('Aemetlink').set(Aemetlink)
				domoticz.openURL({
					url = Aemetlink,
					method = 'GET',
					callback = 'aemetInfo'
				})
			--end processing
			end
			
			
		if (response.ok and response.isJSON and response.trigger == 'aemetInfo') then			
			local city = tostring(response.json.nombre)  	--location
			print('City = '..city)
			local tagua = domoticz.round((tonumber(response.json.prediccion.dia[1].tagua.valor1)),1)		--visibility
			print('Agua = '..tagua)
			local wave = tostring(response.json.prediccion.dia[1].oleaje.descripcion2)
			print('Oleaje = '..wave)
		end
	
		end
	end
}


dannybloe
Posts: 1473
Joined: Friday 29 August 2014 11:26
Target OS: Raspberry Pi
Domoticz version:
Location: Ermelo
Contact:

Re: Nested HTTPResponse

Post by dannybloe » Sunday 06 May 2018 14:31

Have you tried to set it in debug mode and see what is being logged? Also try to log the response headers and the raw data.
Creator dzVents - RPi3, loads of zwave devices, esp8266, evohome.

elmortero
Posts: 233
Joined: Sunday 29 November 2015 21:46
Target OS: Raspberry Pi
Domoticz version: 3.9639
Location: Spain
Contact:

Re: Nested HTTPResponse

Post by elmortero » Sunday 06 May 2018 23:02

Hi,

Seems that the headers say it is text/plain, even though a copy and paste to jsonlint.com says is a valid JSON table.

Code: Select all

dzVents: Info: beach: {["Accept"]="*/*", ["Content-Length"]="2666", ["Server"]="Apache/2.2.15 (CentOS)", ["Connection"]="close", ["Date"]="Sun, 06 May 2018 20:40:14 GMT", ["Content-Type"]="text/plain;charset=ISO-8859-15"}
With that knowledge, I altered HTTPResponse.lua (I know, shouldn't mess there :P ) to validate as JSON also when the headers say that the content-type is text/plain.
And guess what: It works!

dannybloe
Posts: 1473
Joined: Friday 29 August 2014 11:26
Target OS: Raspberry Pi
Domoticz version:
Location: Ermelo
Contact:

Re: Nested HTTPResponse

Post by dannybloe » Monday 07 May 2018 7:04

You don’t need to do that. You can convert the raw data to json in your script using domoticz.utils. And report a bug that they don’t set their content headers properly.
Creator dzVents - RPi3, loads of zwave devices, esp8266, evohome.

dannybloe
Posts: 1473
Joined: Friday 29 August 2014 11:26
Target OS: Raspberry Pi
Domoticz version:
Location: Ermelo
Contact:

Re: Nested HTTPResponse

Post by dannybloe » Monday 07 May 2018 7:06

Besides. Your change will throw errors if the data is text and not json.
Creator dzVents - RPi3, loads of zwave devices, esp8266, evohome.

elmortero
Posts: 233
Joined: Sunday 29 November 2015 21:46
Target OS: Raspberry Pi
Domoticz version: 3.9639
Location: Spain
Contact:

Re: Nested HTTPResponse

Post by elmortero » Monday 07 May 2018 11:00

dannybloe wrote:
Monday 07 May 2018 7:04
You don’t need to do that. You can convert the raw data to json in your script using domoticz.utils. And report a bug that they don’t set their content headers properly.
hi Daniel,

You are talking about fromJSON(json) I assume? Already been looking at that but honestly I am having a hard time figuring out how to use it :-(
About reporting it to the provider, done. Let's see how they respond.

dannybloe
Posts: 1473
Joined: Friday 29 August 2014 11:26
Target OS: Raspberry Pi
Domoticz version:
Location: Ermelo
Contact:

Re: Nested HTTPResponse

Post by dannybloe » Monday 07 May 2018 11:51

This should do the trick (it is exactly the same as in HTTPResponse.lua that you adapted):

Code: Select all

domoticz.utils.fromJSON(response.data)
And response is the same object as in your script.

You see, the reason why the json is available on the response object in dzVents is more of a convenience. IF the response type is application/json THEN I do the conversion to Lua automatically because I can be sure that it can be converted and it is likely you will need it in your script. On the other hand, if the response type is not like this (as in your situation) but you KNOW that it is json, then you can do it yourself with the code I gave you above.
Creator dzVents - RPi3, loads of zwave devices, esp8266, evohome.

elmortero
Posts: 233
Joined: Sunday 29 November 2015 21:46
Target OS: Raspberry Pi
Domoticz version: 3.9639
Location: Spain
Contact:

Re: Nested HTTPResponse

Post by elmortero » Monday 07 May 2018 12:42

Great!
It works now.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest