Import RIVM sensor data

Must be cleaned all topics go to the right subforum
Post Reply
User avatar
phoenixblue
Posts: 61
Joined: Friday 25 November 2016 12:20
Target OS: Raspberry Pi
Domoticz version: 3.7392
Contact:

Import RIVM sensor data

Post by phoenixblue » Friday 28 April 2017 22:14

Hi,

Small background info, because I life nearby chemical industy where it's most of the time safe ;) but for that one moment there is an problem I like to shutdown my mechanical ventilation at home.
So I have found that based on the website https://www.luchtmeetnet.nl/ the info from the RIVM is avalible to use (http://www.lml.rivm.nl/tabel/)
(sorry only usefull for the Netherlands)
From that part is possible to import the data into Domoticz, but therefore you must manual insert total 7 virtual text sensors with the next labels:
PM10 = Fijnstof (particulates)
O3 = Ozon (ozone)
NO = Stikstofmonoxide (nitric oxide)
CO = Koolstofmonoxide (carbon monoxide)
SO2 = Zwaveldioxide (sulfur dioxide)
NH3 = Ammoniak (Ammonia) -> (R717 for the Refrigeration engineers :) )
NO2 = Stikstofdioxide (nitrogen dioxide)

then inside the script:
1) set the idx codes at the right place.
2) set the used AirQuality location from the website nearby your location into the script (sorry it's in the bottem of the script)

Code: Select all

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import xml.etree.ElementTree as ET
import requests
from requests.exceptions import ConnectionError

domoticz_devicesIdx = {"PM10": 1,
                       "O3": 2,
                       "NO": 3,
                       "CO": 4,
                       "SO2": 5,
                       "NH3": 6,
                       "NO2": 7
                       }

domoticz_server = "http://localhost"
domoticz_port = 8080


class AirQuality:

    def __init__(self, location):
        self.link = "http://www.lml.rivm.nl/xml/actueel.xml"
        self.location = location
        timedata = "http://www.lml.rivm.nl/xml/actueel-update.xml"
        z = requests.get(timedata)
        f = ET.fromstring(z.content)
        data = f.getchildren()[0].getchildren()[0].getchildren()
        year = data[0].text
        month = data[1].text
        day = data[2].text
        hour = data[3].text
        self.fetch()

        print "Fetching data from: %s/%s/%s, hour: %s" % (day, month, year, hour)

    def push(self, device, value):
        try:
            requests.get(
                "%s:%d/json.htm?type=command&param=udevice&idx=%d&nvalue=0&svalue=%s" %
                (domoticz_server, domoticz_port, device, value))
        except ConnectionError:
            print "I wasn't able to contact the domoticz server, is it up?"

    def fetch(self):
        # fetch the data and push it
        z = requests.get(self.link)
        elements = ET.fromstring(z.content).getchildren()[0].getchildren()
        components = {}
        for element in elements:
            data = element.getchildren()
            station = data[2].text
            component = data[3].text
            unit = data[4].text
            value = data[5].text
            if not (station.lower() == self.location.lower()):
                continue

            components[component] = '%s%s' % (value, unit)

        for component in domoticz_devicesIdx.keys():
            device = domoticz_devicesIdx[component]
            try:
                value = components[component]
            except KeyError:
                value = "n/a"

            self.push(device, value)


AirQuality("NL01489")

When you run the script it wil import al avalible data, when an sensor station has for an specific gas no sensor there will be send an n/a value.
At this moment there are an few point that must be changed in the future:
- text sensor to airquality sensor with the correct units.
- find out the save thresholds of the sensors and send it maybe to an alert level sensor.
scripts on github: Link

assenzuid
Posts: 95
Joined: Friday 13 November 2015 10:11
Target OS: Raspberry Pi
Domoticz version: Beta
Location: The Netherlands
Contact:

Re: Import RIVM sensor data

Post by assenzuid » Tuesday 06 June 2017 22:09

I get below error

Code: Select all

Traceback (most recent call last):
  File "airquality.py", line 71, in <module>
    AirQuality("NL01489")
  File "airquality.py", line 34, in __init__
    self.fetch()
  File "airquality.py", line 69, in fetch
    self.push(device, value)
  File "airquality.py", line 42, in push
    (domoticz_server, domoticz_port, device, value))
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 60, in get
    return request('get', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 49, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 563, in send
    adapter = self.get_adapter(url=request.url)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 642, in get_adapter
    raise InvalidSchema("No connection adapters were found for '%s'" % url)
requests.exceptions.InvalidSchema: No connection adapters were found for '192.168.1.xxx:8081/json.htm?type=command&param=udevice&i  dx=309&nvalue=0&svalue=n/a'
Moved recently to Emmen.

User avatar
EdwinK
Posts: 787
Joined: Sunday 22 January 2017 22:46
Target OS: NAS (Synology & others)
Domoticz version: 3.8552
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK » Sunday 20 August 2017 22:20

Thanks. Just something I was looking for, living very close to one of the chemical parts of the country (Botlek).
Running latest STABLE on a NAS Synology 413j | Toon Thermostat | RFxcom | Dashticz V2 |

User avatar
phoenixblue
Posts: 61
Joined: Friday 25 November 2016 12:20
Target OS: Raspberry Pi
Domoticz version: 3.7392
Contact:

Re: Import RIVM sensor data

Post by phoenixblue » Monday 21 August 2017 22:47

EdwinK wrote:Thanks. Just something I was looking for, living very close to one of the chemical parts of the country (Botlek).
I think you can test it at this moment ;-)

Grote brand bij Esso in de Botlek

http://www.ad.nl/rotterdam/grote-brand- ... ~a00c1915/
scripts on github: Link

User avatar
EdwinK
Posts: 787
Joined: Sunday 22 January 2017 22:46
Target OS: NAS (Synology & others)
Domoticz version: 3.8552
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK » Tuesday 22 August 2017 8:45

Yes. Works :)
Running latest STABLE on a NAS Synology 413j | Toon Thermostat | RFxcom | Dashticz V2 |

User avatar
rbisschops
Posts: 32
Joined: Monday 12 May 2014 11:20
Target OS: Raspberry Pi
Domoticz version: 2.2563
Location: Schiedam, Netherlands
Contact:

Re: Import RIVM sensor data

Post by rbisschops » Tuesday 22 August 2017 9:00

Did you see this note:
De website www.lml.rivm.nl zal langzaam maar zeker zijn inhoud verliezen.
Recentelijk zijn de kaarten met een geinterpoleerde weergave van de meetdata al verwijderd.
Deze zijn namelijk ook te vinden op de gezamenlijke website.

Apparently the data will move to the other site.

Ralph
1x Raspberry Pi B with P1 connection and weather underground
1x Raspberry Pi 2+ with RFXtrx433, many CoCo units and a Harmony Hub
MySensors with custom built watermeter sensors on Aquadis+

User avatar
Derik
Posts: 1832
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik » Tuesday 22 August 2017 20:34

mmm Is this sensor usefull after the lost of this data??
I cannot find my station [ Nijmegen keizerkarel ]

Looks a great script...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups

User avatar
EdwinK
Posts: 787
Joined: Sunday 22 January 2017 22:46
Target OS: NAS (Synology & others)
Domoticz version: 3.8552
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK » Tuesday 22 August 2017 20:54

NL10741 RIVM Nijmegen-Graafseweg
NL10742 RIVM Nijmegen-Ruyterstraat

Use one of these.
Running latest STABLE on a NAS Synology 413j | Toon Thermostat | RFxcom | Dashticz V2 |

User avatar
Derik
Posts: 1832
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik » Tuesday 22 August 2017 20:57

EdwinK wrote:
Tuesday 22 August 2017 20:54
NL10741 RIVM Nijmegen-Graafseweg
NL10742 RIVM Nijmegen-Ruyterstraat

Use one of these.

Thanks...

Can i run with the internal interface or?

Where can i find the stations??
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups

User avatar
EdwinK
Posts: 787
Joined: Sunday 22 January 2017 22:46
Target OS: NAS (Synology & others)
Domoticz version: 3.8552
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK » Tuesday 22 August 2017 21:29

Station:
from the RIVM is avalible to use (http://www.lml.rivm.nl/tabel/)

I created the script external, but I guess you can use the internal editor.
Running latest STABLE on a NAS Synology 413j | Toon Thermostat | RFxcom | Dashticz V2 |

User avatar
Derik
Posts: 1832
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik » Tuesday 22 August 2017 22:14

mmm internal is working by me...

@ Developer is it perhaps possible to use the internal Python options?
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups

User avatar
Derik
Posts: 1832
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik » Friday 08 September 2017 8:49

is it perhaps possible that the sensors send a notification??


So i can warn my kids there is smog or something??
What are the values that are good and what are bad values?
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups

User avatar
Derik
Posts: 1832
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik » Friday 08 September 2017 21:51

There is also a Roet in the data... Is there a option to use this data to?
And the Nijmegen sensor are both not working...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups

User avatar
Derik
Posts: 1832
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik » Saturday 09 September 2017 8:43

Looks like the sensors from Nijmegen are not in the list:
http://www.lml.rivm.nl/xml/actueel.xml

Perhaps the problem that the siet is getting to a other service?
luchtmeetnet?


Is there a other option?

Love to have to have this option in domoticz...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups

antonlamers
Posts: 1
Joined: Wednesday 08 March 2017 15:46
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: Import RIVM sensor data

Post by antonlamers » Thursday 12 October 2017 8:34

Hi,

It doesn't work in my domoticz.
The virtual text sensor say "hello world"

Anton

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests