pfSense Internet Usage and Bandwidth

All kinds of 'OS' scripts
Post Reply
ben53252642
Posts: 378
Joined: Saturday 02 July 2016 5:17
Target OS: Linux
Domoticz version: Beta
Contact:

pfSense Internet Usage and Bandwidth

Post by ben53252642 » Saturday 25 February 2017 2:45

Hey Folks,

This is a bash script I wrote for getting the last minutes bandwidth information from pfsense and loading it into Domoticz.

Steps:
1) Go to this url on your pfsense box: https://pfsenseip/services_snmp.php
2) Tick the box to enable snmp and set the "Read Community String" to: "public" then scroll down and bind it to the LAN interface and press save.
3) Install requirements on your Domoticz server: apt-get install snmp bc
3) Fill in everything in the script except for the "waninterfacename"
3) Run the script like this: "./update.sh list" and you will get an output looking something like this:

vtnet0
vtnet1
enc0
lo0
pflog0
pfsync0
pppoe0
ovpns1

In this case my wan interface is called pppoe0 so what is what I would enter in "waninterfacename" in the script (go ahead and fill the script with the name of your wan interface).

update.sh

Code: Select all

#!/bin/bash

# Pfsense Configuration
pfsenseip="192.168.0.1"
snmpid="public"
snmpversion="2c"
waninterfacename="pppoe0"
measuretime="60" # If you want avg mbps per minute, this would be 60

# Domoticz Configuration
domoticzserver="192.168.0.5"
domoticzport="443"
domoticzuser="username"
domoticzpass="password"

# Identify the WAN interface
interfacelist=$(snmpwalk -c "$snmpid" -v "$snmpversion" "$pfsenseip" .1.3.6.1.2.1.2.2.1.2)
wansnmpid=$(echo "$interfacelist" | grep "$waninterfacename" | awk '{ print $1 }' | grep -o '.\{1\}$')

# Get list of interfaces and display them in terminal
if [[ $1 == 'list' ]]; then
echo "$interfacelist" | awk '{ print $4 }' | sed 's/\"//g'
exit
fi

# Start work loop
while true; do

# Notify terminal
echo && echo "Interface: $waninterfacename"
echo "Calculating bandwidth per $measuretime seconds in mbps... (wait $measuretime seconds)"

# Get data
waninkb=$(snmpwalk -c "$snmpid" -v "$snmpversion" "$pfsenseip" .1.3.6.1.2.1.2.2.1.10."$wansnmpid" | awk '{ print $4 }')
wanoutkb=$(snmpwalk -c "$snmpid" -v "$snmpversion" "$pfsenseip" .1.3.6.1.2.1.2.2.1.16."$wansnmpid" | awk '{ print $4 }')

# Wait time period
sleep "$measuretime"

# Get data a second time
waninkb2=$(snmpwalk -c "$snmpid" -v "$snmpversion" "$pfsenseip" .1.3.6.1.2.1.2.2.1.10."$wansnmpid" | awk '{ print $4 }')
wanoutkb2=$(snmpwalk -c "$snmpid" -v "$snmpversion" "$pfsenseip" .1.3.6.1.2.1.2.2.1.16."$wansnmpid" | awk '{ print $4 }')

# Do math for Mbps
inbits=$(echo $((("$waninkb2"-"$waninkb") / 60 * 8)) | bc -l)
inmegabits=$(echo "scale=2; $inbits / 1000000" | bc -l)
outbits=$(echo $((("$wanoutkb2"-"$wanoutkb") / 60 * 8)) | bc -l)
outmegabits=$(echo "scale=2; $outbits / 1000000" | bc -l)

# Check data viability
if (( $(echo "$inbits > 2" |bc -l) )) && (( $(echo "$outbits > 2" |bc -l) )); then

# Notify Terminal 2
echo "In: $inmegabits Mbps"
echo "Out: $outmegabits Mbps"

# Send data to Domoticz
curl --max-time 60 -s -k "https://{$domoticzuser}:{$domoticzpass}@{$domoticzserver}:{$domoticzport}/json.htm?type=command&param=udevice&idx=503&svalue=${inmegabits}"
curl --max-time 60 -s -k "https://{$domoticzuser}:{$domoticzpass}@{$domoticzserver}:{$domoticzport}/json.htm?type=command&param=udevice&idx=502&svalue=${outmegabits}"

# Report if there was a problem
else
echo "There was a problem getting the data"
fi

done
This is the startup.sh script I use for running it in the background:

Code: Select all

# Start pfsense monitor
/usr/bin/screen -S pfsense-monitor -d -m /root/pfsense/update.sh
screenshot112532.JPG
screenshot112532.JPG (77.33 KiB) Viewed 749 times
terminaloutput1534.JPG
terminaloutput1534.JPG (41.67 KiB) Viewed 745 times
Last edited by ben53252642 on Thursday 26 April 2018 23:50, edited 5 times in total.

User avatar
Varazir
Posts: 219
Joined: Friday 20 February 2015 23:23
Target OS: Raspberry Pi
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by Varazir » Sunday 12 March 2017 11:33

Great script but wouldn't be good to set the IDX as variables too?

Abit howto setup the dummy devices too.

I added it to the crontab so I don't need to start the screen if I reboot my server.
Removed the while loop and sleep

Added custom icons
Attachments
domoticz_custom_icon_upload.zip
(4.36 KiB) Downloaded 79 times
domoticz_custom_icon_download.zip
(4.38 KiB) Downloaded 62 times
Last edited by Varazir on Sunday 12 March 2017 17:54, edited 1 time in total.
1x Raspberry PI A with a RaZberry board
1x GreenWave 6 port Power node
1x tellstick
1x Qubino ZMNHOD1 ( controlling a Ikea blinder )
1x ZW075 Smart Switch Gen5
Several nexa/proove devices

Live Long and Prosper \\//_

User avatar
Varazir
Posts: 219
Joined: Friday 20 February 2015 23:23
Target OS: Raspberry Pi
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by Varazir » Sunday 12 March 2017 17:52

and this icons

They look like this
pfsense.JPG
pfsense.JPG (51.15 KiB) Viewed 3400 times
Attachments
domoticz_custom_icon_upspeed.zip
(2.72 KiB) Downloaded 62 times
domoticz_custom_icon_downspeed.zip
(2.84 KiB) Downloaded 60 times
1x Raspberry PI A with a RaZberry board
1x GreenWave 6 port Power node
1x tellstick
1x Qubino ZMNHOD1 ( controlling a Ikea blinder )
1x ZW075 Smart Switch Gen5
Several nexa/proove devices

Live Long and Prosper \\//_

ben53252642
Posts: 378
Joined: Saturday 02 July 2016 5:17
Target OS: Linux
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by ben53252642 » Wednesday 29 March 2017 13:47

Awesome! Love the icons! :D

If anyone finds a simpler way to get the data out of pfsense please let me know.

void
Posts: 32
Joined: Wednesday 04 November 2015 11:26
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by void » Monday 06 November 2017 12:03

Anyone tried with the 2.4 branch? Some things got a major update, such as the login screen.

ben53252642
Posts: 378
Joined: Saturday 02 July 2016 5:17
Target OS: Linux
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by ben53252642 » Monday 13 November 2017 10:13

void wrote:
Monday 06 November 2017 12:03
Anyone tried with the 2.4 branch? Some things got a major update, such as the login screen.
Can confirm that existing script works fine with 2.4.* branch.

void
Posts: 32
Joined: Wednesday 04 November 2015 11:26
Target OS: Raspberry Pi
Domoticz version:
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by void » Wednesday 31 January 2018 10:21

Did you consider creating a Wiki article for it at http://www.domoticz.com/wiki/Main_Page ?
Might be interessting for more people, and the forum search function kind of sucks... :)

ben53252642
Posts: 378
Joined: Saturday 02 July 2016 5:17
Target OS: Linux
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by ben53252642 » Thursday 01 February 2018 15:55

void wrote:
Wednesday 31 January 2018 10:21
Did you consider creating a Wiki article for it at http://www.domoticz.com/wiki/Main_Page ?
Might be interessting for more people, and the forum search function kind of sucks... :)
I tried to add something to the wiki recently but was unable to create an account:

viewtopic.php?f=43&p=167358#p167358

I think for now this is ok in the Forum, also people can post if they have problems and it's easy to reply.

When I can access the Wiki perhaps I'll make a page directing people here, Ultimately it would be better if there was a native Domoticz plugin. Until then we have this bash script.

MnM001
Posts: 4
Joined: Wednesday 31 January 2018 4:42
Target OS: Linux
Domoticz version:
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by MnM001 » Thursday 15 March 2018 2:27

Hi guys looking for some help here.

Where I run the script I get:

./test.sh: line 19: /usr/bin/curl: Argument list too long
error : Unknown IO error

The script does log into my pfsense as I can see it in my logs.
DarkStat and Totals configured correctly.

I am a beginner with scripting so I will appreciate if someone can help me resolve this error.

Thanks

domoticzcom1234
Posts: 20
Joined: Saturday 25 June 2016 17:21
Target OS: Raspberry Pi
Domoticz version: Stable
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by domoticzcom1234 » Saturday 24 March 2018 21:04

This can be caused by many things. Answering your question therefore is not very easy.
This implementation can be very complicated and is time consuming to get it working.

The only thing I can do is give you a basic guidance as listed below:
- Did you create an account on Pfsense
- Did you gave the correct permissions to access the right modules
- login as the new user and password and run this in your browser: https://IP-ADDRESS-OF-YOUR-PFSENSE/vnst ... h_json.php, are you getting the right response? No?, your permissions are set wrong.
- Are you using the right PFsense network interface and is it in the same order listed in the PFsense URL in the script (try this in the browser first as listed above)
- did you installed jq, bc AND libxml2-utils
- are you using the latest version of pfsense
- do you have the right permissions in linux

And the best one to check:
- run the script with the -x such as "sh -x scriptname.sh" so you can check what goes wrong



Good luck and this is the only thing I can do.

ben53252642
Posts: 378
Joined: Saturday 02 July 2016 5:17
Target OS: Linux
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by ben53252642 » Tuesday 10 April 2018 23:07

I'm in the process of completely re-writing this to work from SNMP and not require either bandwidth totals or darkstat on Pfsense.

I've completed the bandwidth part, it's currently installed on my Domoticz box updating nicely every minute.

I'm now working on the monthly bandwidth calculations. Should be posted in the next couple of days, stay tuned.

User avatar
Varazir
Posts: 219
Joined: Friday 20 February 2015 23:23
Target OS: Raspberry Pi
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by Varazir » Saturday 21 April 2018 10:31

ben53252642 wrote:
Tuesday 10 April 2018 23:07
I'm in the process of completely re-writing this to work from SNMP and not require either bandwidth totals or darkstat on Pfsense.

I've completed the bandwidth part, it's currently installed on my Domoticz box updating nicely every minute.

I'm now working on the monthly bandwidth calculations. Should be posted in the next couple of days, stay tuned.
Sweet
1x Raspberry PI A with a RaZberry board
1x GreenWave 6 port Power node
1x tellstick
1x Qubino ZMNHOD1 ( controlling a Ikea blinder )
1x ZW075 Smart Switch Gen5
Several nexa/proove devices

Live Long and Prosper \\//_

ben53252642
Posts: 378
Joined: Saturday 02 July 2016 5:17
Target OS: Linux
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by ben53252642 » Thursday 26 April 2018 23:45

I've updated the first post with the new code which I believe is significantly better than the previous version.

Note that for now it's only the last minutes bandwidth (not monthly downloads), I'm still working on a better way to do that.

User avatar
Varazir
Posts: 219
Joined: Friday 20 February 2015 23:23
Target OS: Raspberry Pi
Domoticz version: Beta
Contact:

Re: pfSense Internet Usage and Bandwidth

Post by Varazir » Wednesday 02 May 2018 21:55

ben53252642 wrote:
Thursday 26 April 2018 23:45
I've updated the first post with the new code which I believe is significantly better than the previous version.

Note that for now it's only the last minutes bandwidth (not monthly downloads), I'm still working on a better way to do that.
will not Domoticz keep track of the monthly data traffic ? ?
1x Raspberry PI A with a RaZberry board
1x GreenWave 6 port Power node
1x tellstick
1x Qubino ZMNHOD1 ( controlling a Ikea blinder )
1x ZW075 Smart Switch Gen5
Several nexa/proove devices

Live Long and Prosper \\//_

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests