Do you have a DD-WRT enabled router and want network statistics graphics for all the devices at your home or office? It’s pretty easy with InfluxDB and Grafana!
Installing InfluxDB
InfluxDB is a time series database. This means it is specialized in collecting, storing and querying sequences of measurements over a time interval. It is great doing this job, and since it’s exactly the job that we need, we’re going to use it.
As always the steps on this article are for Debian-based Linux distributions. Just issue these commands:
naikel@htpc ~ $ wget https://s3.amazonaws.com/influxdb/influxdb_0.10.1-1_amd64.deb naikel@htpc ~ $ sudo dpkg -i influxdb_0.10.1-1_amd64.deb naikel@htpc ~ $ sudo /etc/init.d/influxdb start Starting the process influxdb [ OK ] influxdb process was started [ OK ]
If you prefer to add a PPA to your repository list, follow these instructions instead.
If everything went fine then we can login through the console like this:
naikel@htpc ~ $ influx Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. Connected to http://localhost:8086 version 0.10.1 InfluxDB shell 0.10.1 >
Or we could log in through the GUI at http://localhost:8083
Configuring InfluxDB
We need to configure the database for our network traffic data. Let’s just create a new database called traffic. You can do it through the console or through the Web GUI:
naikel@htpc ~ $ influx Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. Connected to http://localhost:8086 version 0.10.1 InfluxDB shell 0.10.1 > create database traffic
Collecting Data
Now we need to populate the database with time based data.
First we need to create a script that will run in the DD-WRT enabled router that will let us gather statistics from every device in the same LAN the router is. The idea is that per every device in your LAN the script will create two entries in a new chain in the enabled router’s iptables, one for incoming packets, and another one for outgoing packets, both with target RETURN, so it returns to the original chain. Then in the FORWARD chain we are going to send to the RRDIPT chain every single packet there, the RRDIPT chain will count the packet, and then will return to the FORWARD chain.
The FORWARD chain will have an entry like this:
root@DD-WRT:/jffs/bin# iptables -L FORWARD -n Chain FORWARD (policy ACCEPT) target prot opt source destination RRDIPT 0 -- 0.0.0.0/0 0.0.0.0/0
And the RRDIPT chain will have an entry per every device like this:
root@DD-WRT:/jffs/bin# iptables -L RRDIPT -n Chain RRDIPT (1 references) target prot opt source destination RETURN 0 -- 192.168.100.107 0.0.0.0/0 RETURN 0 -- 0.0.0.0/0 192.168.100.107
You can find my complete script here. By the way this was not my idea. This is all based in a script called wrtbwmon.
This script has to run every minute in the router’s cron to gather statistics and every hour or so to detect new devices in your LAN. Also remember to change the DBURL environment variable with the URL of your InfluxDB database in your LAN.
To do this you have a couple of choices depending on if you can enable JFFS or not in your router.
JFFS Capable Router
If your router has enough memory then you can enable JFFS. Follow the instruction on how to enable JFFS in your DD-WRT enabled router first and copy the script in /jffs/bin and name it trafficmon (no extension). Remember to give the script execution permissions (chmod +x trafficmon).
Now you have to enable cron in the Administration tab of your router and paste the following lines in the Additional Cron Jobs field:
* * * * * root /jffs/bin/trafficmon update 0 * * * * root /jffs/bin/trafficmon setup
If you want to be more elegant, create instead a startup script in /jffs/etc/config to include the cron lines you need every time you restart your router:
root@DD-WRT:/jffs/etc/config# cat trafficmon.startup #!/bin/sh echo '* * * * * root /jffs/bin/trafficmon update' >/tmp/trafficmon echo '0 * * * * root /jffs/bin/trafficmon setup' >>/tmp/trafficmon mv /tmp/trafficmon /tmp/cron.d/trafficmon /jffs/bin/trafficmon setup
Don’ t forget to give execution permission to the startup script (chmod +x trafficmon.startup).
No JFFS Available
If your router has no enough memory and you don’t have or can’t create a JFFS partition, then you need a web server in your LAN to host the script. Enable cron in the Administration tab of your router and paste the following lines in the Additional Cron Jobs field. Remember to edit them to point to your web server since in the following example we’re assuming the script is hosted at http://192.168.100.100/trafficmon
* * * * * root if [ -f /tmp/root/trafficmon ]; then ( /tmp/root/trafficmon update ); else ( /usr/bin/wget -q http://192.168.100.100/trafficmon -O /tmp/root/trafficmon && chmod +x /tmp/root/trafficmon && /tmp/root/trafficmon setup ); fi 0 * * * * root /tmp/root/trafficmon setup
Checking Work So Far
If you have left your router doing its job for more than a minute then you should already have data in your InfluxDB. You can query the data using the InfluxDB’s Web Interface at port 8083 in your server (remember to choose the database first before querying the data):
Everything looks great! Now we just need to create graphs for this data.
Installing Grafana
Grafana is the software that generates graphics for several time series databases, including of course InfluxDB. Just issue the following commands to install it:
naikel@htpc ~ $ curl -O https://grafanarel.s3.amazonaws.com/builds/grafana_latest_amd64.deb naikel@htpc ~ $ sudo dpkg -i grafana_latest_amd64.deb naikel@htpc ~ $ sudo service grafana-server start * Starting Grafana Server [ OK ] naikel@htpc ~ $
If everything went well you can login at http://localhost:3000. The default login and password are admin.
Configure the Data Source
Let’s configure our InfluxDB as a data source in Grafana. Go to Data Sources and click on New. Fill the form with a name for your InfluxDB database (I used “InfluxDB”) and select InfluxDB as Type. If Grafana and InfluxDB are running in the same server, the URL should be http://localhost:8086. We named our InfluxDB database “traffic” and just type anything in the user and password fields. Those are not used but if they are empty Grafana won’t let you go on.
It should look like this:
Click on Add and we are ready to create some graphs.
Create Some Graphs
Create a new Dashboard clicking on Home and then New and then add a new graph clicking in the green icon at your left (that’s the menu of the row) and then on Add Panel and then Graph.
Select InfluxDB as your query type and then add two queries (one for the downstream and another one for the upstream of the host you want to create the graph for).
Now define the queries like this in the Metrics tab:
As you can see since we are collecting data every minute but we want to show bytes per second, we have to divide the average (the mean) by 60.
You can define some nifty legends and stuff in the Axis & Grid tab:
And the graph should look like this:
Do this for every device in your home or office and that’s it! If you want a graph with all the traffic select the MAC address 00:00:00:00:00:00.
I followed your guide to the letter, and I am getting stats on the 00:00 MAC address, so I know the script and cron are working on my router(R7000). However I’m not getting any network stats for all the other MAC addresses of my various devices. They all show as 0 in InfluxDB. Can you provide any insight? Thanks!
Sounds like the “setup” command is not running properly. That’s the one that looks for all the active IP addresses in your network and create separate rules for them. Remember the cron file has two entries:
* * * * * root /jffs/bin/trafficmon update
0 * * * * root /jffs/bin/trafficmon setup
You can also try to run the command manually in your router (/jffs/bin/traficmon/setup) to recreate the individual hosts rules.
Same here – getting all traffic on 00:00. Also using an R7000.
Tried running setup script both with cron and manually to no avail. I /am/ able to see individual MAC addresses in InfluxDB, but everything besides 00:00 has a zero value.
If you type in your router:
iptables -L RRDIPT -n -v
Are you able to see all the IP addresses in your network with meaningful values on pkts and bytes columns?
If you are seeing RRDIPT empty change this line (line #79) in the trafficmon script (I’m updating the link as well):
iptables -L FORWARD –line-numbers -n | grep “RRDIPT” | grep “1” > /dev/null
to:
iptables -L FORWARD -n | grep RRDIPT[^2] > /dev/null
Or just replace the script with the new one in the link.
What is MAC address 00:00:00:00:00:00?
It seems to be working on all the other MAC addresses on my network but then i get random data on 00. I thought this would be the total but its way less then my top user. My top user had 40gb yesterday and 00 had 2.8.
It is the total.
Check that the very first two rules on the FORWARD chain of iptables are your RRDIPT rules like this:
root@DD-WRT:~# iptables -L FORWARD -n -v
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
86 8046 RRDIPT 0 — * * 0.0.0.0/0 0.0.0.0/0
86 8046 RRDIPT2 0 — * * 0.0.0.0/0 0.0.0.0/0
Also, RRDIPT2 checks for from br0 to eth1 and from eth1 to br0. Maybe your router is different?
root@DD-WRT:~# iptables -L RRDIPT2 -n -v
Chain RRDIPT2 (1 references)
pkts bytes target prot opt in out source destination
62 4717 RETURN 0 — br0 eth1 0.0.0.0/0 0.0.0.0/0
67 5901 RETURN 0 — eth1 br0 0.0.0.0/0 0.0.0.0/0
Check what are the interfaces for LAN and for WAN in your router. In my case (and the case of the script) eth1 is WAN (it has the WAN IP address assigned) and br0 is LAN (it has the LAN IP address). Maybe you have a different router where DD-WRT assigns different interfaces for WAN and LAN.
Actually the script is smart enough to know your interfaces:
LAN_IFNAME=$(nvram get lan_ifname)
WAN_IFNAME=$(nvram get wan_ifname)
But you should double check they are the correct interfaces.
Ya i see now that 00:00:: is inverted.
The issue i am now having is that I have 3 LAN interfaces Its working for my wired but not counting for wireless. I have eth0, ATH0, and ATH1 on my LAN. eth1 is my WAN.
Disregard. I was connecting in a weird VPN loop by accident. Its all working now.
This is a great script, but I was curious if its counting loop back bytes. I mean, A device on my network seems to route to the modem and then back to the internal device. They are both wired devices, but I think the software on the machine is targeting the wan IP which then loops back in. If this is the case, is there a way to not count those values?
Well, I am utterly stuck, and hoping someone is able to help. I had been using the script successfully for a long time, but I upgrading my router this weekend. Running an Asus ac3100. Switched firmware to AsusWRT-Merlin because I need to be able to schedule jobs in cron for this script.
The problem is a little hard to identify because everything LOOKS right, it’s just that the packets and bytes are MUCH lower than expected.
LAN traffic is br0 (bridging vlan1 for ethernet ports, eth1 for 5GHz wifi, eth2 for 2.4GHz wifi)
WAN traffic is eth0. Note that I had to modify script to WAN_IFNAME=$(nvram get wan_ifnames) to get it to return a value. wan_ifname key didn’t exist.
Here’s some of my iptables.
admin@RT-AC3100-DC58:/tmp/home/root# iptables -L FORWARD -n -v
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
3 144 RRDIPT all — * * 0.0.0.0/0 0.0.0.0/0
3 144 RRDIPT2 all — * * 0.0.0.0/0 0.0.0.0/0
2359 172K ACCEPT all — * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 DROP all — !br0 eth0 0.0.0.0/0 0.0.0.0/0
78 3240 DROP all — * * 0.0.0.0/0 0.0.0.0/0 state INVALID
101 13986 ACCEPT all — br0 br0 0.0.0.0/0 0.0.0.0/0
704 66608 NSFW all — * * 0.0.0.0/0 0.0.0.0/0
52 2744 ACCEPT all — * * 0.0.0.0/0 0.0.0.0/0 ctstate DNAT
652 63864 ACCEPT all — br0 * 0.0.0.0/0 0.0.0.0/0
admin@RT-AC3100-DC58:/tmp/home/root# iptables -L RRDIPT2 -n -v
Chain RRDIPT2 (1 references)
pkts bytes target prot opt in out source destination
25 2760 RETURN all — br0 eth0 0.0.0.0/0 0.0.0.0/0
22 2660 RETURN all — eth0 br0 0.0.0.0/0 0.0.0.0/0
admin@RT-AC3100-DC58:/tmp/home/root# iptables -L RRDIPT -n -v
Chain RRDIPT (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all — * * 192.168.2.225 0.0.0.0/0
0 0 RETURN all — * * 0.0.0.0/0 192.168.2.225
4 336 RETURN all — * * 192.168.2.175 0.0.0.0/0
4 336 RETURN all — * * 0.0.0.0/0 192.168.2.175
1 52 RETURN all — * * 192.168.2.69 0.0.0.0/0
1 52 RETURN all — * * 0.0.0.0/0 192.168.2.69
8 576 RETURN all — * * 192.168.2.83 0.0.0.0/0
0 0 RETURN all — * * 0.0.0.0/0 192.168.2.83
….
Everything seems right, just the values are really small.
admin@RT-AC3100-DC58:/tmp/home/root# iptables -L RRDIPT -vnx -t filter
Chain RRDIPT (1 references)
pkts bytes target prot opt in out source destination
12 528 RETURN all — * * 192.168.2.225 0.0.0.0/0
8 368 RETURN all — * * 0.0.0.0/0 192.168.2.225
4 336 RETURN all — * * 192.168.2.175 0.0.0.0/0
4 336 RETURN all — * * 0.0.0.0/0 192.168.2.175
5 273 RETURN all — * * 192.168.2.69 0.0.0.0/0
2 88 RETURN all — * * 0.0.0.0/0 192.168.2.69
….
ANy clue on what else I can look at?
Thanks a ton – had been a really great script for Influxdb + Grafana!
milhouse
Are you saying that your numbers are low?
Well I can say that the script heavily depends on running trafficmon update every minute. If you changed your cron and somehow the script runs every 5 minutes or whatever, it will give lower numbers.
Well. When there is heavy traffic.. it seems to my capture some of it. Low traffic seems to be ok. heavy traffic my be 100 megabit or more sustained. The built in ddwrt counters however are correct. I was curious if the collection of the values followed by the clearing of the statistics was long enough in time to create missed bytes. I think my collection method is every second or every couple seconds. Maybe if I move that to once per minute it would be more accurate.
Working great here for a solid year. Question, though. I’ve recently started an OpenVPN server on my Asus-WRT router. That traffic seems to be on iface tap22 which is then bridged to vlan1 on br0.
This is now seen to be LAN traffic instead of WAN traffic, and invisible in my graphs. This can be significant traffic (visible in the router admin tool), and I want to monitor it. I modified the script to create VPN-LAN rules in RRDIPT2 chain
# Add the LAN->VPN and VPN->LAN rules to the RRDIPT2 chain
iptables -nvL RRDIPT2 | grep ${VPN_IFNAME}.*${LAN_IFNAME} >/dev/null
if [ $? -ne 0 ]; then
iptables -I RRDIPT2 -i ${VPN_IFNAME} -o ${LAN_IFNAME} -j RETURN
fi
iptables -nvL RRDIPT2 | grep ${LAN_IFNAME}.*${VPN_IFNAME} >/dev/null
if [ $? -ne 0 ]; then
iptables -I RRDIPT2 -i ${LAN_IFNAME} -o ${VPN_IFNAME} -j RETURN
fi
Then in the “update” processing I add a duplicate of the total traffic and assign to 00:00:00:00:00:01 in InfluxDB
IN=0
OUT=0
grep ${VPN_IFNAME} /tmp/global_$$.tmp | while read PKTS BYTES TARGET PROT OPT IFIN IFOUT SRC DST
do
if [ “${IFIN}” = “${VPN_IFNAME}” ]; then
IN=${BYTES}
fi
if [ “${IFIN}” = “${VPN_IFNAME}” ]; then
OUT=${BYTES}
fi
curl -is -XPOST “$DBURL/write?db=$DBNAME” –data-binary “hosts,mac=00:00:00:00:00:01 inBytes=${IN},outBytes=${OUT} ${CURDATE}000000000” >/dev/null 2>&1
done
So the global file looks right (4 records, two for WAN total and two for VPN totals) but the VPN traffic is always 0.
Do I need to go into EBTABLESinstead of IPTABLES to solve this?
I was probably overthinking. A simple bash script and pulling the transferred bytes from /sys/class/net/tap22/statistics/rx_bytes and /sys/class/net/tap22/statistics/tx_bytes worked well.
Make sure the files /jffs/bin/INPREV andf /jffs/bin/OUTPREV exist and have a baseline. Next run the script does simple subtraction and updates the current values into the previous stored files.
Note that a reverse the in/out for posting to influx. I did this to match the totals from 00:00:00:00:00:00. When someone outside my house vpns in, if that person downloads it’s an upload from my perspective (and my ISPs persepctive).
I store the values against mac 00:00:00:00:00:01 for the VPN.
#!/bin/sh
#
DBURL=http://192.168.2.52:8096
DBNAME=traffic
CURDATE=`date +%s`
# VPN processing
INCURR=$(cat /sys/class/net/tap22/statistics/rx_bytes)
OUTCURR=$(cat /sys/class/net/tap22/statistics/tx_bytes)
INPREV=$(cat /jffs/bin/INPREV)
OUTPREV=$(cat /jffs/bin/OUTPREV)
#echo “$INCURR is current IN”
#echo “$OUTCURR is current OUT”
#echo “$INPREV is previous IN”
#echo “$OUTPREV is previous OUT”
NEWIN=$(($INCURR-$INPREV))
NEWOUT=$(($OUTCURR-$OUTPREV))
#echo “Difference of IN is $NEWIN”
#echo “Difference of OUT is $NEWOUT”
echo “$INCURR” > /jffs/bin/INPREV
echo “$OUTCURR” > /jffs/bin/OUTPREV
curl -is -XPOST “$DBURL/write?db=$DBNAME” –data-binary “hosts,mac=00:00:00:00:00:01 inBytes=${NEWOUT},outBytes=${NEWIN} ${CURDATE}000000000” >/dev/null 2>&1
Here’s the global file:
Chain RRDIPT2 (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all — br0 tap22 0.0.0.0/0 0.0.0.0/0
0 0 RETURN all — tap22 br0 0.0.0.0/0 0.0.0.0/0
383 36489 RETURN all — br0 eth0 0.0.0.0/0 0.0.0.0/0
379 93711 RETURN all — eth0 br0 0.0.0.0/0 0.0.0.0/0
I believe I tried br0tap22 (LAN bridge to OpenVPN) as above, as well as vlan1tap22 (LAN to OpenVPN) and tap22eth0 (OpenVPN to WAN). All show 0 traffic even though ifconfig shows increasing traffic values on the tap22 interface.