📜 ⬆️ ⬇️

Accounting for Internet traffic tenants (netflow)

On the issue of statistics in general, and netflow in particular, there are quite a few rather extensive articles (the study of which requires a decent time). Here I will give my as-is scripts for a not very complicated TK for collecting and sending monthly statistics to tenants.


Several tenants get the Internet through NAT (NAT itself is tied through policy-routing to two providers). Each tenant has his own Wilan. Wilan "unfolds" into the network of the tenant through a managed switch. Each tenant is given its own local network segment with the prohibition to go to neighboring segments.

Objective : to tell each tenant and building manager at the end of the month how much he ate the Internet and stick to those users who have reused too much traffic. Those. we need: aggregate statistics for the segment, Internet consumption by each address (destination) in the network under consideration.
Tools: use netflow on Ziska, flow-tools on Linux, MTA, cron.


The router combines information about similar packets into flows (flows). Packages with the same src / dst IP / port combination are considered similar. Those. The ip-packages of the file downloaded from the file dump will be combined into one flow. Information about each flow includes from whom to whom (4 fields, address and port), the number of packets, the interface through which the packet was sent, the size of the transmitted data and the time (timestamp). These data are sent periodically in udp-packages to the collector - the server, ready to accept this data. It is clear that the collector can collect data from several senders, so when you save the data, the sender is usually also written there. It should be understood that statistics is collected on all ports where it is said to collect it. If you say “collect everything on all ports”, then the traffic will be counted twice - at the time of reception (as inbound) and at the time of sending to the client interface (as outgoing). Fortunately, nating occurs between receiving and sending, so with NAT, we will see internal addresses in the flows for only one (internal) interface.

Router configuration

I will not quote it in its entirety, show only the part related to netflow (i.e., the ACLs, NAT, DHCP are omitted for those who asked for it, etc.), besides, only one tenant interface is shown (the rest is strictly by analogy) :
 interface Vlan31
  description RENT: OOO AVTOTRADE room 315,316,317,318
  ip address
  ip flow ingress
  ip flow egress
  ip nat inside

 ip flow-export source Vlan255
 ip flow-export version 5
 ip flow-export destination 3000

As is clear, vlan31 (192.168.31 / 24) are tenants, similarly, the other tenants. Vlan255 - Vlan, through which to send data (service). The remaining values ​​are clear. (note the netflow version, flow-tools does not support version 9).

Configuration flow-tools


It needs to be installed (for debian: aptitude install flow-tools ). Settings in /etc/flow-tools/capture.conf:

 -w / srv / netflow / 0/0/3000

Note: netflow is eating a LOT of space. Get ready for dozens of gigabytes of data that you also need to store. It is better to put this data on a separate section.

Statistics processing

Actually, this is the topic of the article. We need to configure, firstly, the filters by which we will select information, and secondly, the script that will send this information to anyone. At the moment there are not very many tenants, so for everyone everything is configured separately.


Filters are needed for the flow-nfilter program, which accepts a binary flow at the input, filters it and outputs a binary flow again.

Filters in /etc/flow-tools/cfg/filter.cfg. They consist of primitives (roughly speaking, names for filter elements) and filters. When calling flow-nfilter, it specifies the name of the filter by which to filter.

Here is an example for two tenants:

 filter-primitive rent31-range
   type ip-address-prefix
   default deny

 filter-primitive rent3233-range
   type ip-address-prefix
   default deny

 filter-definition rent31-out
   match dst-ip-addr rent31-range

 filter-definition rent3233-out
   match dst-ip-addr ren32t33-range


Actually, the main thing:

 #! / bin / sh
 # $ 1 - rent (01,02, ...), 
 # $ 2 ... $ N - email adresses to be sent
 year = `date --date = '- 1 month' +% Y`
 month = `date --date = '- 1 month' +% m`
 net = $ 1
 total = `flow-cat / srv / netflow / $ year / $ year- $ month | flow-nfilter -F rent $ net-out | flow-stat -f 15 | grep -v" # "| awk '{print $ 3 } '`
 (echo "Total incoming: $ total Mb"; flow-cat / srv / netflow / $ year / $ year- $ month | flow-nfilter -F rent $ net-out | flow-stat -f 8) | mail -s "report for net 192.168. $ net.0 / 24 for $ year- $ month" $ *

Key points: the script sends statistics for the past month. (thanks to the corresponding date option). Flow is processed twice (I’m satisfied with the speed so far, if it starts to slow down, I’ll need to filter the file and run the two reports separately). The first processing gives us a human-readable number in megabytes, the second - a sign about who eats so much.

If necessary (conflict / dispute), it will be possible to manually (with the help of flow-print and grep) give out full statistics for each chain.


I use cron features in debian, it's a bit more convenient than usual in unix - it has the /etc/cron.monthly directory. It contains the /etc/cron.monthly/rent-acc file:
 / usr / local / bin / rent-acc 31 admin@domain.ru rentuser@example.com manager@company.ru
 / usr / local / bin / rent-acc 3233 admin@domain.ru rentuser2@foobar.baz manager@company.ru

This file, at the same time, is also a configuration file for tenants (however, filters for new ones need to be manually registered!). Do not forget about chmod + x to this file.


It looks like this:
 Total incoming: 67673.187 Mb

 # --- ---- ---- Report Information --- --- ---
 # Fields: Total
 # Symbols: Disabled
 # Sorting: None
 # Name: Destination IP
 # Args: flow-stat -f 8 
 # IPaddr flows octets packets
 # 1321968 4301821906 24858456 2118220 62029649663 51512160 226383 193301694 1129250 105319 350858966 3027875 3240 60888068 681056 31289 13434866 217217 200 32104404 97818 1356 137727274 1977666 35310 553217946 10290038


Please note that the provision of communication services requires a license. The only loophole is re-invoice from the provider for services. That is why the statistics should be considered. However, all of the above is not a certified billing, i.e. requires some kindness on the part of the payer.

Source: https://habr.com/ru/post/101819/

All Articles