History of visiting sites through Mikrotik router logs
In my work, I often have to use these routers for solving my tasks. In this case, it was necessary to provide access to the Internet through the router RB450G, having the ability to block certain sites by the name mask and save the history of visits. Next, an example of solving such a problem using free software will be shown.
For a start, it was decided to configure a transparent proxy. The router has its own Web-proxy, to make it transparent we execute according to the example in the documentation
code> ip firewall nat add in-interface = ether1 dst-port = 80 protocol = tcp action = redirect to-ports = 8080 chain = dstnat
ip proxy set enabled = yes port = 8080
that sends requests to the 80 (HTTP) port on the port of the proxy server 8080. Now you can add blocked addresses, for example,
')
In this case, requests containing the word “facebook” will be blocked and redirected to the internal page 192.168.0.1/404 (which is of course optional). After the colon in the dst-host parameter, regular expressions can also be used.
Then the question arose of how to actually accumulate and process the log of visits. There was no built-in or other product from the manufacturer. After reading the documentation and searching the thematic forums, the Webproxy-log product was found. The product is somewhat clumsy, but prompting direction (although for a small load it will also fit perfectly). How it works:
1. In the logging destination setting, a section is added to transfer to the syslog server.
192.168.0.1 - the address of the syslog server to which we will send the log. 192.168.0.3 - the internal address of the router.
2. Create a section of the journal itself that will use the created destination and send proxy server logs there.
.
At this stage, the router will send logs like
where from the prefix web-proxy, account the address of the user who sent the request will be recorded, and the request itself.
3. As a server, you can use the above product , which consists of two parts:
WebProxy Log Catcher - an application (as a service is not installed) a simple syslog server itself that collects logs for further processing and adds time stamps.
WebProxy Log is a log viewing interface that each time it starts importing the accumulated logs into the local database.
Setting up the application does not cause difficulties and is sufficiently described in the documentation.
Taking into account the simple format of the logs, it is not difficult to write your own log analyzer sent by the router and later a small service with a web interface was written for your own purposes, using Delphi and ICS components, which save logs to the database based on SQL server express.
Perhaps all of the above will help you save time in solving such problems and will introduce you a bit to the capabilities of the routers of this manufacturer.
For a start, it was decided to configure a transparent proxy. The router has its own Web-proxy, to make it transparent we execute according to the example in the documentation
code> ip firewall nat add in-interface = ether1 dst-port = 80 protocol = tcp action = redirect to-ports = 8080 chain = dstnat
ip proxy set enabled = yes port = 8080
that sends requests to the 80 (HTTP) port on the port of the proxy server 8080. Now you can add blocked addresses, for example,
')
/ip proxy access add action=deny redirect-to=192.168.0.1/404 dst-host=:facebookIn this case, requests containing the word “facebook” will be blocked and redirected to the internal page 192.168.0.1/404 (which is of course optional). After the colon in the dst-host parameter, regular expressions can also be used.
Then the question arose of how to actually accumulate and process the log of visits. There was no built-in or other product from the manufacturer. After reading the documentation and searching the thematic forums, the Webproxy-log product was found. The product is somewhat clumsy, but prompting direction (although for a small load it will also fit perfectly). How it works:
1. In the logging destination setting, a section is added to transfer to the syslog server.
/system logging action add name=proxylog target=remote remote=192.168.0.1 src-address=192.168.0.3
192.168.0.1 - the address of the syslog server to which we will send the log. 192.168.0.3 - the internal address of the router.
2. Create a section of the journal itself that will use the created destination and send proxy server logs there.
.
/system logging add topics=web-proxy action=proxylogAt this stage, the router will send logs like
web-proxy, account 192.168.0.59 GET imgcdn.ptvcdn.net/pan_img/appDownload/PandoraService/Service_Info .
xml action = allow cache = MISS
web-proxy, debug GET /pan_img/appDownload/PandoraService/Service_Info.xml HTTP / 1.1
debug Cache-control web-proxy: no-cache
debug Pragma web-proxy: no-cache
web-proxy, debug Host: imgcdn.ptvcdn.net
web-proxy, debug Accept: text / html, * / *
web-proxy, debug Accept-Encoding: identity
web-proxy, debug User-Agent: Mozilla / 3.0 (compatible; Indy Library)
web-proxy, debug X-Proxy-ID: 1074695054
web-proxy, debug X-Forwarded-For: 192.168.0.59
debug Via web-proxy: 1.1 192.168.0.3 (Mikrotik HttpProxy)
web-proxy, debug
where from the prefix web-proxy, account the address of the user who sent the request will be recorded, and the request itself.
3. As a server, you can use the above product , which consists of two parts:
WebProxy Log Catcher - an application (as a service is not installed) a simple syslog server itself that collects logs for further processing and adds time stamps.
WebProxy Log is a log viewing interface that each time it starts importing the accumulated logs into the local database.
Setting up the application does not cause difficulties and is sufficiently described in the documentation.
Taking into account the simple format of the logs, it is not difficult to write your own log analyzer sent by the router and later a small service with a web interface was written for your own purposes, using Delphi and ICS components, which save logs to the database based on SQL server express.
Perhaps all of the above will help you save time in solving such problems and will introduce you a bit to the capabilities of the routers of this manufacturer.
Source: https://habr.com/ru/post/179233/
All Articles