This article is the logical conclusion of a small discussion with Comrade. vvzvlad , which unfolded under the topic "We clean the home Internet from very annoying ads (Ad's blocker for OpenWRT ) ", where the author using wget , sed and cron on OpenWRT successfully merges the files of the advertising hosts, parses and slips the dns server dnsmasq .

Redirecting a client when requesting an “advertising” domain, for example, on a loopback ( 127.0.0.1 - 127.255.255.255 ), instead of a quotation of an advertisement, the client will receive a beneficial “nothing” (of course, provided that the local web server does not work for us) ). The filtering mechanism is quite old and not without flaws. For example, you cannot specify host masks (* .ad-domain.tld) ​​or “cut out” ads whose banners are hosted on the requested resources. But it is not tied to any protocol or rather easy to use. Moreover, if you use it, for example, on your home or office router, which is used as a DNS server, the advertisement will be successfully cut on all gadgets, where the IP of our piece of iron is registered first as the DNS server.

But what if instead of a router with a custom firmware, we use ... MikroTik ( RouterOS ), the functionality of which imposes some limitations? Under the cat, you will learn how to successfully " convert " the hosts file into a suitable format, how to automate this business, and that a small service was created as a bonus for habraluy just to automate this process ( small, absolutely free and open source ) .

Customize MikroTik

Setup of the piece of iron will hardly cause any difficulties. We specify the IP of the router as the first DNS server in DHCP - “IP” → “DHCP Server” → “Networks” →% default config%. The first is to register the IP of MikroTik himself, click "OK":
')


The DNS server works out of the box, and you can even not change its stock settings:


The only interesting button for us is “Static” (“Static Routes”), in which we have to register “advertising” domains, indicating where such requests should be redirected.

Convert the hosts file

The hosts file has the format:

# Any comments 127.0.0.1 localhost 127.0.0.1 domain-a.tld 127.0.0.1 domain-b.tld 

Script format for MikroTik using static DNS routes:

 # Any comments /ip dns static add address=127.0.0.1 name=localhost add address=127.0.0.1 name=domain-a.tld add address=127.0.0.1 name=domain-b.tld 

To get the current lists of the first and convert them to the second, we will execute simple commands in a shell on, for example, the desktop ( bash ):

1. Download the lists and neatly add them under the names ./hosts_list.1 , ./hosts_list.2 etc:
   

    $ src=('http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext' 'https://adaway.org/hosts.txt'); i=0; for file in ${src[*]}; do i=$((i+1)); wget --no-check-certificate -O "./hosts_list.$i" "$file"; done; 

   
   
2. We get everything that starts at ' 127.0.0.1 ', delete comments, leave only domain names, remove duplicates, remove empty lines, and decorate each domain as a command to import:
   

    $ in="./hosts_list.*" && out="./adblock_dns.rsc" && host='127.0.0.1'; echo "/ip dns static" > $out && grep '127.0.0.1 ' $in | grep -v '^#' | cut -d' ' -f 2 | sort -u | grep . | sed "s/^/add address=$host name=/" >> $out && rm -f $in; wc -l $out; 

   
   

   Yes, it would be necessary to use a regular season and in general to redo everything, but the most important thing is that I conveyed my thought to you, then the matter is on

   
   

At the output we get the file adblock_dns.rsc , which is not superfluous to additionally check for the correctness of the contents.

Import to MikroTik

In order to import the resulting file, we cling to the router via ftp, fill in adblock_dns.rsc , and then cling to ssh or open a terminal in which we execute:

1. Make a backup ( it is better to save the resulting backup on your machine ):
   

    /system backup save 

   
   
2. If you have no important routes in the table, then we can bang all the available entries:
   

    /ip dns static remove [/ip dns static find] 

   
   
3. Import the downloaded file:
   

    /import adblock_dns.rsc 

   
   
4. We clean up after ourselves:
   

    /file remove adblock_dns.rsc 

   
   

Now you can ( optionally ) reboot MikroTik, check the correctness of obtaining DNS server addresses via DHCP and try to open any resource that was previously flooded with advertising - it should become significantly less.

It has been empirically proven that when importing ~ 5500 records of the Mikrotik hAP lite (650MHz @ RAM 32 Mb) it gets almost a stake when trying to open a table of static routes in the GUI. Reloading helps

Automation

One of the simplest automation options is a bash script that keeps cron up to date with adblock_dns.rsc , which is publicly available via, say, ftp. Its basis has already been described above, it remains only to issue it at your discretion ( a separate car is required ).

Another option is to use MetaRouter on the actual or neighboring MikroTik-e, on which the same OpenWRT is installed (we get rid of the need for a separate machine; this option is not detailed for lack of a decent MikroTik at hand ).

The second script ( on MikroTik ) takes it ( adblock_dns.rsc ), makes a backup and, if the file downloaded successfully, and the backup was saved successfully, it is imported, slamming all available routes beforehand ( attention ):

 :local hostScriptUrl "ftp://user:login@ftp_host:21/adblock_dns.rsc"; :local scriptName "adblock_dns.rsc"; :local backupFileName "before_stopad"; :local logPrefix "[StopAD]"; do { /tool fetch mode=ftp url=$hostScriptUrl dst-path=("./".$scriptName); :if ([:len [/file find name=$scriptName]] > 0) do={ /system backup save name=$backupFileName; :delay 1s; :if ([:len [/file find name=($backupFileName.".backup")]] > 0) do={ /ip dns static remove [/ip dns static find]; /import file-name=$scriptName; /file remove $scriptName; :log info "$logPrefix AD block script imported, backup file (\"$backupFileName.backup\") created"; } else={ :log warning "$logPrefix Backup file not created, importing AD block script stopped"; } } else={ :log warning "$logPrefix Backup file not downloaded, script stopped"; } } on-error={ :log warning "$logPrefix AD block script download FAILED"; }; 

Bonus for habrauda

And what to do if you have a MikroTik at your home / office / garage and you want to either just cut the advertisement, or automate the update of the hosts, without raising any metarowers and hosts with ftp files, so just without problems?
That's right, you need someone to pick it up for you. And as a bonus - such a thing has already been raised.

The meaning is as follows - you just go to the desired page, specify your settings and get the URL to which the script will be available with the settings you specified. The source script of the finished script for you will also be provided there. Opportunities that are currently implemented:

• Select from a list of public file hosts ( thanks to AdAway for Android );
• Want to indicate your sources? No problem! ( for example, you can specify three public host files and two of your own, located on the same gist.github.com , thus obtaining a convenient and convenient means of managing static routes on the ward routers );
• Configurable address ( IP v4 ) redirection;
• Indication of the limit ( limit ) of the number of records in the final script ( in order not to “accidentally” put a “weak” piece of iron );
• Specify your own routes ( they will simply be added first to the generated script );
• Ability to specify exceptions - the required domains will never be "blocked";
• Quite convenient generation of a ready script for MikroTik;

The parser is written in PHP ( requires php5-curl ), MIT license , source code is available at mikrotik-hosts-parser . How much service will live - I do not dare to think, but I hope that for a long time. The site, resistant to a habraeffekt, was kindly provided to comrade. drakmail , for which thanks to him and the cgood.ru team.

Reference: MikroTik DNS Stop AD

ATTENTION! If you previously used the script on the link above, and it suddenly stopped working for you, you know, the reason is in updating it. First of all, try everything in the same place just to get its updated edition, after that everything should work successfully. I apologize for any inconvenience!

Dicklamer: 1. The cgood.ru team is not related to the service. 2. It is not good to allow anyone to execute code on your routers. Therefore, I urge to use it solely for informational purposes. If you like everything, make a fork, set yourself up and use your health. For possible interruptions in work, habraeffekt or errors of responsibility no one bears. 3. I will make every effort so that everything works “as it should” as long as possible, but you need to understand the degree of responsibility.

If you find errors in the service or typing errors in the text - please report this to the PM. Thank you in advance!

UPDATE!

The script source is rewritten a little more than full. Now this is a full-fledged application on Lumen, and all the sources are in the repository.
"Stopad.generate.club" lay down forever, unfortunately. But it is not all that bad! Two mirrors are raised (one is the “cgood.ru” team; quite powerful), the second is on my modest VDS-ke. The one that mine is true is more than modest, use it with caution.

Link :
The new address (from the team cgood.ru): stopad.cgood.ru
Reserve area: stopad.kplus.pro