SOC for beginners. Chapter 3. Using external sources of threat data for the Security Operation Center
In previous articles of the SOC for beginners, we described how it is arranged and how to organize basic monitoring of incidents and monitoring infrastructure security . Today we will talk about Threat Intelligence - the use of external sources of data on threats.
For all the seeming simplicity, launching work with Threat Intelligence is almost the longest and most painful process. The only exception is probably those cases when you have reference OS images with workstations enabled on your workstations, users are not administrators, and Internet access is whitelisted. Unfortunately, we have not yet met such companies for the whole time. In this regard, all interested in the topic of Threat Intelligence - welcome under cat.
In the case of a global, complex and terrible attack, it is always sad if you are in the ranks of the first victims. Antiviruses have no signatures yet, SZI has blacklists, MSSP providers and SOCs have indicators and rules that could help in detection. In this case, the customer is actually forced to fight the attack on his own (perhaps not without buying expensive incident investigation services).
')
Threat Intelligence, in general, allows the “second victims” to get ready to repel an attack. Of course, this is not just another silver bullet, TI will not save from all attacks. But with a well-constructed process and correctly selected sources of knowledge, it can often help.
There are a lot of articles already written about what Threat Intelligence is, what “good” should consist of and what “bad” indicators look like, so let's focus on what these indicators are and how to work with them in each case.
Feeds (they are feeds)
Probably the most voluminous and frequently occurring information available to most in both paid and free versions. As a rule, these are constantly updated by the vendor for malicious addresses, urls, e-mails, etc.
The most important thing here is not to start tightening all existing black lists in SOC. When analyzing the quality of feeds, we recommend focusing your attention on the presence of extended information on threats. Example below:
IOC (Indicators of Compromise, Indicators of Compromise)
Usually is part of a summary report on the analysis of a particular threat. And this means that each of the indicators can and should be considered not separately, but together with other indicators from the corresponding report. In addition, since there is a certain report and analytics, in addition to the indicator, we also get the context.
Vulnerabilities
Detailed descriptions of vulnerabilities and their ranking by severity, taking into account the possibility of exploitation in your infrastructure, allows you to get information about chains of events and indicators that you can use to identify such exploitation attempts in real life.
Attack scenarios
Attack scripts, as a rule, contain examples of the sequence of actions of the attacker to advance the system. The main body of such TI knowledge is conducted by penetration tests at the customer and vendor reports on APT attacks. By obtaining and analyzing this information, you can compare your current detection scenarios with the attack phases described in the reports.
From here two conclusions suggest themselves:
Users
Security Awareness works wonders. Phishing, social engineering - with the right approach, people will give you as many samples as no sandbox can catch.
Further, when speaking about IoC / indicators, we will mean that in addition to the indicator itself, we have at least minimal information about what malware / threat it relates to.
So, as a rule, indicators can be divided into several groups:
In each group, indicators, as a rule, give a different degree of reliability of the result when they are detected. And the more indicators you have on the same threat, the more chances to filter out false positives.
But can one unconditionally trust information coming from any of the Threat Intelligence subscriptions? After all, this is usually the analysis of quite specific samples of HVO, made by truly qualified specialists. We return to the examples.
The conclusion is, in fact, simple: the information supplied by vendors is still too general. Depending on the context and features of the infrastructure of each individual customer, a mechanism will be needed to filter out a portion of the information received.
As a result, if you start building work with Threat Intelligence, you will need to consider the following points:
Setting up logging and selecting event sources
Try to determine what types of indicators you can use and how you will check them. For example, you have information on MD5. Which SZI / logs in your infrastructure may contain relevant information?
From our experience:
As a result, a table will appear with a description of which types of indicators on which sources will be detected. For example, this:
Separately, it is worth noting that when choosing sources of events it is important to correctly evaluate the completeness of information. For example, if SSL is not enabled on the proxy server, then the ability to analyze indicators by URLs will be severely limited, and this should be taken into account. For example:
Response capabilities
Try to write on paper for the beginning an approximate sequence of actions when each indicator is detected. Here is a rough plan:
Infrastructure coverage
As a rule, in order to enrich events with external data is really effective and does more good than harm, you need to be able to connect network indicators with the host ones, and various indicators from the same Threat Intelligence report - among themselves. For example, when detecting an appeal to a malicious domain (or even url), check on the AWP / server for the presence of host indicators (files, registry branches, MD5, services) that are related to the corresponding threat. This can be done manually (by connecting to the host), or automated: through a script or security scanner.
Ways of working and pollinating knowledge
Okay. Sources hooked, data is collected. But how to work with it?
First, we add feeds and indicators of compromise for any new (or not so) threat in real-time monitoring. At the same time, we do not forget to first check the relevance of these indicators as described above, in order to exclude flood and irrelevant indicators. In an amicable way, this should be a continuous process. Sources of such indicators, in addition to paid subscriptions and services of specialized companies, may be, for example, publications of leading specialized media, newsletters of leading manufacturers of protective equipment and research groups, analysis of malicious mailings that are sent to company employees. They can enter the information security service both from vigilant users and climb from antispam / mail sandboxes.
Secondly, we conduct a retrospective search for these indicators in order to identify the facts of infection of the infrastructure in the past (for example, over a month). The search can be carried out by event logs (network indicators, email, processes), as well as security scanners or scripts (host indicators: files, registry, MD5). Through the scanner, you can immediately collect information on the vulnerabilities available on the host.
Thirdly, we send indicators of compromise for blocking to IT or to the information security administrators of the appropriate security tools.
Fourth, we test the detectability of the obtained indicators in your infrastructure. An important stage in working with various malware and attacks analyzes is the reproduction of at least some of these attacks in order to test the operation of their content and protection tools in the “test environment”.
Threat Intelligence is undoubtedly an important part of any SOC, since allows you to more quickly respond to external threats. But you need to carefully select IOC / TI suppliers and clearly understand how to work with this information within a particular company, taking into account both the available human resources and the existing technical means to search for evidence of the manifestation of these indicators in the infrastructure. Otherwise, the whole process may turn into one nightmare or, at best, simply not produce results.
For all the seeming simplicity, launching work with Threat Intelligence is almost the longest and most painful process. The only exception is probably those cases when you have reference OS images with workstations enabled on your workstations, users are not administrators, and Internet access is whitelisted. Unfortunately, we have not yet met such companies for the whole time. In this regard, all interested in the topic of Threat Intelligence - welcome under cat.
In the case of a global, complex and terrible attack, it is always sad if you are in the ranks of the first victims. Antiviruses have no signatures yet, SZI has blacklists, MSSP providers and SOCs have indicators and rules that could help in detection. In this case, the customer is actually forced to fight the attack on his own (perhaps not without buying expensive incident investigation services).
')
Threat Intelligence, in general, allows the “second victims” to get ready to repel an attack. Of course, this is not just another silver bullet, TI will not save from all attacks. But with a well-constructed process and correctly selected sources of knowledge, it can often help.
Threat Intelligence Structure
There are a lot of articles already written about what Threat Intelligence is, what “good” should consist of and what “bad” indicators look like, so let's focus on what these indicators are and how to work with them in each case.
Feeds (they are feeds)
Probably the most voluminous and frequently occurring information available to most in both paid and free versions. As a rule, these are constantly updated by the vendor for malicious addresses, urls, e-mails, etc.
The most important thing here is not to start tightening all existing black lists in SOC. When analyzing the quality of feeds, we recommend focusing your attention on the presence of extended information on threats. Example below:
List of exit-node TOR:
torstatus.blutmagie.de - the feed source itself contains only TOR nodes + there is information on exit-node ports + status information (offline / online).
panwdbl.appspot.com/lists/ettor.txt - and here we only have a list of addresses (although this is not an official upload, but not the essence).
In the first case, we can configure a narrower and more correct rule. In the second, most likely, the rules will give a greater number of false positives.
IOC (Indicators of Compromise, Indicators of Compromise)
Usually is part of a summary report on the analysis of a particular threat. And this means that each of the indicators can and should be considered not separately, but together with other indicators from the corresponding report. In addition, since there is a certain report and analytics, in addition to the indicator, we also get the context.
Vulnerabilities
Detailed descriptions of vulnerabilities and their ranking by severity, taking into account the possibility of exploitation in your infrastructure, allows you to get information about chains of events and indicators that you can use to identify such exploitation attempts in real life.
Attack scenarios
Attack scripts, as a rule, contain examples of the sequence of actions of the attacker to advance the system. The main body of such TI knowledge is conducted by penetration tests at the customer and vendor reports on APT attacks. By obtaining and analyzing this information, you can compare your current detection scenarios with the attack phases described in the reports.
For example, recently the number of encrypters has greatly increased. In this case, most of them are delivered in the same old way - through mailing lists with a dropper file in the attachment. And despite all the “new” methods of using the same Word to execute third-party scripts, the behavior in the OS is the same for everyone.
From here two conclusions suggest themselves:
- We must pay attention to the antispam. As our practice shows, a very large number of customers do not have even the simplest checks for spoofing sender addresses.
- It is necessary on a regular basis to raise awareness of employees about new security threats and rules for the safe use of external services and internal resources of the company itself (Internet, mail, its AWP, etc.).
Users
Security Awareness works wonders. Phishing, social engineering - with the right approach, people will give you as many samples as no sandbox can catch.
Further, when speaking about IoC / indicators, we will mean that in addition to the indicator itself, we have at least minimal information about what malware / threat it relates to.
So, as a rule, indicators can be divided into several groups:
- Network indicators (ip, fqdn, url, email, port, etc.).
- Host indicators (file / process / service names, MD5 sums, descriptions of executed commands, registry key values, user / group names, etc.).
- Other (informative) indicators. For example, sometimes there are alerts from the “we notify you that during the period from X to Y you are scheduled to have a DDoS attack” or “the subject of emails in one form or another contain the words“ tax return ”.
In each group, indicators, as a rule, give a different degree of reliability of the result when they are detected. And the more indicators you have on the same threat, the more chances to filter out false positives.
Top issues with Threat Intelligence
But can one unconditionally trust information coming from any of the Threat Intelligence subscriptions? After all, this is usually the analysis of quite specific samples of HVO, made by truly qualified specialists. We return to the examples.
- The lack of complete information on the threat.
Let's imagine that from the indicators you only have the IP addresses of the control centers of Malvari. On these addresses there can be far more than one domain. And you have Skype / torrents enabled or some other p2p application. An example of testing indicators from one of the vendors:- The use of legitimate software by cybercriminals.
According to one of the reports of Kaspersky Lab, attackers use another quite legitimate tool for remote access, winexec, for their own purposes. And everything would be fine, but you, let's say, have an IMS, which for the inventory of hosts uses exactly the same tool. And the indicators for installing the winexesvc services and running the corresponding processes and detecting MD5 sums will give you hundreds of hits across the entire infrastructure.
The conclusion is, in fact, simple: the information supplied by vendors is still too general. Depending on the context and features of the infrastructure of each individual customer, a mechanism will be needed to filter out a portion of the information received.
As a result, if you start building work with Threat Intelligence, you will need to consider the following points:
- Ideally, before making a decision on the use of certain IOC / TI databases / suppliers in your work, it is advisable to conduct their piloting in your infrastructure - so to speak, try it on "live". For example, for a couple of months, start them up and perform an analysis of the positives, assessing the percentage of confirmed.
- After selecting IOC / TI providers, you need to rank them by relevance. In this case, depending on the source and even the type of indicator, the reaction to alerts will differ. For trusted sources / highly relevant indicators - start the response process. For the rest - collecting statistics and alerts when thresholds are exceeded.
- Before launching indicators in real-time monitoring, it is best to check the information received. This can be a quick check on historical data in order to exclude obvious falls, as well as a check of the relevance of indicators. For example, checking the IP address for belonging to a hosting (if not among indicators of a specific url or domain) or checking the MD5 hash for belonging to standard utilities, system files or administration tools.
- It is necessary to immediately identify scenarios for responding to the identification of certain indicators in the company's infrastructure in order to clearly know how to act in each case — what other information to collect and analyze, where to get it, etc. This will significantly reduce the response time.
How to start the process of working with Threat Intelligence
Setting up logging and selecting event sources
Try to determine what types of indicators you can use and how you will check them. For example, you have information on MD5. Which SZI / logs in your infrastructure may contain relevant information?
From our experience:
- Kaspersky (and indeed any antivirus) stores information about all executable files on each host (but only about the first launch).
- DLP / Antispam can be configured to log MD5 sums of all email attachments.
- Sysmon logs MD5 sums of all running processes.
As a result, a table will appear with a description of which types of indicators on which sources will be detected. For example, this:
Separately, it is worth noting that when choosing sources of events it is important to correctly evaluate the completeness of information. For example, if SSL is not enabled on the proxy server, then the ability to analyze indicators by URLs will be severely limited, and this should be taken into account. For example:
The malicious code is an obfuscated JavaScript file “List of documents for the Federal Tax Service of January 20, 2015.docx_I quarter 2015g. Signed by the head __ TESTED Dr.Web_ef73f6db_ls.js ”, located at hxxps: //yadi.sk/d/AXf0WGaqBpXh.
Without SSL Inspection enabled, we get a response to each input on yadi.sk:
If it’s possible to parse SSL, we can only respond to very specific links, and therefore use the URL in correlation:
Response capabilities
Try to write on paper for the beginning an approximate sequence of actions when each indicator is detected. Here is a rough plan:
The network indicator worked (for example, BadRabbit: 1dnscontrol.com/flash_install.php). What will you do next?
- How will you determine the host?
- Can you check the process that initiated this connection?
- Will you check the host indicators (you have them)?
- Do you have access to all the hosts in the infrastructure to do an automatic check on each trigger?
- Do you understand which OS is on which host, and are the indicators from the report applicable to this system?
- Do you have the ability to isolate the user's machine at any time?
- Is it possible to automate all the activities listed above?
Infrastructure coverage
As a rule, in order to enrich events with external data is really effective and does more good than harm, you need to be able to connect network indicators with the host ones, and various indicators from the same Threat Intelligence report - among themselves. For example, when detecting an appeal to a malicious domain (or even url), check on the AWP / server for the presence of host indicators (files, registry branches, MD5, services) that are related to the corresponding threat. This can be done manually (by connecting to the host), or automated: through a script or security scanner.
Ways of working and pollinating knowledge
Okay. Sources hooked, data is collected. But how to work with it?
First, we add feeds and indicators of compromise for any new (or not so) threat in real-time monitoring. At the same time, we do not forget to first check the relevance of these indicators as described above, in order to exclude flood and irrelevant indicators. In an amicable way, this should be a continuous process. Sources of such indicators, in addition to paid subscriptions and services of specialized companies, may be, for example, publications of leading specialized media, newsletters of leading manufacturers of protective equipment and research groups, analysis of malicious mailings that are sent to company employees. They can enter the information security service both from vigilant users and climb from antispam / mail sandboxes.
Secondly, we conduct a retrospective search for these indicators in order to identify the facts of infection of the infrastructure in the past (for example, over a month). The search can be carried out by event logs (network indicators, email, processes), as well as security scanners or scripts (host indicators: files, registry, MD5). Through the scanner, you can immediately collect information on the vulnerabilities available on the host.
Thirdly, we send indicators of compromise for blocking to IT or to the information security administrators of the appropriate security tools.
Fourth, we test the detectability of the obtained indicators in your infrastructure. An important stage in working with various malware and attacks analyzes is the reproduction of at least some of these attacks in order to test the operation of their content and protection tools in the “test environment”.
Threat Intelligence is undoubtedly an important part of any SOC, since allows you to more quickly respond to external threats. But you need to carefully select IOC / TI suppliers and clearly understand how to work with this information within a particular company, taking into account both the available human resources and the existing technical means to search for evidence of the manifestation of these indicators in the infrastructure. Otherwise, the whole process may turn into one nightmare or, at best, simply not produce results.
Source: https://habr.com/ru/post/343574/
All Articles