Successful implementation of SIEM. Part 1
Hello.
It so happened that the last few years spent in the process of implementing SIEM Arcsight ESM in one major telecommunications provider. I think that I did it very successfully and eventually went on a slightly different path, because ran into some ceiling, which is in Russia as a whole in the direction of security. In order to avoid a false understanding of the reader, I’ll clarify that I was working inside the company, and not on a project from a system integrator, and SIEM was used exclusively for internal needs, and not to create a commercial Security Operation Center (hereinafter SOC). In general, the subject of implementation and use of SIEM is poorly covered, and in general, those installations that I saw usually did not bring sufficient effectiveness to those who implemented it. In my opinion, my colleagues and I managed to implement my own SOC as the core, which is SIEM, which is of great benefit not only to the information security department, but also to other departments in particular and the company as a whole, including management.
This article will be useful for IS managers who are thinking about introducing SIEM and IS specialists who are involved in the operation and development of SIEM in their company. The article will consist of several parts, because I will talk a lot about it :). In the first part of the article we will talk about the choice of iron for the SIEM and the effective collection of logs, I will talk a little about the features of my infrastructure, in the second part I will talk about the correlation and visualization of logs, what it is, why it is necessary, when it is effective and when it is not very the third part, we discuss how to properly administer the SIEM and monitor its health, what processes can be tied to the SIEM, how to do it, what personnel are needed, how the incident investigation process went with us, what benefits does the company as a whole and the leadership in particular from SIEM in practice, and not in marketing presentations of salespeople from integrators.
Go.
')
I'll start by describing the features of my architecture. Its main feature was that it is very large and geographically distributed, but at the same time there were no user workstations as a class, as well as the protected infrastructure was constantly growing. Consider this when building your SIEM architecture.
We turn to the architecture of the SIEM. The first thing you need to understand here is that it does not matter what kind of SIEM you implement and how many servers you need for it and what are the hardware requirements, depending on the SIEM, a different number of servers may be required. So for example with Arcsight we used separate servers on which we installed software for collecting logs, and the database and log correlator were on other servers. The main point in the architecture of the SIEM is a data storage system (DSS). What I immediately want to say is that not a single integrator can count how many logs you really have, don’t trust anyone here, consider yourself, look at the limitations of the SIEM on the maximum storage size and dance from it. Here, the more space the better, you also need fast disks, preferably a direct connection of the storage system with the SIEM. It is also necessary to connect to the backup system (RMS). In my opinion, the best time to store logs for a hot one month is at least six months. The main problems that can be expected of you here are an emphasis on the speed of reading and writing, which will greatly affect how quickly you can upload the logs you need and when several people work with SIEM. Therefore, the faster the storage system the better. This is how smoothly we approached the second question of the first part of the article, namely, the management of logs.
Log management in the terminology of Arcsight SIEM, yes I think, and in general, any SIEM is very convenient to be divided into the following stages: log acquisition (gathering), event normalization, event aggregation, event filtering, event categorization, event prioritization. In Arcsight, all these tasks are performed by connectors, for some of the event sources are delivered out of the box, some need to be written by yourself.
Features of logging - the main feature is that you need to plan the load on the network, especially when using SYSLOG, because Spam is possible and you need to work with networkers to think about the best way to send logs.
Event normalization - log parsing, i.e. bringing the received event into a view that is understandable to a person and a database that spins under the hood of SIEM. If in more detail we take an event, we beat it into parts and we throw it in a cell of the table of the SIEM database, i.e. The address of the event source flies to the database event source column. In general, all events should be normalized.
Event aggregation is a very interesting and useful point (not all SIEMs support this function), which allows reducing the amount of garbage in the logs. Aggregation is the collapse of several identical events into one, for example, we have 30 calls in 30 seconds from one IP address to our server, which our border firewall recorded, in the database it will be 30 lines, aggregation is needed to make 30 of these only one event. Aggregation is most effective for logs of firewalls, netflow, IDS / IPS, web servers.
Filtering events - throwing away what is not needed or leaving only what is needed. Very useful thing especially for logs from operating systems, antiviruses, IDS / IPS. What is most effective to do to determine that the event is of no interest to you. You need to write a rule that will write to the list (in the terminology of Arcsight, I don’t know if the competitors have such a function) or build a report on all the events that occurred during the week from each source and analyze it from the point of view of informativeness. The result of this analysis is a change in the logging level at the event source, or setting up filters on the SIEM log collector. Looking ahead, I’ll say that it’s better not to filter the firewalls, since They are very useful in investigating incidents.
Event categorization is the assignment of events of the same type from different sources of the same category in order to be more convenient to handle. I’ll say right away that the thing looks more promising than it really is if you have a lot of different sources and you just don’t have time to set up the categorization correctly. For those whose infrastructure is growing slowly, I think it will be useful.
Prioritization - setting the event priority within the SIEM. For example, we received an event with severity CRIT, but we know that for IS it did not criticize and set it an INFO.
Let's move on a bit to practice and talk about the most informative sources of logs.
My undisputed leader is vulnerability scanners that allow you to make an inventory, a list of open ports, services and vulnerabilities on the network. Here I would recommend using several tools at once and collecting everything into sheets or reports, and then tying the whole thing to ITIL, i.e. create internal tickets and close problems according to internal IS policies. Actually SIEM here is a tool on which all scanners have thrown off the information and the analyst is already analyzing it by looking at all the reports in only one place. Also here I would refer samopisnye scripts that can collect information from DNSBL, C & C servers from the Internet.
The second place for me is occupied by firewalls, netflow, VPN gateway and IPS / IDS / WAF, which allow detecting all sorts of scans, attempts of DDoS attacks, other network security incidents, including. internal users and to optimize the work of protection.
The third place is operating systems, from the logs of operating systems it can be understood that we were hacked or the admin is naughty.
Fourth place - the database, on their requests, you can also see attempts to divert important information from us.
Fifth place - antivirus.
I think on this the first part can be finished. Wait for the second one, in which I will tell you what is best to visualize, what to correlate with, what to notify, and what you can react with with scripts and why.
It so happened that the last few years spent in the process of implementing SIEM Arcsight ESM in one major telecommunications provider. I think that I did it very successfully and eventually went on a slightly different path, because ran into some ceiling, which is in Russia as a whole in the direction of security. In order to avoid a false understanding of the reader, I’ll clarify that I was working inside the company, and not on a project from a system integrator, and SIEM was used exclusively for internal needs, and not to create a commercial Security Operation Center (hereinafter SOC). In general, the subject of implementation and use of SIEM is poorly covered, and in general, those installations that I saw usually did not bring sufficient effectiveness to those who implemented it. In my opinion, my colleagues and I managed to implement my own SOC as the core, which is SIEM, which is of great benefit not only to the information security department, but also to other departments in particular and the company as a whole, including management.
This article will be useful for IS managers who are thinking about introducing SIEM and IS specialists who are involved in the operation and development of SIEM in their company. The article will consist of several parts, because I will talk a lot about it :). In the first part of the article we will talk about the choice of iron for the SIEM and the effective collection of logs, I will talk a little about the features of my infrastructure, in the second part I will talk about the correlation and visualization of logs, what it is, why it is necessary, when it is effective and when it is not very the third part, we discuss how to properly administer the SIEM and monitor its health, what processes can be tied to the SIEM, how to do it, what personnel are needed, how the incident investigation process went with us, what benefits does the company as a whole and the leadership in particular from SIEM in practice, and not in marketing presentations of salespeople from integrators.
Go.
')
I'll start by describing the features of my architecture. Its main feature was that it is very large and geographically distributed, but at the same time there were no user workstations as a class, as well as the protected infrastructure was constantly growing. Consider this when building your SIEM architecture.
We turn to the architecture of the SIEM. The first thing you need to understand here is that it does not matter what kind of SIEM you implement and how many servers you need for it and what are the hardware requirements, depending on the SIEM, a different number of servers may be required. So for example with Arcsight we used separate servers on which we installed software for collecting logs, and the database and log correlator were on other servers. The main point in the architecture of the SIEM is a data storage system (DSS). What I immediately want to say is that not a single integrator can count how many logs you really have, don’t trust anyone here, consider yourself, look at the limitations of the SIEM on the maximum storage size and dance from it. Here, the more space the better, you also need fast disks, preferably a direct connection of the storage system with the SIEM. It is also necessary to connect to the backup system (RMS). In my opinion, the best time to store logs for a hot one month is at least six months. The main problems that can be expected of you here are an emphasis on the speed of reading and writing, which will greatly affect how quickly you can upload the logs you need and when several people work with SIEM. Therefore, the faster the storage system the better. This is how smoothly we approached the second question of the first part of the article, namely, the management of logs.
Log management in the terminology of Arcsight SIEM, yes I think, and in general, any SIEM is very convenient to be divided into the following stages: log acquisition (gathering), event normalization, event aggregation, event filtering, event categorization, event prioritization. In Arcsight, all these tasks are performed by connectors, for some of the event sources are delivered out of the box, some need to be written by yourself.
Features of logging - the main feature is that you need to plan the load on the network, especially when using SYSLOG, because Spam is possible and you need to work with networkers to think about the best way to send logs.
Event normalization - log parsing, i.e. bringing the received event into a view that is understandable to a person and a database that spins under the hood of SIEM. If in more detail we take an event, we beat it into parts and we throw it in a cell of the table of the SIEM database, i.e. The address of the event source flies to the database event source column. In general, all events should be normalized.
Event aggregation is a very interesting and useful point (not all SIEMs support this function), which allows reducing the amount of garbage in the logs. Aggregation is the collapse of several identical events into one, for example, we have 30 calls in 30 seconds from one IP address to our server, which our border firewall recorded, in the database it will be 30 lines, aggregation is needed to make 30 of these only one event. Aggregation is most effective for logs of firewalls, netflow, IDS / IPS, web servers.
Filtering events - throwing away what is not needed or leaving only what is needed. Very useful thing especially for logs from operating systems, antiviruses, IDS / IPS. What is most effective to do to determine that the event is of no interest to you. You need to write a rule that will write to the list (in the terminology of Arcsight, I don’t know if the competitors have such a function) or build a report on all the events that occurred during the week from each source and analyze it from the point of view of informativeness. The result of this analysis is a change in the logging level at the event source, or setting up filters on the SIEM log collector. Looking ahead, I’ll say that it’s better not to filter the firewalls, since They are very useful in investigating incidents.
Event categorization is the assignment of events of the same type from different sources of the same category in order to be more convenient to handle. I’ll say right away that the thing looks more promising than it really is if you have a lot of different sources and you just don’t have time to set up the categorization correctly. For those whose infrastructure is growing slowly, I think it will be useful.
Prioritization - setting the event priority within the SIEM. For example, we received an event with severity CRIT, but we know that for IS it did not criticize and set it an INFO.
Let's move on a bit to practice and talk about the most informative sources of logs.
My undisputed leader is vulnerability scanners that allow you to make an inventory, a list of open ports, services and vulnerabilities on the network. Here I would recommend using several tools at once and collecting everything into sheets or reports, and then tying the whole thing to ITIL, i.e. create internal tickets and close problems according to internal IS policies. Actually SIEM here is a tool on which all scanners have thrown off the information and the analyst is already analyzing it by looking at all the reports in only one place. Also here I would refer samopisnye scripts that can collect information from DNSBL, C & C servers from the Internet.
The second place for me is occupied by firewalls, netflow, VPN gateway and IPS / IDS / WAF, which allow detecting all sorts of scans, attempts of DDoS attacks, other network security incidents, including. internal users and to optimize the work of protection.
The third place is operating systems, from the logs of operating systems it can be understood that we were hacked or the admin is naughty.
Fourth place - the database, on their requests, you can also see attempts to divert important information from us.
Fifth place - antivirus.
I think on this the first part can be finished. Wait for the second one, in which I will tell you what is best to visualize, what to correlate with, what to notify, and what you can react with with scripts and why.
Source: https://habr.com/ru/post/271745/
All Articles