OSSIM - we use integrated open source security management system
This article is written under the impression of the article " OSSIM - Deploying a comprehensive open source security management system ." I will not repeat and describe the process of installing the system. I just want to make some clarifications and explanations related to practical experience in the use of OSSIM.
In any instructions you read that "uniquely on the virtual". My answer: "definitely on iron." Both answers are correct. Why?
The first answer is correct, since OSSIM is installed “bundled” with the operating system and the image disk may not contain the drivers necessary for your hardware. If you come across this, it will be your problem that you will have to solve yourself. Installation on a virtual machine provides some versatility and no problems. In addition, there is no guarantee that after an update, problems will not have to be solved anew. There are official commercial "iron implementations" that are sold as an appliance and, accordingly, are supported, but we are looking at Open Source OSSIM and all the iron problems here are your problems.
The second answer is correct, since any hypervisor is a brake. Advocates of universal virtualization can trumpet as much as they want into pipes, beat into drums, swing flags, but hypervisors dramatically reduce the performance of the guest system. As a rule, this can be neglected, but OSSIM likes speed. As a result, a mini-computer with a 4-core atom, 16GB DDR3 and a 128GB SATA3 SSD self-assembled on a barebone takes the virtual machine to HP dl380 easily. You will need to give the virtual machine far more resources than the physical needs. And it will cost more. As for the drivers, during installation you will be asked to insert a USB flash drive with drivers, if required.
')
Important. On the iron machine, the installer will start only from the boot CD. If you upload an image to a flash drive, the OSSIM installation will not happen, but you can install Debian.
Very important. Never try to put OSSIM "second system" on a work computer. The installer believes that he is put on a bare car and does not ask idiotic questions "how to partition the disc?". First of all, it will rewrite the partition table without asking for it and format the disk.
Installing OSSIM does not take “15 minutes” at all. Much longer. At the last stage, it will even seem to you that the installer is frozen and everything is gone. Wait. He works.
The speed of the disk subsystem. Properly use SSD. Volumes are not important. 100GB is enough. But speed is very important. There are two reasons for this. First, the text logs that go to syslog and then read by OSSIM plugins. Secondly, a database that runs on the same machine. SSD dramatically improves system performance.
The number of processor cores. The performance of each core is not as important as the number of cores. OSSIM performs very simple operations, but there are many of them at once and they can be executed in parallel. This is also important for its Network IDS (Suricata), as you know, it is multi-threaded.
The amount of RAM. It is less critical, but it is better that nothing is pushed into the paging file.
A suitable configuration for a virtual machine based on a daily flow of 2 million events in the logs and 10 terabytes of traffic on the “illegible” interface: 8 cores, 16GB. But it is butt. Resources will be eaten to capacity.
And yes, of course, an illegible interface, to which you intend to throw traffic from the span port of your switch, collected for the analysis of meerkats from all the networks of your network, should be 10Gbs. Otherwise he will simply choke.
If you are going to manage the OSSIM server from a Windows machine, you will need putty and WinSCP. Well, or FAR, if you love him. It's more pleasant to drive from a Ubuntu machine. At least it is not necessary to be checked every time whether it is not accidentally recorded in any script the end of the lines in the form of CRLF.
After installation, you will not be able to connect to the server via SSH from a remote machine. In Debian 8, the default configuration for sshd is `PermitRootLogin without-password`, which must be changed from` / etc / ssh / ssh_config` to `PermitRootLogin yes`.
Another important thing is the correct configuration of the time zone of plug-ins. The fact is that even when all the sources of the logs are in the same time zone, they are not necessarily using the same local time. For example, System Center Configuration Manager considers that time is reasonable to store in a database in UTC. And if you have a plugin that reads new events from its database (and I have), then you need to take into account that they are not recorded in local time.
The time zone for plugins is set in two places: firstly, in `/ etc / ossim / agent / config.cfg`, the default time zone is set for all plugins, and secondly, it can be overridden in the config file of each individual plug-in. Determining the time zone means the instruction for the plugin generator "to assume that data comes from such a time zone and convert the time into ours." In this case, “our” is the local server time. In fact, the time will be recorded in the database in UTC, but there is a separate field in which the “local system” offset will be recorded.
Interesting things start when you have two of the same type in different time zones. For example, you take logs of cisco-asa routers from different branches. In this case, you will need to process them with different plugins, in the config files of which you specify a different parameter `tzone =`. This parameter is specified in the `[default]` section. I am writing about this because in the documentation you will not find it, I do not know why. POSIX format: `tzone = Europe / Moscow`.
My answer: AlienVault HIDS “native” means, OSSEC. Of course I will briefly explain this.
In theory, you can use any of the many agents to send a Windows event log to syslog or use WMI. OSSIM has a WMI client and there is a standard plugin for reading Windows logs. There is also a standard plugin for the SNARE agent, but if you are dealing with Russian Windows, then it will not be useful to you. The thing is that SNARE sends data from Russian Windows in cp1251 encoding, and the standard parser for SNARE is written under cp1252. We'll have to edit the regulars.
But the most interesting thing will start when you collect logs from "multilingual" systems. For example, you have Russian Windows on workstations, and English on servers. And how to analyze it now? In fact, this is a common problem for all SIEM. They solve it in different ways. For example, ArcSight uses a rather convoluted system of collecting Windows logs, which allows you to collect logs only in English, regardless of the localization of the system. OSSEC uses a very simple technique. When installing the agent on Windows, a csv file with a table of string messages of the Windows log in English is written into its working directory. Therefore, the “essential part” of a message that is needed for analysis always comes in the same language, regardless of the localization of the system. But the "data" come "in the original language." It is quite convenient.
In addition, the standard plugin for the OSSEC agent is very well defined. He carefully examines events by type. If you use a different way to collect logs, you will have to sweat a great deal over the unfolding of events on the shelves. Well, finally, OSSEC is not just “sending logs” - this is really Host IDS and is not very bad. Not for nothing TrendMicro decided to use this engine in its "advanced" antivirus. And yes, OSSEC is reliable, as a control shot. You can safely install the agent on any critical system.
And further. Rabbits are not only valuable fur, but also 3-4 kilograms of dietary meat. OSSEC is not a log collector, it is SIEM itself. In OSSIM, it does not transmit the Windows log at all, but its own alert.log, which is formed on the basis of preprocessing of events received from agents. There will be, for example: events for changing monitored files, or such aggregated events as “multiple error” or “multiple change of the checksum of the key in the registry”. It is more useful than just a collector. OSSEC is very widely distributed over the Internet, it is very often used to protect web servers, so the community is large and active.
Of course, you can play around with other ways of collecting Windows logs. It is interesting.
On Windows machines, the agent is installed with some default configuration file. This file is here: `/ usr / share / ossec-generator / installer / ossec.conf`. OSSEC supports downloading configurations from the server. The file `/ var / ossec / etc / shared / agent.conf` is used for this purpose. By default it is absent. This file can be created from the web interface of the OSSIM console (Environment - Detection - Agents - agent.conf). Or just write in a text editor.
There should be contained in the format of XML configuration directives that will be "merged" with the local agent configuration file. Directive blocks for different agents can be labeled to apply only to the agents they need. You can mark by OS type, agent name, profile name (in the local configuration of the agent, in this case, the name of his profile must be specified).
In this way, agent configurations can be changed centrally and pointwise. The term "merged" requires clarification. What happens if the local section of the configuration file is overridden? The theory says that the local file is read first, then the server file, the rule read last overrides all previous ones. In practice, I have never tried such an intersection of configuration parameters. It just wasn't necessary. Hope, when needed, will work as I think.
Another important point. The template for the local configuration of agents indicates the method of connecting to the server:
As you can see here ip address. I do not know why the authors preferred such a pattern. You can specify the FQDN then this section should look different:
However, this is not done. This means that when changing the server’s ip address, all agents will fall off. Not a good idea. Of course, SIEM is not a device where ip address is changed, but somehow unpleasant. Of course, you can change this section in the template, but most likely, with the next update, the template will be generated anew. It will be necessary to constantly monitor this. I don’t care about this problem because I don’t have the habit of changing addresses.
On the Windows agent, it is most convenient to install from the OSSIM web console, using the Automatic deployment button. But you will need the local administrator's credentials of the target host (or domain). Installation from group policies or using SCCM is very difficult. The fact is that each installation file is individual for a particular host, because it contains a unique key for encrypting the exchange with the server. Such is the secrecy at the PCI DSS level. Pechalka.
For Linux, agentless can work, but it requires setting up connections to hosts over SSH. In my opinion, this is a bad idea. I prefer to put agents. In this case, a completely manual installation with compilation of the agent on the target machine from the raw. Version 2.8.2 is officially supported, but version 2.8.3 also works without problems. In fact, there are packages for different systems, for example, for Debian. Read more here.
Click the support button in the web console menu and get links. Required to read:
USM 5.x Plugins Management Guide
Customizing Correlation Directives or Cross Correlation Rules
Intrusion Detection in AlienVault
Policy Management Fundamentals
Using USM and OSSIM 5.1 with OTX - AlienVault
Assets, Groups & Networks
System Errors, Warnings and Suggestions
It seems everyone understands, but as it comes to the point, it turns out that people, for the most part, do not truly understand the purpose of SIEM. First of all, it is not a means of protection. And not even a "security management system", excuse me the title of the article, but not the first one I started. In fact, it is a means of detecting successful security breaches. The standard scheme of building protection begins with the identification of possible attacks (actual threats) and the methods of their commission. Then technical and organizational protection measures are invented and implemented, which are placed on possible “attack vectors”. And after all this, “controls” are set up, the purpose of which is to see that all your protective measures do not work. SIEM belongs to this class.
This is what you need to understand. SIEM is designed to detect violations or circumvention of protective measures, both technical and organizational, that have already occurred. This is something like a video surveillance system. First of all, you must determine by what signs you will detect these violations, how SIEM will detect these signs, from where it will take the necessary information. Do you have any "dead zones" that SIEM does not see. Situations may be the most unexpected.
OSSIM saves a suspicious package in pcap format for analysis. Here he is:
What was it?
And so it was. In one office, IT specialists tested some kind of thing that was spinning in the tomcat application server environment. They installed this thing on the first server that came up, of course, leaving the login to the console with an empty password. By tested and forgotten. The first available server was Terminal Services. And, as you know, tomcat worked under the system account. An evil phishing hacker slipped a Trojan onto a computer of one of the employees and looked at what was useful on the network. Found this forgotten web server and was delighted. I poured a library on it to support console commands. It is the moment of testing this library is shown in the picture. The net user command was run, and suricata whistled when she saw the output of this command in http-responce. While we figured out what kind of garbage, the evil hacker had already put a mimicacu there and started collecting user passwords, since they all go to the terminal server.
The situation is very vital. People always make mistakes. Not only IT pros, but also hackers.
Or here's a simpler story. Here is the alarm page is very fresh.
This is the so-called OTX Pulse. Here are its details:
What was it?
And this is the browser of one of the employees when he looked through one of the forum threads on the site bankir.ru, rushed to the url owqkq.ne1t3v8.top, and this is one of the parking pages that use the Angler Exploit Toolkit. Also an interesting story.
In order to avoid “dead zones”, it is necessary first of all to place the HIDS agents (OSSEC) on ALL hosts on the network. Then, set up the transfer to the "illegible" OSSIM interface of ALL traffic on your internal network so that it is processed by NIDS (Suricata). Be sure to configure and run periodic vulnerability scans with the OpenVAS scanner of at least all DMZ nodes. It is important. Observations show that even a very small company passes 20-50 enemy scanners every day. It will be better if you find a vulnerability earlier than they. I do not exaggerate, only understate. Here is a real fragment of the daily report:
And this is only the part that belongs to the well-known community of villains, and there is another part of the report with a list of unknown villains. Of course, most of them do not press hard. For example: like this (this is a cut from the same report):
But there are also very intrusive personalities.
You will also need firewall logs. This is the minimum that is in any office. And then there are already internal things: access logs to the DBMS and separate business applications, web server logs, etc. etc. You can collect anything. If there is no standard log parsing plugin, it’s very easy to write your own. True easy. Developing your first plug-in won't take you more than a couple of days. And the development of the second will take a couple of hours.
As for the control of violation of organizational security measures, for example, in your office it is forbidden to make changes to the configuration of the production environment without coordinating the information security service. OSSEC will tell you about any configuration changes and you can check whether it was agreed or not. The most interesting thing begins when it turns out that no one has made this change.
I will not give advice on which rules of control should be applied. Each village has its own traditions. If you have questions, ask.
Important. AlienVault OSSIM is not only SIEM in the classic sense. This is a complete shelf that includes: Host IDS, Network IDS, Wireless IDS, Volnurability Scaner, NetFlow Collector. That is, it is a complete classic set of "controls" for the organization of video surveillance over the company's network.
If it seems to you that something is wrong, you should start looking at the logs. Where?
OSSEC logs and errors:
`/var/ossec/logs/ossec.log`- here you will see OSSEC errors. The most common mistake is the loss of communication with the agent, it happens with some fatal communication failure on the network, or after reinstalling the agent. Rare, but with a large number of agents may not be so rare. If on the console you see that any agent is not active, and you know that the computer is turned on, it means you are here. In the log, the error looks like `ERROR: Duplicated counter for agent-name '. Eliminated simply. On the web console (environment - detection - agents) or in the `/ var / ossec / etc / client.keys` file, look for an agent with this name and look at its number (the number in the leftmost column). Then we climb into the directory `/ var / ossec / queue / rids` and delete the file with the name - agent number in it. We go to SSH on the server console, exit to the command line and do `/etc/init.d/ossec restart`. All things. To be honest, I did not see any other errors with OSSEC.
`/var/ossec/logs/alerts/alerts.log` is the log in which OSSEC adds chewed events from agents, it is from here that OSSIM reads events and processes them with a plugin for OSSEC. Here you can see what falls there and how it looks.
The rest of the logs in the traditional `/ var / log` of them are useful` / var / log / alienvault / agent / agent.log` and in the same place `agent_error.log`. They can be useful when debugging your own plugins. Note that on a production system, the size of `agent.log` is in gigabytes.
I have never seen any serious errors in the work of OSSIM. Once after the next update of the system, the regular backup stopped working. It turned out they incorrectly assigned the rights to one of the files. This is fixed literally the next day. Information about such problems is held on the community forum. One more time, after the next update, the following updates stopped working. This time it was a warning that the update that was posted four hours ago contains a bug and that it needs to be fixed this way, but the corrected update was posted at such and such time. By the way, if the update standard means (from the web or ssh console) falls and does not work, it can be done with the commands
from the server console.
Perhaps this is all I wanted to say. And most importantly. This is open source. Whatever you want, Python is with you. And if you do not want it to be with you, look at the commercial version - Python with them. And here we come to the last question.
My answer: if you need a SIEM for the needs of the company, then Enterprise. You will never have enough human and technical resources to bring the Open Source version to the Enterprise level, of course, unless your company aims to develop its own SIEM. But, if this is not your business, get yours. Unfortunately, any Enterprise SIEM is very expensive. But they are so expensive not because they have some unusual correlation server engines, but because they have enormous costs for creating libraries of correlation rules for events, analyzing the "signatures" of complex attacks and how to detect them, debugging these methods. This is a job that you cannot do on your own. I do not want to talk about the benefits of commercial products here, these advantages seem obvious to me and not worth discussing.
So why do I spend time on this open source? First, I like him. Of all such open source projects, this is a good one. Secondly, something like “we want to make security accessible to all” is written on the AlienVault website. Not bad idea. I support. Many companies simply can not afford to buy a commercial product of this class. I know that there is a whole army of system administrators, freelancers, let's call them so serving small companies. It makes sense to look in this direction. It’s not for nothing that I mentioned a mini-computer at the very beginning. The little SIEM is the real thing.
Enterprise - enterprise, and the community - community. Land - to peasants, factories - to workers, money - to bank employees. “Happiness to everyone, and to have enough for everyone” (A. and B. Strugatsky. “Roadside Picnic”).
Used sources:
https://www.alienvault.com/documentation
https://alienvault.ru/open-threat-exchange/
http://ossec.imtqy.com/docs/
http://suricata-ids.org/docs/
What to put? Iron or virtualka?
In any instructions you read that "uniquely on the virtual". My answer: "definitely on iron." Both answers are correct. Why?
The first answer is correct, since OSSIM is installed “bundled” with the operating system and the image disk may not contain the drivers necessary for your hardware. If you come across this, it will be your problem that you will have to solve yourself. Installation on a virtual machine provides some versatility and no problems. In addition, there is no guarantee that after an update, problems will not have to be solved anew. There are official commercial "iron implementations" that are sold as an appliance and, accordingly, are supported, but we are looking at Open Source OSSIM and all the iron problems here are your problems.
The second answer is correct, since any hypervisor is a brake. Advocates of universal virtualization can trumpet as much as they want into pipes, beat into drums, swing flags, but hypervisors dramatically reduce the performance of the guest system. As a rule, this can be neglected, but OSSIM likes speed. As a result, a mini-computer with a 4-core atom, 16GB DDR3 and a 128GB SATA3 SSD self-assembled on a barebone takes the virtual machine to HP dl380 easily. You will need to give the virtual machine far more resources than the physical needs. And it will cost more. As for the drivers, during installation you will be asked to insert a USB flash drive with drivers, if required.
')
Important. On the iron machine, the installer will start only from the boot CD. If you upload an image to a flash drive, the OSSIM installation will not happen, but you can install Debian.
Very important. Never try to put OSSIM "second system" on a work computer. The installer believes that he is put on a bare car and does not ask idiotic questions "how to partition the disc?". First of all, it will rewrite the partition table without asking for it and format the disk.
Installing OSSIM does not take “15 minutes” at all. Much longer. At the last stage, it will even seem to you that the installer is frozen and everything is gone. Wait. He works.
The most important resources for OSSIM in descending order of importance
The speed of the disk subsystem. Properly use SSD. Volumes are not important. 100GB is enough. But speed is very important. There are two reasons for this. First, the text logs that go to syslog and then read by OSSIM plugins. Secondly, a database that runs on the same machine. SSD dramatically improves system performance.
The number of processor cores. The performance of each core is not as important as the number of cores. OSSIM performs very simple operations, but there are many of them at once and they can be executed in parallel. This is also important for its Network IDS (Suricata), as you know, it is multi-threaded.
The amount of RAM. It is less critical, but it is better that nothing is pushed into the paging file.
A suitable configuration for a virtual machine based on a daily flow of 2 million events in the logs and 10 terabytes of traffic on the “illegible” interface: 8 cores, 16GB. But it is butt. Resources will be eaten to capacity.
And yes, of course, an illegible interface, to which you intend to throw traffic from the span port of your switch, collected for the analysis of meerkats from all the networks of your network, should be 10Gbs. Otherwise he will simply choke.
If you are going to manage the OSSIM server from a Windows machine, you will need putty and WinSCP. Well, or FAR, if you love him. It's more pleasant to drive from a Ubuntu machine. At least it is not necessary to be checked every time whether it is not accidentally recorded in any script the end of the lines in the form of CRLF.
After installation, you will not be able to connect to the server via SSH from a remote machine. In Debian 8, the default configuration for sshd is `PermitRootLogin without-password`, which must be changed from` / etc / ssh / ssh_config` to `PermitRootLogin yes`.
Time zone
Another important thing is the correct configuration of the time zone of plug-ins. The fact is that even when all the sources of the logs are in the same time zone, they are not necessarily using the same local time. For example, System Center Configuration Manager considers that time is reasonable to store in a database in UTC. And if you have a plugin that reads new events from its database (and I have), then you need to take into account that they are not recorded in local time.
The time zone for plugins is set in two places: firstly, in `/ etc / ossim / agent / config.cfg`, the default time zone is set for all plugins, and secondly, it can be overridden in the config file of each individual plug-in. Determining the time zone means the instruction for the plugin generator "to assume that data comes from such a time zone and convert the time into ours." In this case, “our” is the local server time. In fact, the time will be recorded in the database in UTC, but there is a separate field in which the “local system” offset will be recorded.
Interesting things start when you have two of the same type in different time zones. For example, you take logs of cisco-asa routers from different branches. In this case, you will need to process them with different plugins, in the config files of which you specify a different parameter `tzone =`. This parameter is specified in the `[default]` section. I am writing about this because in the documentation you will not find it, I do not know why. POSIX format: `tzone = Europe / Moscow`.
What is the best way to collect Windows logs?
My answer: AlienVault HIDS “native” means, OSSEC. Of course I will briefly explain this.
In theory, you can use any of the many agents to send a Windows event log to syslog or use WMI. OSSIM has a WMI client and there is a standard plugin for reading Windows logs. There is also a standard plugin for the SNARE agent, but if you are dealing with Russian Windows, then it will not be useful to you. The thing is that SNARE sends data from Russian Windows in cp1251 encoding, and the standard parser for SNARE is written under cp1252. We'll have to edit the regulars.
But the most interesting thing will start when you collect logs from "multilingual" systems. For example, you have Russian Windows on workstations, and English on servers. And how to analyze it now? In fact, this is a common problem for all SIEM. They solve it in different ways. For example, ArcSight uses a rather convoluted system of collecting Windows logs, which allows you to collect logs only in English, regardless of the localization of the system. OSSEC uses a very simple technique. When installing the agent on Windows, a csv file with a table of string messages of the Windows log in English is written into its working directory. Therefore, the “essential part” of a message that is needed for analysis always comes in the same language, regardless of the localization of the system. But the "data" come "in the original language." It is quite convenient.
In addition, the standard plugin for the OSSEC agent is very well defined. He carefully examines events by type. If you use a different way to collect logs, you will have to sweat a great deal over the unfolding of events on the shelves. Well, finally, OSSEC is not just “sending logs” - this is really Host IDS and is not very bad. Not for nothing TrendMicro decided to use this engine in its "advanced" antivirus. And yes, OSSEC is reliable, as a control shot. You can safely install the agent on any critical system.
And further. Rabbits are not only valuable fur, but also 3-4 kilograms of dietary meat. OSSEC is not a log collector, it is SIEM itself. In OSSIM, it does not transmit the Windows log at all, but its own alert.log, which is formed on the basis of preprocessing of events received from agents. There will be, for example: events for changing monitored files, or such aggregated events as “multiple error” or “multiple change of the checksum of the key in the registry”. It is more useful than just a collector. OSSEC is very widely distributed over the Internet, it is very often used to protect web servers, so the community is large and active.
Of course, you can play around with other ways of collecting Windows logs. It is interesting.
Something about OSSEC Agent Configuration
On Windows machines, the agent is installed with some default configuration file. This file is here: `/ usr / share / ossec-generator / installer / ossec.conf`. OSSEC supports downloading configurations from the server. The file `/ var / ossec / etc / shared / agent.conf` is used for this purpose. By default it is absent. This file can be created from the web interface of the OSSIM console (Environment - Detection - Agents - agent.conf). Or just write in a text editor.
There should be contained in the format of XML configuration directives that will be "merged" with the local agent configuration file. Directive blocks for different agents can be labeled to apply only to the agents they need. You can mark by OS type, agent name, profile name (in the local configuration of the agent, in this case, the name of his profile must be specified).
<agent_config name="agent001|agent002|agent018">
</agent_config>
<agent_config os="Linux|FreeBSD">
</agent_config>
<agent_config os="Windows">
</agent_config>
<agent_config profile="web-server">
</agent_config>
In this way, agent configurations can be changed centrally and pointwise. The term "merged" requires clarification. What happens if the local section of the configuration file is overridden? The theory says that the local file is read first, then the server file, the rule read last overrides all previous ones. In practice, I have never tried such an intersection of configuration parameters. It just wasn't necessary. Hope, when needed, will work as I think.
Another important point. The template for the local configuration of agents indicates the method of connecting to the server:
<server-ip>172.17.2.10</server-ip>
<notify_time>120</notify_time>
<time-reconnect>240</time-reconnect>
As you can see here ip address. I do not know why the authors preferred such a pattern. You can specify the FQDN then this section should look different:
<server-hostname>fqdn</server-hostname>
...
However, this is not done. This means that when changing the server’s ip address, all agents will fall off. Not a good idea. Of course, SIEM is not a device where ip address is changed, but somehow unpleasant. Of course, you can change this section in the template, but most likely, with the next update, the template will be generated anew. It will be necessary to constantly monitor this. I don’t care about this problem because I don’t have the habit of changing addresses.
Notes on installing agents on hosts
On the Windows agent, it is most convenient to install from the OSSIM web console, using the Automatic deployment button. But you will need the local administrator's credentials of the target host (or domain). Installation from group policies or using SCCM is very difficult. The fact is that each installation file is individual for a particular host, because it contains a unique key for encrypting the exchange with the server. Such is the secrecy at the PCI DSS level. Pechalka.
For Linux, agentless can work, but it requires setting up connections to hosts over SSH. In my opinion, this is a bad idea. I prefer to put agents. In this case, a completely manual installation with compilation of the agent on the target machine from the raw. Version 2.8.2 is officially supported, but version 2.8.3 also works without problems. In fact, there are packages for different systems, for example, for Debian. Read more here.
Where to look for documentation and what is best to read
Click the support button in the web console menu and get links. Required to read:
USM 5.x Plugins Management Guide
Customizing Correlation Directives or Cross Correlation Rules
Intrusion Detection in AlienVault
Policy Management Fundamentals
Using USM and OSSIM 5.1 with OTX - AlienVault
Assets, Groups & Networks
System Errors, Warnings and Suggestions
Why do I need a SIEM?
It seems everyone understands, but as it comes to the point, it turns out that people, for the most part, do not truly understand the purpose of SIEM. First of all, it is not a means of protection. And not even a "security management system", excuse me the title of the article, but not the first one I started. In fact, it is a means of detecting successful security breaches. The standard scheme of building protection begins with the identification of possible attacks (actual threats) and the methods of their commission. Then technical and organizational protection measures are invented and implemented, which are placed on possible “attack vectors”. And after all this, “controls” are set up, the purpose of which is to see that all your protective measures do not work. SIEM belongs to this class.
This is what you need to understand. SIEM is designed to detect violations or circumvention of protective measures, both technical and organizational, that have already occurred. This is something like a video surveillance system. First of all, you must determine by what signs you will detect these violations, how SIEM will detect these signs, from where it will take the necessary information. Do you have any "dead zones" that SIEM does not see. Situations may be the most unexpected.
OSSIM saves a suspicious package in pcap format for analysis. Here he is:
What was it?
And so it was. In one office, IT specialists tested some kind of thing that was spinning in the tomcat application server environment. They installed this thing on the first server that came up, of course, leaving the login to the console with an empty password. By tested and forgotten. The first available server was Terminal Services. And, as you know, tomcat worked under the system account. An evil phishing hacker slipped a Trojan onto a computer of one of the employees and looked at what was useful on the network. Found this forgotten web server and was delighted. I poured a library on it to support console commands. It is the moment of testing this library is shown in the picture. The net user command was run, and suricata whistled when she saw the output of this command in http-responce. While we figured out what kind of garbage, the evil hacker had already put a mimicacu there and started collecting user passwords, since they all go to the terminal server.
The situation is very vital. People always make mistakes. Not only IT pros, but also hackers.
Or here's a simpler story. Here is the alarm page is very fresh.
This is the so-called OTX Pulse. Here are its details:
What was it?
And this is the browser of one of the employees when he looked through one of the forum threads on the site bankir.ru, rushed to the url owqkq.ne1t3v8.top, and this is one of the parking pages that use the Angler Exploit Toolkit. Also an interesting story.
In order to avoid “dead zones”, it is necessary first of all to place the HIDS agents (OSSEC) on ALL hosts on the network. Then, set up the transfer to the "illegible" OSSIM interface of ALL traffic on your internal network so that it is processed by NIDS (Suricata). Be sure to configure and run periodic vulnerability scans with the OpenVAS scanner of at least all DMZ nodes. It is important. Observations show that even a very small company passes 20-50 enemy scanners every day. It will be better if you find a vulnerability earlier than they. I do not exaggerate, only understate. Here is a real fragment of the daily report:
2016:02:18 - 2016:02:19
IP
2016-02-18 09:22:46 180.97.106.37 Nanjing Malicious Host
2016-02-18 09:38:37 216.218.206.123 Fremont Malicious Host
2016-02-18 09:52:57 85.25.214.226 Germany Scanning Host
2016-02-18 10:08:11 146.185.250.105 Saint Petersburg Malicious Host
2016-02-18 10:22:54 178.62.14.193 London Malicious Host
2016-02-18 10:23:24 94.102.49.79 Netherlands Malicious Host
2016-02-18 10:47:52 195.88.209.6 Moscow Malicious Host
2016-02-18 10:53:29 222.186.34.177 Nanjing Malicious Host
2016-02-18 11:07:48 71.6.135.131 San Diego Malicious Host
2016-02-18 11:58:17 193.105.134.220 Sweden Malicious Host
2016-02-18 11:58:51 62.210.206.219 France Malicious Host
2016-02-18 12:28:13 193.109.69.150 Russia Malicious Host
2016-02-18 12:43:40 216.218.206.96 Fremont Malicious Host
2016-02-18 13:08:50 209.126.124.67 St Louis Malicious Host
2016-02-18 13:53:19 178.33.17.241 France Malicious Host
2016-02-18 14:23:52 198.20.70.114 Chicago Malicious Host
2016-02-18 14:32:49 104.219.238.10 Rye Malicious Host Scanning Host
2016-02-18 14:38:38 198.23.112.119 Dallas Scanning Host
2016-02-18 15:02:58 198.20.69.98 Chicago Malicious Host
2016-02-18 15:03:29 64.125.239.136 United States Malicious Host
2016-02-18 15:28:35 162.248.74.2 Clarks Summit Malicious Host
2016-02-18 15:43:36 222.174.5.28 Jinan Malicious Host
2016-02-18 15:57:42 66.240.236.119 San Diego Malicious Host
2016-02-18 16:13:09 74.82.47.45 Fremont Malicious Host
2016-02-18 16:13:44 64.125.239.92 United States Malicious Host
2016-02-18 17:07:57 142.54.162.74 Kansas City Malicious Host
2016-02-18 17:22:41 64.125.239.107 United States Malicious Host
2016-02-18 17:58:54 23.239.66.99 United States Malicious Host
2016-02-18 18:07:50 61.216.2.14 Taiwan Malicious Host
2016-02-18 18:08:03 198.20.69.74 Chicago Malicious Host
2016-02-18 18:08:18 141.212.122.84 Ann Arbor Malicious Host
2016-02-18 18:08:18 141.212.122.81 Ann Arbor Malicious Host
2016-02-18 19:52:53 185.94.111.1 Russia Malicious Host
2016-02-18 19:58:27 162.244.35.24 United States Malicious Host
2016-02-18 20:23:00 162.244.35.22 United States Malicious Host
2016-02-18 20:23:37 89.248.160.192 Netherlands Malicious Host
2016-02-18 20:43:55 222.174.5.17 Jinan Malicious Host
2016-02-18 21:23:55 185.130.5.201 Republic of Lithuania Malicious Host
2016-02-18 21:47:39 92.60.184.34 Ukraine Scanning Host
2016-02-18 22:33:48 209.126.102.181 St Louis Malicious Host
2016-02-18 22:57:37 71.6.167.142 San Diego Malicious Host
2016-02-18 23:13:37 212.83.148.78 France Malicious Host
2016-02-19 00:07:54 185.130.5.240 Republic of Lithuania Scanning Host
2016-02-19 00:48:22 64.125.239.224 United States Malicious Host
2016-02-19 01:13:11 66.240.192.138 San Diego Malicious Host Scanning Host
2016-02-19 02:33:05 198.204.234.74 Kansas City Scanning Host Malicious Host
2016-02-19 02:57:03 104.243.223.8 Tampa Malicious Host
2016-02-19 02:58:02 198.20.99.130 Netherlands Malicious Host
2016-02-19 03:27:43 162.244.35.25 United States Malicious Host
2016-02-19 03:28:13 89.163.251.200 Germany Malicious Host
2016-02-19 04:28:25 71.6.165.200 San Diego Malicious Host
2016-02-19 04:52:08 93.174.93.181 Netherlands Malicious Host
2016-02-19 04:58:24 184.105.247.238 Fremont Malicious Host
2016-02-19 05:23:07 192.162.101.79 Russia Malicious Host
2016-02-19 05:23:20 64.125.239.112 United States Malicious Host
2016-02-19 06:12:43 188.138.1.218 Germany Malicious Host Scanning Host
2016-02-19 06:12:44 74.82.47.55 Fremont Malicious Host
2016-02-19 06:43:55 209.239.123.106 St Louis Malicious Host
2016-02-19 07:13:45 185.56.28.67 Netherlands Malicious Host
2016-02-19 08:13:09 184.105.247.228 Fremont Malicious Host
2016-02-19 08:28:51 184.105.139.72 Fremont Malicious Host
And this is only the part that belongs to the well-known community of villains, and there is another part of the report with a list of unknown villains. Of course, most of them do not press hard. For example: like this (this is a cut from the same report):
Netflow 180.97.106.37 : Nanjing : Malicious Host
2016-02-18 09:22:46.585 0.000 ICMP 91.111.111.9:0 180.97.106.37:3.0 1 56 1
2016-02-18 09:22:46.586 0.000 TCP 180.97.106.37:46024 91.111.111.9:3128 1 40 1
2016-02-18 20:32:23.247 0.000 TCP 180.97.106.37:37254 91.111.111.101:22 1 40 1
2016-02-18 20:43:38.783 0.000 TCP 180.97.106.37:45840 91.111.111.25:22 1 40 1
2016-02-18 20:43:38.783 0.000 ICMP 91.111.111.25:0 180.97.106.37:3.0 3 204 1
2016-02-18 22:07:25.502 0.000 TCP 180.97.106.37:54895 91.111.111.36:22 2 80 1
2016-02-18 22:41:06.739 0.000 TCP 91.111.111.13:22 180.97.106.37:48365 1 40 1
2016-02-18 23:16:01.974 0.996 TCP 180.97.106.37:13302 91.111.111.32:80 10 539 1
2016-02-18 23:16:01.975 0.679 TCP 91.111.111.32:80 180.97.106.37:13302 7 3048 1
2016-02-18 23:20:07.473 0.000 TCP 180.97.106.37:43667 91.111.111.9:22 2 80 1
2016-02-18 23:20:07.473 0.000 ICMP 91.111.111.9:0 180.97.106.37:3.0 1 56 1
2016-02-19 05:50:52.757 12.217 TCP 91.111.111.44:80 180.97.106.37:53461 5 260 1
But there are also very intrusive personalities.
You will also need firewall logs. This is the minimum that is in any office. And then there are already internal things: access logs to the DBMS and separate business applications, web server logs, etc. etc. You can collect anything. If there is no standard log parsing plugin, it’s very easy to write your own. True easy. Developing your first plug-in won't take you more than a couple of days. And the development of the second will take a couple of hours.
As for the control of violation of organizational security measures, for example, in your office it is forbidden to make changes to the configuration of the production environment without coordinating the information security service. OSSEC will tell you about any configuration changes and you can check whether it was agreed or not. The most interesting thing begins when it turns out that no one has made this change.
I will not give advice on which rules of control should be applied. Each village has its own traditions. If you have questions, ask.
Important. AlienVault OSSIM is not only SIEM in the classic sense. This is a complete shelf that includes: Host IDS, Network IDS, Wireless IDS, Volnurability Scaner, NetFlow Collector. That is, it is a complete classic set of "controls" for the organization of video surveillance over the company's network.
Trouble and Shooting
If it seems to you that something is wrong, you should start looking at the logs. Where?
OSSEC logs and errors:
`/var/ossec/logs/ossec.log`- here you will see OSSEC errors. The most common mistake is the loss of communication with the agent, it happens with some fatal communication failure on the network, or after reinstalling the agent. Rare, but with a large number of agents may not be so rare. If on the console you see that any agent is not active, and you know that the computer is turned on, it means you are here. In the log, the error looks like `ERROR: Duplicated counter for agent-name '. Eliminated simply. On the web console (environment - detection - agents) or in the `/ var / ossec / etc / client.keys` file, look for an agent with this name and look at its number (the number in the leftmost column). Then we climb into the directory `/ var / ossec / queue / rids` and delete the file with the name - agent number in it. We go to SSH on the server console, exit to the command line and do `/etc/init.d/ossec restart`. All things. To be honest, I did not see any other errors with OSSEC.
`/var/ossec/logs/alerts/alerts.log` is the log in which OSSEC adds chewed events from agents, it is from here that OSSIM reads events and processes them with a plugin for OSSEC. Here you can see what falls there and how it looks.
The rest of the logs in the traditional `/ var / log` of them are useful` / var / log / alienvault / agent / agent.log` and in the same place `agent_error.log`. They can be useful when debugging your own plugins. Note that on a production system, the size of `agent.log` is in gigabytes.
I have never seen any serious errors in the work of OSSIM. Once after the next update of the system, the regular backup stopped working. It turned out they incorrectly assigned the rights to one of the files. This is fixed literally the next day. Information about such problems is held on the community forum. One more time, after the next update, the following updates stopped working. This time it was a warning that the update that was posted four hours ago contains a bug and that it needs to be fixed this way, but the corrected update was posted at such and such time. By the way, if the update standard means (from the web or ssh console) falls and does not work, it can be done with the commands
apt-get update
apt-get upgrade
from the server console.
Perhaps this is all I wanted to say. And most importantly. This is open source. Whatever you want, Python is with you. And if you do not want it to be with you, look at the commercial version - Python with them. And here we come to the last question.
Open Source or Enterprise?
My answer: if you need a SIEM for the needs of the company, then Enterprise. You will never have enough human and technical resources to bring the Open Source version to the Enterprise level, of course, unless your company aims to develop its own SIEM. But, if this is not your business, get yours. Unfortunately, any Enterprise SIEM is very expensive. But they are so expensive not because they have some unusual correlation server engines, but because they have enormous costs for creating libraries of correlation rules for events, analyzing the "signatures" of complex attacks and how to detect them, debugging these methods. This is a job that you cannot do on your own. I do not want to talk about the benefits of commercial products here, these advantages seem obvious to me and not worth discussing.
So why do I spend time on this open source? First, I like him. Of all such open source projects, this is a good one. Secondly, something like “we want to make security accessible to all” is written on the AlienVault website. Not bad idea. I support. Many companies simply can not afford to buy a commercial product of this class. I know that there is a whole army of system administrators, freelancers, let's call them so serving small companies. It makes sense to look in this direction. It’s not for nothing that I mentioned a mini-computer at the very beginning. The little SIEM is the real thing.
Enterprise - enterprise, and the community - community. Land - to peasants, factories - to workers, money - to bank employees. “Happiness to everyone, and to have enough for everyone” (A. and B. Strugatsky. “Roadside Picnic”).
Used sources:
https://www.alienvault.com/documentation
https://alienvault.ru/open-threat-exchange/
http://ossec.imtqy.com/docs/
http://suricata-ids.org/docs/
Source: https://habr.com/ru/post/278809/
All Articles