📜 ⬆️ ⬇️

SOC is people. Downloading expo or how to become a level 20 analyst

The previous article dealt with the search and training of engineers for the first-line center for monitoring and responding to cyber attacks. Today we will talk about search and training for the second line - analysts who investigate atypical incidents and work with the content of the SIEM system, as well as engineers of the GIS operation responsible for setting up protection tools, analyzing attacks and developing custom signatures.

If you ask what requirements we place on candidates, the answer may seem very trivial: certain technical competencies, analytical mind, attentiveness ... However, how to check these qualities, what should we rely on to reduce to zero the effect of subjective assessment? We tell what we pay attention to and what tasks we give candidates.


We also consider external candidates with specialized experience for the opening vacancies of second-line analysts, but internal employees are always given priority in translation. Thus, we increase the motivation of the guys: we have before our eyes a large number of examples of colleagues who have traveled from a first-line engineer to an expert analyst (see the article “ SOC for beginners. How to organize monitoring of incidents and responding to attacks in 24x7 mode ) or experienced SZI administrator. And in the labor market there is an obvious lack of competent information security specialists, and you will not find people with experience in SOC during the day with fire. It is a bit easier to find someone with knowledge of the tools used in Solar JSOC, but even here the choice is not rich.
')
Therefore, we do not just encourage the desire for development in the first-line engineers, but also constantly teach the children themselves. For example, among other things, we have implemented the practice of case review - the second line daily devotes a certain amount of time to recheck some of the incidents that were dismantled by the first. Of course, this does not apply to all incidents - it makes no sense just to duplicate someone else's work, and the flow of events is too large. First of all, we draw attention to high-critical incidents that have been closed as false. The goal here is not only to identify possible errors, but also to give first-line analysts advice on incident handling methods and develop their professional skills.

With proper attitude to work, the engineer is potentially ready to move to the next level after a year of work on the first line. Of course, everything depends on a particular specialist - someone “ripens” earlier, someone later, but empirically it was found that for about a year the engineer has time to join the team, get to the core of the work and gain practical skills in dealing with incidents - in general, become a full-fledged combat unit Solar JSOC.

Monitoring direction


A first-line engineer can develop in one of two directions — monitoring or operating a GIS. In the first half, the discussion will focus on monitoring specialists, who are guarding the information security of Solar JSOC day and night. We tell what qualities we are looking for in candidates and how we check their presence.

Toolkit knowledge

Firstly, the candidate for the second line should be perfectly fluent in the basic functionality of the SIEM system. It is important that from a certain moment SIEM ceases to be a program for an engineer that can collect and filter certain events and becomes a tool with which you can get the necessary information - a kind of Palantir, a seeing stone, with which SOC learns what happens in controlled customer information systems.



Interpretation of events

From this follows the second important requirement. A second-line engineer should be able to interpret technical information into a human-readable description that is understandable to non- IT professionals. Sometimes this requires supplementing it with relevant data from third-party sources. Compare, for example, the two incident alert options that can be sent to the customer:
The process ““ C: \ Windows \ System32 \ msiexec.exe ”/ i“ Z: \ Product Development Department \ Analytical Department \ _General \ Enterprise_Architect \ Sparx.Systems.Enterprise.Architect.13.0.1310.Corporate is launched on the NB-SIVANOV host. .Edition \ easetupfull.msi ”under the account s.ivanov.
or
Sergey Ivanov, Senior Analyst of the Product Analysis Group, installed the application software for modeling business processes Sparx Systems Enterprise Architect from the distribution kit located in the general directory Z: \ Product Development Department \ Analytical Department \ _General \ Enterprise_Architect.

Feel the difference? From a technical point of view, both options are correct, but, as always, the devil is in the details. The correlation rule worked to launch the msiexec.exe process, which most likely indicates the installation of a new software, which for a critical AWS is a reason for analysis by the information security officer. In the first notification, the engineer copied the technical data from the most important fields of event 4688 (host, process, UZ, process launch parameters) and, in principle, did everything correctly in accordance with the instructions.

BUT the second alert is what we expect from an experienced engineer. True technical information together with business intelligence carries much more useful information about the event to the customer.

To achieve this level of immersion, the engineer must investigate more than one hundred tickets, study the events of the main infrastructure sources, realize which way to begin to unwind the investigation, and which event patterns are interesting first of all.

Experience in-depth analytics

The gaze of a good first-line engineer should cling to non-standard things. A second-line specialist in the event of the detection of such atypical incidents can, if necessary, deviate from the instructions.

For example, on the domain controller, the antivirus worked on the malware in the file \\ tsclient \ C \ Users \ a.razumov \ Desktop \ shadow_miner_win.exe. Since this is a critical server infection, according to the instructions, such an incident should immediately escalate at any time of day towards a second-line analyst, service manager and customer. However, if the engineer takes note of the location of the potentially malicious file and conducts an in-depth investigation, he will understand that the user with the a.razumov US has connected via RDP to the domain controller, and a local “C” drive with the user directory was sent to the RDP session, which was tested by server antivirus. Such an incident, of course, also requires an investigation, but its level of severity is somewhat lower.


Attentiveness

An experienced first-line engineer should stay alert. For every hundred reported security events, there are 5-10 combat incidents. It is clear that all people are analyzing the positives, and it is human nature to make mistakes, but it is critically important not to miss these combat incidents, because SOC services are acquired for this purpose.

Here is an example: in the Solar JSOC arsenal, there is a fairly simple script for tracking the installation of new Windows services. Creating exceptions for this scenario is problematic, since the legitimacy of a service depends on the role of the server, launch rights, the context of its installation, etc. For this reason, the specialists of the first monitoring line have to look at a fairly large number of responses of this scenario, especially in the case of connected workstations, and conduct an initial assessment of the legitimacy of the service. And here it is very important that the engineer’s view is not blurred.

More recently, there was a situation where a trainee engineer turned to a responsible analyst with a ticket in which he was not clear which process a freshly installed service was launching. It turned out to be a PowerShell process with an obfuscated script argument. The installation of this service was the Kill Chain stage, and its goal was to pin down the malware that got to the host. Attentiveness trainee allowed to stop the infection on the infrastructure of the customer.

Or here's another example related to geolocating VPN entries. As a standard, a white list of countries / cities from which they are allowed to access is compiled for each VPN service user. One of the customers had compromised the domain ultrasound specialist of the IT department, and there was no two-factor authentication on the VPN gateway. On one “beautiful” night, the attackers connected via VPN using this UZ from a proxy server located in Germany. According to the profile, input for this employee was considered legitimate at any time of the day, but only from the IP addresses of Russia. Initially, it was agreed with the customer that we would notify him of an e-mail about the triggering of this scenario, even without duplication by telephone. However, the engineer decided to check the HR system and, seeing that the employee was not on vacation, escalated the incident by calling the service manager, who in turn notified the customer. The compromised UZ was immediately turned off, and the attackers did not even have time to really figure out where they even got, not to mention the fixing in the infrastructure. Obviously, if the customer’s specialists studied the alert only in the morning, having come to work, the story could have been completely different.

Experience with content

An experienced engineer should have an idea how the content works. Understanding the logic of the scenarios greatly simplifies the process of conducting investigations, since the specialist knows which particular chain of events the incident has led to. False Positive criteria are also often tied to the technical nuances of the correlation mechanism, and for their detection an engineer must be able to read the content.


Translation tasks

And here we have found an excellent candidate for the transfer to the 2nd line. The engineer has a positive “credit history” based on the results of the case review (sampling control of his conclusions on incidents), complies with the SLA when working with tickets, does not fall below a certain, consistently high level during investigations and has all the qualities listed above in the article. What tasks do we give to the engineer as transferable?

Usually we have several requests from customers to connect new types of sources. To do this, it is necessary to analyze the existing types of events, determine the best way to receive these events, create a connector to collect them. And this is a great task for the growth engineer!

The second task is directly related to the first: after connecting a new type of source, it is necessary to integrate its events into existing scenarios. For example, after connecting anti-virus software as part of MS SCCM System Center Endpoint Protection, event categorization was configured so that existing scripts for virus activity would work for this source. In the event that this is a completely new type of source, the engineer analyzes what types of attacks we can detect for events from this source and develops new scenarios.

The third task is usually the optimization of any content block of the SIEM system, i.e. correlation rules, filters, etc. Do not think that once written content remains unchanged. Either minor improvements or global rethinking of how it should be good are constantly happening :). Accordingly, the solution of this problem allows the engineer to dive deeper into the concept of writing content Solar JSOC.

And finally, the fourth task is related to the Threat Intelligence process. Primary processing of incoming data, the analysis of their relevance, the addition to real-time monitoring, and the second line are involved in accompanying indicators of compromise. The first line, in turn, is assigned the task of conducting a retrospective check on the indicators of compromise, and as part of the translation task, we want to hear from the candidate what he sees as the pitfalls and weak points in this rather technical process from a technical point of view below. So to say, we ask to give the person a look "out of the field".

Direction of operation


So, congratulations: we have successfully “raised” a 2-line monitoring engineer. But in Solar JSOC, in addition to monitoring, there is a direction of operation of information protection tools, on which 1 and 2 line of specialists are also built. What heights should the first line engineer reach to get level up?

Immersion in the subject

First of all, an experienced administration engineer should clearly understand what these or other PIS protects from. To do this, it must have an idea of ​​the main types of vulnerabilities of the protected system, methods of their operation and potential impact on the customer’s system.

A second-line specialist should know the main methods of implementing information security threats - the most common types of attacks on protected systems. In particular, we are talking about attacks from the Internet on the client’s protected web resources (which are primarily targeted by hackers), as well as on other resources accessible from the outside.

Toolkit knowledge

For successful opposition to complex and sophisticated attacks, the engineer must possess 100% of his tools. The idea is not new and coincides with the statement from the section on monitoring, but this does not become less relevant. Know what you work with. In Solar JSOC, engineers are armed with various IPS, WAF, AntiDDoS solutions, which, according to the vendors, work well out of the box. But we strive to ensure that our engineers possess the full functionality of these products: they know how to read default policies, edit pre-installed signatures, understand what lies “under the hood” of a beautiful graphical interface.


A classic case for a WAF or IPS administrator is the development of a custom signature for a fixed long-term attack on a protected web resource. Just this situation was recently encountered by our engineers: monitoring tools recorded a long brute-force account of the personal account of one well-known online store. Preset signatures for brute force attacks did not work, because they did not take into account the specifics of a particular web application. We carried out an on-line analysis of dangerous web requests and, on its basis, created a WAF signature blocking this activity. Hackers noticed changes in the system of protection of the attacked resource and several times slightly changed the contents of the requests, trying to circumvent the signature, but each time our engineers successfully tracked these manipulations and corrected the protection policy. Security vendors provide a similar service, but they may often lack responsiveness and immersion in the specifics of the protected system.

Retrospective analysis

However, there are unpleasant situations when the attack is missed, and the attacker, exploiting the vulnerability, hit the target host. After some time, the penetration was detected, access was closed, the Fornsica team analyzed the compromised hosts and found artifacts on the file system that are traces of the attack. This is great, but what to do with them next? Driving the indicators by means of a security scanner and self-written scripts is a standard procedure, but there is always room in street magic for street magic.

The administration engineer, with the help of his fellow foremeners, is trying to figure out how this attack could be detected and blocked even “on the way” on the exploited defenses. To do this, attempts are made to reanimate the remnants of a combat exploit to add its payload as a signature, the engineer notes specific attack nuances that can transform into monitoring triggers, software vulnerabilities successfully exploited during the attack are determined. In general, work is being done, without which it is impossible to learn from successful hacking. And this task requires an administrative engineer to administer a deep technical background and quite high qualifications.

Architectural approach

The second line engineer must be able to see two steps ahead. Any exploitation of a complex product is inevitably associated with changes in its configuration. Taking into account the specifics of the IB solutions, it may turn out that any changes threaten the loss of accessibility of not only the protection itself, but also the system being protected, and as a result - financial damage for the customer. That is why the information security administrator should have an architectural vision when preparing RFCs: be able to plan the work, predict downtime, assess the risks and impact of changes on other systems, and always keep a path of retreat in store - reliable methods of configuration rollback to the operating state of the product.

Also, do not forget that the protected systems are usually not static, but are in a state of continuous change. And once a well-developed protection policy can stop working adequately within a few days after its deployment. Therefore, an experienced second-line engineer should be able to build a technical process consisting of the steps of profiling legitimate activity, testing and refining an adapted policy, launching this policy in blocking mode into production, etc. We will not dwell on this; this topic is described in great detail in the article “ The problem of continuous protection of web applications. View from the side of researchers and operators ”.

Instead of conclusion

On the keyboard, the letters of the word “engineer” are beginning to fall, so we’ll wrap up :). Summing up, I would like to say that technical skills for a specialist in our field is very important, but first of all we pay attention to people with burning eyes who are “sick” with the information security topic, and in its “white” manifestation. Enthusiasm for their work allows the specialist to avoid stagnation and is the main driver for professional growth, which is important in our continuously changing industry. The first or second line is not so important. If a specialist enters this path with a great desire, success is guaranteed. Like cookies :).

Source: https://habr.com/ru/post/351200/


All Articles