A miracle happened, a friend saved a friend's life

Until recently, this phrase reminded me of only one thing: a wonderful super-hero who learned to fly much earlier than all the Boomers, Supermen, and others like them.

But times change, and with them change associations. You understand that in order to change such a vivid impression, something really important had to happen.

Let's see what happened after all, and it’s not for nothing that my brain was reprogrammed.
')
The reason turned out to be banal: a month and a half ago, a new version of the Symantec Endpoint Protection 12.1 product was released, and one of the main innovations is the addition of reputation technologies to the product. In addition, there are many innovations in the product, but I personally, apart from my reputation, distinguish for myself a technology called Shared Insight Cache. Exactly these two technologies have more and more recently been squeezing me out of my head, dear one. Therefore, I will briefly try to describe how these technologies work in theory and in practice.

Chapter one. Mythical-theoretical

The logic of the first technology (reputational analysis) has already been described on Habré ( here ), so I will not go into details, but briefly tell you: Symantec collects information about what executable files exist in the world, when they were created, how many people are used, where they come from, etc. This allows, without analyzing the contents, to understand the category of a file: a dangerous file or not.

The second technology (Shared Insight Cache) allows scanning any files only once per infrastructure. Those. if there are identical files on several servers or workstations, then only on one machine the file will be scanned, and on all other machines scanning will not be performed. Immediately comes to mind the analogy with the technology of deduplication, relevant in the means of backup, archiving, only in this case we are talking about protection against malicious software. Isn't that great?)

I think that even an inattentive reader has already guessed why in my narrative we were talking about the phrase "a friend saved a friend's life." Reputation technology has helped, at the expense of information from other clients, understand whether I launch a dangerous file (or download) or not. Shared Insight Cache technology has reduced the use of resources that do not need to be spent on re-scanning.

It is important to note that the more mess is present in the infrastructure, the more tangible will be the result of the technology Shared Insight Cache. Who do not know the situation when administrators copy distributions to different servers at the time of installation, and then forget to remove them. Antiviruses scan everything, and on all computers. And the situation when users upload videos, music or photos to publicly available resources, and then the rest of them are spread across their workstations. And here antiviruses try, scan the same thing on all computers. The Shared Insight Cache technology makes it easier for the antivirus to live, while at the same time taking the load off the computer on which it works.

Chapter Two Dynamically practical

At the moment it may seem that a Symantec employee (whom I am) decided to write an article on the order of the marketing department and everything written beautifully only in words. The situation is somewhat different. My past in the form of work as a software tester still does not let me go, and I try to check in practice everything that in words looks very “appetizing”, but it is not known how it is implemented in life.

Therefore, I have done a few simple tests that anyone can conduct in their own environment, if there are doubts about what I have written in black and white.

Testing technology.

It was possible to choose many different test scenarios. The complexity and time costs for them could be different. I decided to go one of the easiest ways:
1. Found the most ancient and actively used virtual machine.
2. Cloned it, installed it on one of the machines SEP version 12.1, on the other version 11.
3. Scanned both cars several times. I will share the results now.

The first scan on a machine with version 11 took about two hours, while about 600 thousand files were scanned. Scanning on a machine with version 12.1 took about an hour and a half; during scanning at the level of 75 thousand files were taken as "trusted", i.e. not scanned. When scanning, the same policy was used (only for version 11, the Insight and Shared Insight Cache technologies were not used, since they did not exist in version 11).

The second scan was supposed to show the effectiveness of the Shared Insight Cache and ScanLess technologies. And showed!)

The second scan on version 11 was not very different from the first. But on version 12.1 the second full scan took less than 10 minutes. In this case, 100K files were scanned. The remaining files were not scanned due to the ScanLess technology, i.e. only modified files were scanned.

Here the question immediately arises, where does Shared Insight Cashe? He did not help ... The question is logical and it is better to check this technology on various machines. What I did in the next test.

Test Shared Insight Cache.
Here the test is extremely simple:
- Took several folders randomly selected from one of the directories on the C: drive, put them in the archive. The size of the archive turned out about 800MB.
- Scanned this archive on the client on which it was created. Scanning time was about one minute, I didn’t measure it exactly, because understood that the result would be different at times.
- Copied the archive to another computer with SEP 12.1 installed and scanned the archive on it. The result is about a couple of seconds.

At this decided not to finish his experiment. Suddenly it was a coincidence of events.
- Added a new file to the archive on the second computer. Scanned it again. Scanning time is about a minute.
- Copied the new archive back to the first computer, scanned it. Time again about a couple of seconds.

Now there is confidence. A miracle happened ... Or, of course, not a miracle, but just the technology worked. But to a friend, anyway, thanks!

By the way, for example, here is a screenshot from Perfmon, which can monitor the operation of Shared Insight Cache:


In this case, it is clear that the number of cache accesses is almost 20 times the number of files in the cache. This means that each file was scanned an average of 20 times. And this is all in the infrastructure of about 15 test computers. Accordingly, you can assume the possible benefits for infrastructures consisting of thousands of servers and workstations.

Chapter three Final narrative

"If a friend was suddenly both a friend and not an enemy, and so ...". This is the result I did not want to see in testing.
But to my great pleasure, I did not see him!)

Both technologies showed themselves in the best light and more than corresponded to my ideas about them. I hope that those of you who will work with the new version of Symantec Endpoint Protection will also like both technologies.

Take care of resources, they will still be useful to you!)