Winning Broadcast Flood on the Corporate Local Area Network

Symptoms

It happened in our organization, a terrible thing - the network worked, worked, and suddenly, it seemed, for no particular reason, it began to work unstably. It all looked very strange (first encountered the problem of a subject) - some computers on the network (few of them) stopped receiving IP addresses (they write in the logs that they did not receive a response from DHCP), and some in the morning, others from lunch they call, they worry, and we cannot understand anything.


If this is a hardware failure, then it must, according to all canons, be located in any one place, or at least appear more massively (such as with a ring on Ethernet), and then there are some rare bursts (approximately 5 out of 300), but in general everything works. A more detailed analysis of the geography of sick computers showed that they are located on switches 3 or more queues from the gateway (Figure 1).


Figure 1. The geography of the problem.
')

Search and identification

The hypothesis of a problem with iron was not immediately abandoned - the downstream switches were disconnected, and they seemed to receive a more or less short-term improvement, but the problem did not completely disappear.

A version naturally emerged that this is a kind of virus on a PC - it prevents them from receiving an IP address. She was rejected after, as the address did not receive a network printer. As it turned out in vain (more precisely, almost in vain).

In parallel, they tried to analyze the traffic, but due to the inexperience of the experts, only DHCP traffic was analyzed.

So, the first few days did not bring a solution to the problem. I had to expand the field of vision sniffer. And at this moment the cause of the problem was discovered - when analyzing all the broadcast traffic, it turned out that more than 80% of requests are looking for, some server - in the sense of the same.

How. later we learned from the Internet, this problem is called broadcast flood.
Oh ... if you knew about it before.

It turned out that a certain service called “PcounterPrint” is very hysterically trying to find its server, which strangely enough is not there. The service conducts an audit of the printing of employees of the corporation, and is known in the world as FollowMe Printing. As it turned out later, the server of this service was successfully decommissioned, of course, without any notice, by higher-level corporate system administrators.
In fact, users' PCs acted as bots for the DDOS attack of our network equipment.

The case is left for strangling this service on users' PCs.

Bulk removal

For good, it was necessary to give this task to the above-described system administrators, but then it’s also interesting and here, after a 25-minute search on the Internet, a script was born in the power-shell:



The $ computers variable gets the list of computers from the file, the script sequentially goes around all the PCs from this list, and disables the ill-fated service on them.

Next, we check the broadcast traffic with a sniffer, if someone is left - we correct the list, and execute the script again, and so we do several iterations until the malicious traffic is completely removed.

Naturally, after that the network has worked steadily.

findings

As they say in one well-known, pre-transferable joke: so for the same you need a candelabra on the head ...

In general, the administrative conclusions, I will not write here, although it is mainly they who suggest themselves.

From a technical point of view, there are several measures for the prevention of this trouble:
1. Segment a network into several virtual networks
2. Reduce the network depth using the first item
3. Install smarter switches

Although these events are controversial, of course: is it necessary, because you have to spend time and money, especially since the staff is now familiar with this situation and subsequently will be able to quickly defeat it, although who knows ...