Reflections on safety at work

Read the article Why I love Microsoft. Notes zombies , left a comment and wanted to write an article about the organization of a secure production management infrastructure. Not the activities of the enterprise, but directly production .

I want to warn in advance that this article has nothing to do with the topic of terrorism. And the picture is not about that.


The realities of our life are such that the majority ( about 100% more than half of the global market for DCS systems according to Wikipedia, and in fact will be more ), modern process control systems are powered by Microsoft OS. It's no secret that they are crammed with vulnerabilities. My nightmare is the following:
An unknown virus that destroys all defenses is declared, which simultaneously destroys all Windows. As a result, we get a full stop of the enterprise, at least for a day. If the automation was carried out sensibly (until now, all abnormal situations took place in a very regular mode), then nothing will explode, but the correct stopping of technological processes will just happen. During the day, the “candle factory” produces 1.5 million green pieces of paper. The cost of resources spent on stopping and starting is the same.

Example:

Approximately it looks like an emergency stop. The products in the apparatus and pipelines of the workshop (natural gas, gaseous ammonia, synthesis gas) are discharged and burned at the flare unit. Then white smoke. This excess steam is dumped through the same flare. After the reset, the workshop is stopped completely. When starting at the same flare unit, natural gas burns for several days at a rate of $ 200 per minute. The shop according to the plan works without stops 1-2 years. Then overhaul and again in battle.

')
Attention, the question is: can an enterprise file claims against Microsoft and demand compensation for the losses?
The direction, where the plant should go with its claims, is written in the license agreement:

23. LIMITATION AND EXCLUSION OF LIABILITY FOR DAMAGES AND DAMAGES. With the exception of monetary compensation, which can provide the manufacturer or installer, you can not recover any other damages, including consequential, special, consequential or incidental damages, as well as losses due to loss of profits.
This restriction applies to:
• any issues related to the software, services and content of third-party websites (including code), as well as with third-party programs;
• Claims for breach of contract, guarantee or other conditions, strict liability, negligence or other civil wrong, as far as permitted by applicable law.
This restriction applies even if:
• repair, replacement of software or monetary compensation does not compensate for all losses and damage or
• The manufacturer, installer, or Microsoft knew or should have known about the possibility of such losses and damages.
Some state laws do not allow the exclusion or limitation of liability for incidental or consequential damages. In this case, the above limitations and exclusions may not apply to you. They may also not apply to you if the legislation of your country does not allow the exclusion or limitation of liability for incidental, consequential or other damages and damage.

Approximately the same thing is written in the license agreements of the software that manages the technical processes, although it has a bunch of security certificates and other blah blah.
Ensuring the safety of such systems is a separate headache. Neighborhood with antivirus on one computer, they are experiencing extremely painful. In my practice, there is even an example of an automated process control system collapsed due to a bug in Microsoft Office.


This image (please do not count as advertising) shows what a heaped management system can be.

In practice, all this greatly reduces the overall reliability of the system, mainly due to the careless attitude of users to computer security.

To avoid trouble, you have to use all available methods of contraception:

• Cut off the process control system from the outside world and the inquisitive mind of the operators. As a consequence of this, you can forget about all the advantages of integration (remote control and other amenities).
• To account for raw materials and products, separate controllers are installed that do not affect the process control.
• When urgently needed - access to the outside world through the gateway station with antivirus, firewall and complex passwords.

In principle, the controllers are able to control the system themselves, but if the top level of control is still cut down and refuses to turn back on, you will have to pull out the cord and extrude the glass and press the emergency stop button. It is always implemented in hardware, and acts directly on the lower level.

Let me turn to more specific examples:

When the product from the tap drips regularly, the last scanword is already an hour full, and the boss will not come soon, there is an overwhelming desire to entertain your loved one with something. And at hand is the whole computer used for such a primitive occupation, as the management of a boring process technology. This is where the search for a circumvention of the restrictions set by caring administrators begins. With the help of various key combinations and mouse poking, sometimes there are quite unexpected ways to launch unnecessary software. And there are flash drives and live-cd. Reliable protection against such manipulations is only a system unit locked in a cabinet. In the event that the suffering, in spite of the obstacles, still gets to the coveted scarf, he is waiting for a nice surprise. The printer prints A4 paper as follows:


Yes, I forgot to clarify, it will print the printer, located in the office of the foreman.

If someone has a question: “Why is there such a leaky system?”, Then the answer is simple - what we bought, so we work.

Prohibiting physical access to equipment is very important:

Servers and controllers are protected from a voltage drop as follows:
Redundant servers, redundant controllers, power backup from two inputs, 2 UPS at 30 kW. But 4 “small” miscalculations happened:

• Automatic machines at the power inputs of servers and tsisok are only 6A;
• The key from the cabinet in which this equipment was forgotten;
• The presence of a night shift electric kettle;
• Last backup was a week earlier.

As a result, the work week was lost.

I would like to thank you separately for the implementers who provided each user of the network with a small portion of Win32.Neshta. It is usually implied that the computer with the control system is pristine, and it can be safely connected to any network.

What is the moral of this fable? Yes, in fact, it is not. More precisely, there is: do not skimp on too much, at first glance, equipment.

• An additional computer with a good anti-virus will act as a gateway to the external network.
• Cabinets for servers and system units ensure their safety.
• About a separate room for the controllers do not even speak.
• Backing up all critical control equipment will protect against iron failure.
• Well and still nobody canceled backups. It is advisable to store them not on those disks that are backed up.

I, for my part, make every effort to ensure the reliability of control systems. For example, when viruses were rampant on the network, I only found out about this when admins chopped off the Internet gateway for prevention.
UPD: clarify what is meant by ASUTP.
This is a set of equipment controllers + visualization system with whistles and other things.
The controller is such a piece of iron, which only an explosion can prevent the stable operation.
SCADA - dispatch control and data collection. Can collapse from any careless sneeze.
I will cite statistics for the last 2 years:
Average uptime of windows machines is 1 year. During the repair, the computers are dust-free. Server under Windows Server 2003 with uptime 2 years. Not a single case of unauthorized access. To come to this it took time, thinking and some cones. To heaps had to get rid of crutches and props left by the predecessor.
Now about the OS. I don't care under which operating system the scud will spin, just to be stable.
An RTOS, such as QNX, is somewhat different. Their main niche is embedded systems.
I will give options for systems that are used in chemical production:
Experion PKS - windows
Experion TPS - windows
Emerson Delta-V - windows
Foxboro - unix or in windows under the emulator (which is much more often)
Siemens SIMATIC - windows
ABB System 800xA - windows
YOKOGAWA CENTUM - Windows
IFix - windows
Intouch - windows
If possible, I want to have a controller and a system of one manufacturer with convenient tools. I do not sell RTOS, Linux and Unix. It is just that the main global players in the automation market mainly promote distributed control systems operating under windows.