OpenGear - reducing business downtime using console server with Out-of-Band control

It would seem that a simple device is a console server, but today it is not just a stupid device, but a smart platform for creating an ecosystem for managing and automating the deployment of active network equipment. After all, it has the most important thing - a classic console, which gives a limitless set of possibilities.

Everything revolves around increased availability with high attention to safety and usability . If you are building an IT system in which you need to ensure 99.999% availability, then OpenGear solutions will help. In them you will find a classic console with centralized access and NetOps / DevOps modules for automation.

The solution includes:
')

• Failover to Cellular - Emergency switching to cellular communication - this feature allows you to maintain communication via 4G LTE or 3G channels when the main channel is not available.
• Smart Out-of-Band - operates independently of the main network, automatically detects and fixes problems. This reduces costs and minimizes downtime.
• Zero Touch Provisioning - simplifies the process of equipment deployment, as well as automates repetitive tasks, reducing the proportion of human intervention in the process, which in turn reduces the number of possible errors.
• Centralized management - allows you to easily access any active network equipment in 3 clicks, wherever it is located.

I hope you will be interested)

About Opengear

Opengear is a fast-growing company and is the most advanced in its sector. The company initially developed from Australia, but quickly became global and currently has a presence in all major regions from Asia to America. Development centers are located in Australia and Silicon Valley.

Opengear is a universal solution that can be applied in different sectors, ranging from education to retail and banks.

The company was founded in 2004. Currently has a number of new products that are regularly updated.

Opengear - as a console server

Opengear belongs to the class of console server solutions. All active network equipment connects to Opengear via the console port. It supports both classic RS-232 and modern USB. You can also connect Ethernet control ports.

Supported hardware: any.

Lineup and selection

In terms of functionality, all hardware is identical. Only the form factor of the equipment is different. Among the Opengear models there are both distributed compact infrastructure managers, and quite impressive console servers with up to 96 console ports in 1U. The set of ports differs, there are various combinations .

Opengear - in the role of smart out-of-band solutions

Out-of-Band (OOB) management allows you to make a network fault-tolerant, no matter what situation you are in. You will always have “Plan B”. Using this technology, you will be able to gain access to a remote site and diagnose a breakdown, thereby reducing the network downtime — increase MTBF, decrease MTTR.

Options for external communication channels:

• two communication channels: primary and backup for copper / optics;
• 4G LTE / 3G modem (2 SIM cards in small devices);
• V.92 analog modem for PSTN lines;
• Wifi

All these communication channels can be used to build a connection to the control system. No matter what NATOM the OpenGear device is in, it will reach your data center.

Built-in TFTP, DHCP servers and large amounts of memory allow you to store firmware and backup configurations directly on the device. Therefore, even if the connection is bad (for example, via a satellite modem) - you can always restore the connection.

Now we will consider the scheme of work of Opengear. Suppose there is a network and some remote site and you access this site through the main communication channel. In the case when the failure of the main communication channel occurs, you will be able to access the backup channel. Thus, you will not lose access to the remote site.


The main advantages of Smart OOB:

1. Remote access to network equipment whenever you want, even when your network is not available;
2. Providing situational awareness during disruptions;
3. Minimize network and IT infrastructure downtime;
4. Fast recovery from network and IT failures with “Failover to Cellular” (F2C);
5. Proactively detect problems before they cause disruptions by monitoring devices and the physical environment;
6. Centralized management platform.

Opengear - as a monitoring and control center

Opengear devices can manage smart outlets (PDUs), monitor UPS status, and even monitor the environment. For example, to detect penetration into the server, to respond to an increase in temperature or humidity.

External sensors:

• temperature and humidity (built into small devices), you can remote to measure the temperature of a particular point in the rack;
• penetration;
• vibrations;
• flooding;
• Any third-party sensor that can be connected to a relay.

Opengear supports more than 100 manufacturers and PDU and UPS models that can be connected via Serial Console, Ethernet or USB. APC, Eaton, Server Tech, etc. are supported, the full list is openly available .


Functions:

• monitor the status of UPS batteries and load PDU outlets;
• power management on the Hot Key via the console, out-of-band;
• automatic power control.

Embedded Open Source Tools:

• Network UPS Tools (NUT)
• Powerman
• IPMI Tool Kit

OpenGear can be integrated into any monitoring system via SNMP, SolarWinds, Zabbix, etc. The device has a built-in Nagios agent that can be integrated into an upstream system. It will monitor the servers at the remote site, and transmit information to a centralized system. This allows not to allocate a separate machine for the agent.

Automate reaction to events

Perhaps the most interesting feature for me is automation. OpenGear allows you to check the input signals (Check), to respond to these signals (Respond) and the response to the disappearance of this signal (Resolve).

For example:

• you can monitor the console output and when Kernel panic appears, reset the equipment by power;
• monitor the temperature and when it rises automatically send a command to shut down the servers, then the data storage. After the temperature recovers, turn everything back on. Thus, we will save data and equipment.

The platform is flexible and does not block on built-in actions, you can write your own scripts / plugin that will check something and perform the desired action.

Lighthouse - centralized access to any device in 3 clicks

Lighthouse is the central hub and management portal for Opengear equipment. Console servers themselves make “Call home” using secure LHVPN (OpenVPN with X.509 certificates).

Lighthouse aggregates information from all Smart OOB console servers:

• Current list of all console servers;
• Console Gateway: List search and connect to console ports via an HTML5 web terminal or SSH.

Lighthouse supports clustering mode, including geo-backup up to 10 nodes. Supports over 100,000 console ports under unified management. Lighthouse is essentially a virtual machine and runs on the desired virtualization system. You can connect not only Opengear hardware, but also third-party solutions, such as from Cisco or regular SSH / Telnet. Lighthouse supports the powerful RESTful API.

User experience at the height

Opengear hardware supports more than 50 simultaneous sessions per port, both via SSH and HTML5. This allows you to simultaneously restore complex systems, connect colleagues and vendor representatives. HTML5 does not use Java, so you don’t have to search for the required software version and monitor security updates. The web interface works by copying and pasting text - which greatly speeds up the work.

Calculation of equipment payback and cost of downtime

Everyone probably wondered how much is the installation of equipment of this type? To answer it, we will simulate the following situation: suppose a certain company, whose head office is in Moscow, has an extensive network of branches, one of which is located, for example, in the city of Yakutsk. The turnover of this branch is 1 million rubles per day, and in terms of 1 hour (at the operating mode of 16 hours per day) 62.5 thousand rubles per hour.

Suppose one day we lost contact with the branch due to a malfunction in the firmware after the update. The administrator, of course, is not around, ask a nearby competent person to reload the firmware, or there is no possibility, or does not allow access to the object. And then you have to send a staff member from the head office to fix the problem. Well, time has gone, we begin to count the time and financial costs. 1 hour in total to the airport and from the airport to the branch, 6.5 hours from Moscow to Yakutsk and 15 minutes to eliminate the malfunction itself, totaling 7 hours and 45 minutes. In monetary terms, it turns out $ 7689, which is several times more than the cost of such equipment. Thus, its cost pays off for one failure, which is quite a weighty fact. We summarize the calculations in a single table and this is what we get:

Otherwise, if we are able to contact the branch office using Opengear, the time to fix the problem will be 15 minutes. idle time, which is the financial equivalent in our example is $ 15,625 or $ 248.

Warranty and Reliability

In the article, we often said that this device allows access to a remote site even when the network is not available, but you can ask a logical question - how reliable is the Opengear equipment at all? By production of this equipment in it reliability is put 10 times more, than at usual network. This is achieved by testing component parts and increased requirements for it. The manufacturer gives a guarantee on the equipment for at least 4 years. You can talk endlessly about the reliability of the equipment Opengear, but we give one fact. At one of the exhibitions, at which the Opengear equipment was presented, a man approached the stand and became interested in the products presented. During the conversation, he shared the fact that in the company where he works, Opengear equipment has been installed, which has been running continuously for 12 years.

High security requirements

Since Opengear has such low-level access, security requirements are as high as possible.

I will list the main points:

• regular release of firmware - every 3 months. Intermediate firmware with the closure of vulnerabilities in the components used;
• firewall built-in, the default ban policy;
• IPSec, OpenVPN, PPTP, SSH, HTTPS;
• work behind the firewall, public / private APN, failover;
• strict isolation of access rights (role model) for each user for each port;
• LDAPS, TACACS +, RADIUS, 2-factor authentication;
• PCI DSS 3.0+ compliance;
• integration with SIEM systems;
• audit logs of all output and input on console ports;
• alerts for events, up to pulling the console cable.
• SSHv2 support and the ability to disable SSHv1;
• access to source codes is available;
• the ability to create your own firmware images (Firmware) and Linux kernel modules.

I will dwell on the last two points in more detail.

Access to source codes and create your own firmware

Perhaps you had a unique task and you had to write a special program for it or you wanted to delve into the source code of the solution used in the company. Opengear gives this opportunity, because not for nothing in the title is the word “open”.

Instructions for the Custom Development Kit (CDK) can be downloaded here , the source itself can be downloaded via FTP . You can put your own program next to it and it will be included in your own firmware.

NetOps / DevOps

Constant changes are a challenge for modern companies. Virtualization and digitalization continue to change the traditional ways of setting up and providing services in companies. IT environment is constantly changing, the speed increased by several orders of magnitude.

Opengear integrates into any workflow and any application. The NetOps Automation platform is designed around proven components: Docker, Ansible, and Git. Various runtime environments are supported: Python, Ruby, Perl, bash, x86 binary. This allows you to deploy a remote site from scratch - on hand you will have everything you need.

The basis of the NetOps Automation platform are:

• device OM2000, works directly on the site;
• Lighthouse, centralization.

Deployment order:

1. auto-tuning (Zero Touch Provisioning) OG devices using DHCP options;
2. auto check-in at Lighthouse;
3. detection of active network equipment connected to a specific port;
4. collection using Ansible configuration from Git;
5. applying settings on active network equipment.

Knowledge Base and Documentation

Documentation to the solution is open and available on the website . Images of Lighhouse and firmware can be downloaded on FTP . The knowledge base is available openly , you can easily clarify the issue through a search engine.

Examples of using

The solution is used in various fields:

• public clouds and data centers;
• finance;
• retail;
• telecom;
• industry;
• health care;
• education.

I will give an example of DigitalOcean , I think many people know what the company does. It guarantees its customers 99.99% uptime of the work provided by KVM virtual machines. To provide this, the Out-of-band solution must meet the criteria:

• provide reliable, permanently active backup connection;
• remove the cost of time and cost of departure of the engineer;
• provide reliable offline access to equipment in remote data centers;
• minimize the capital and operating costs of the backup connection solution.

Opengear successfully solved the tasks of DigitalOcean.


Thus, Opengear provides remote configuration, maintenance, and disaster recovery of critical IT infrastructure, power, and networking. No one else has the functionality equivalent to Opengear devices and centralized management through the Lighthouse. The article describes only a small part of the various use cases and functions, and shows the main directions.

If you are interested in the decision, you can contact us - the company Factor Group , distributor of Opengear. It is enough to write in free form at opengear@fgts.ru .

Authors: popov-as and dima_go .