Private cloud for the Internet of Things

Greetings, Habr!

We are a team of IT architects from IBM, which is engaged in the creation and promotion of industrial solutions for various industries. In our spare time, we explore new technologies, experiment and try out the best practices.
')
Recent years, the subject of the Internet of Things is very popular. Our customers ask about universal box solutions and platforms that can be located locally in their Data Processing Centers. Most importantly, it is expected that on the basis of such "boxes", all new services and applications should be developed and launched very quickly, or rather instantly or "just yesterday."

We asked the question: “What if we put together something similar on open and publicly available technologies?”

Much has already been said and shown about the use of Docker containers and management tools based on Kubernetes . However, it has not yet been possible to see how all these advanced technologies can be used on an industrial scale for business tasks, specifically under the Internet of Things. So let's fill this gap. The concept of microservices, as well as fashionable development approaches (DevOps), application integration and deployment (CI / CD), will be considered a slightly different task and set aside for later.

In this article we want to talk about the preparation and installation of one of the key components - IBM Cloud Private (in fact, Docker / Kubernetes industrial assembly and a little more), which can act as a conditional infrastructure core when creating a local platform of the Internet of Things. Ask why him?

Because we promised open and publicly available technologies (free community version and community support): hub.docker.com/r/ibmcom/icp-inception

IBM Cloud Private Overview

IBM Cloud Private (ICP) is a private cloud platform for local deployment and operation. ICP is an integrated environment that allows you to design, develop, deploy and manage local applications created with the container architecture. The ICP delivery includes a Kubernetes based orchestrator (K8s), a local image repository, as well as a management console and monitoring tools.

What is a private or private cloud?

The private cloud is a cloud computing model that runs exclusively for one organization, usually on its territory and resources. Such a cloud can be managed and maintained by its own specialists, as well as by employees of third parties. Such a platform can be placed both behind the company's firewall and outside it. A private cloud provides the benefits of a public cloud, including rapid deployment and scalability, as well as ease of use, management and flexibility. At the same time, the local version provides greater control over the system, increased productivity, accurately predictable costs, increased security requirements and flexible management options depending on the needs of the organization. Private cloud allows you to customize it directly for specific business tasks and customer security requirements.

Before proceeding with the installation of ICP, we define the terminology:

Master node

• Monitors and manages the K8s cluster.
• kubectl: command line tool for cluster management.
• REST API: interface for working with nodes.
• etcd: is a permanent and accessible keystore, it is used as storage of K8s backups for all cluster data.
• kube-scheduler: this component monitors newly created containers (pods) that do not have an assigned node, and selects the node to launch them.
• Usually, 3 or more major nodes are required for stable operation and resiliency. In our case, to save resources, we will install all the necessary ICP components on one dedicated server (Single Node).

Work node

• The working machine in the K8s formerly known as the Minion. This node can be any of the hosts that receives tasks from the main node and also controls the process of launching containers on it. Depending on the cluster settings, the work node can be either a virtual machine or a physical server.
• The following components can be run on each node:
• kubelet: is the main work node service. It monitors the containers that have been assigned to its node (either apirusver for a secure connection, or through a local configuration file).
• kube-proxy: displays the services defined in the K8s API on each node, supporting network rules on the host and performing connection redirection.
• docker: an environment for running and managing containers.
• supervisord: This is a lightweight monitoring process and management system that can be used to maintain the kubelet and docker.
• fluentd: This is a service that helps ensure logging at the cluster level.

Containers (containers)

• Application running in an isolated environment.
• The set of executable processes and the run image of the program.

Cocoons (pods)

• A pod is a group of one or more Docker containers, with shared storage / network and a specification of how to run containers on a working node.
• Each Pod has its own IP.
• Pod divides the PID namespace, network and host name.

Replication controller

• Provides availability and scalability.
• Responsible for controlling the number of deployed Pods.
• Uses a template that describes what each Pod should contain.

Labels

• Metadata assigned to K8s resources, for example: containers, services.
• Key-value pairs attached to objects for identification.
• These parameters are critical because they are based on a cluster request for resources with certain labels.

Services

• A collection of containers set as an endpoint.
• Information about the status of the K8s cluster and network settings that apply to all work nodes.
• With the right configuration, Pods created by the replication controller will be automatically registered as a corresponding service.

Secret Information (secrets)

• Confidential information to be read or used by containers.
• Special sections (volumes) of the disk drive are mounted automatically so that the containers can read the contents.
• Each entry has its own way.

Architecture

Below is the high-level ICP architecture which lists the main components broken down into main, workers and proxy nodes. It is worth noting that in this article the main emphasis is placed on a bunch of Docker containers and the Kubernetes management environment. These open source components are key and fundamental to the ICP platform. However, to fully work with ICP, you also need to know Helm, the manager of software package management. K8s is used to deploy and host application instances that are embedded in the Helm schemas; they in turn refer to Docker images. The Helm diagrams contain information about the details of your application, and the Docker images contain all the software packages that are required to run your application.



For detailed information about the architecture and components of the ICP platform, you can use the link to the documentation in the IBM Knowledge Center .

Training

In order to comfortably complete the ICP installation in 20-30 minutes, the following minimum hardware resources and Internet access are needed:
• One host (physical or virtual)
• CPU = 8 cores (or virtual cores)
• RAM = 10 GB (in fact, it may take up to 14 GB)
• Disk = 40 GB (as always faster on SSD)

Note: if you have a little more free time and a little less hardware resources, then somewhere in 40-60 minutes you can install ICP on 1 virtual core i5 and 10 GB of RAM.

Installation

Ubuntu boot

Download the Ubuntu Server distribution version 04/16/05 . Perform a standard OS installation on a virtual or physical machine with the above characteristics.

Getting an IP Address

After the OS installation process is completed, on the command line, find the IP address of your virtual machine using the following command:

ifconfig 

You need to write / remember the Ethernet IP address (inet addr), as well as Bcast and Mask. These parameters will appear after the execution of the corresponding command:

Figure 1. An example of the execution of the command "ifconfig"

SSH connection

If you used the “Easy Install” function when setting up a virtual machine, then after you have fully installed the Ubuntu Server, you need to install the OpenSSH server. You can do this with the following commands:

 apt install openssh-server sudo systemctl enable sshd 

After executing the appropriate commands, you will be able to connect to the virtual machine using SSH. Depending on the OS, you can use different connection methods. For Windows, you can use the popular PuTTY tool, for MacOS, you can use the ssh command in the terminal.


Figure 2. An example of connecting via PuTTY for Windows

In the next step, you need to set a password for the super-user (root):

 sudo -i passwd 

Figure 3. Setting a new password for the root user

Network Interface Update

After we configured the ability to connect via SSH and updated the super-user password, we need to configure the network interface to determine the static network parameters. This is done with the following command:

 nano /etc/network/interfaces 

We update the settings of the network interface (primary network interface) based on the information obtained during the execution of the ifconfig command, our example looks like this:

 auto ens33 iface ens33 inet static address 192.168.175.7 netmask 255.255.255.0 network 192.168.175.0 broadcast 192.168.175.255 gateway 192.168.175.2 dns-nameservers 192.168.175.2 8.8.8.8 

Note: It is important to correctly specify the DNS server, since Internet access is necessary for the subsequent installation of ICP components.

Figure 4. Configuring the network interface

Save the updated file with the following actions: Ctrl + X, Yes, Enter.

To simplify the login to the virtual machine with super user rights (root), you need to make changes to the SSH service configuration. To do this, run the following command:

 nano /etc/ssh/sshd_config 

Next, find the access permission string and edit it accordingly:

 PermitRootLogin yes 

Figure 5. Updating SSH configuration

Save the file as in the previous step: Ctrl + X, Yes, Enter.

After making all the changes, to apply the new settings, it makes sense to completely restart the virtual machine or restart the corresponding services (networking and ssh), for example, with the following commands:

 shutdown -r 0 

or

 service <service_name> restart 

After restarting the machine, you must log in again through SSH.

Component Update

Update Ubuntu Server using the following command (the command should run as root):

 apt update 

Note: If an error occurs during the upgrade: "The repository 'cdrom: // Ubuntu-Server 16.04.5 LTS _Xenial Xerus_ - Release amd64 (20180731) xenial Release' does not have a Release file.". You must run the following command:

 sudo nano /etc/apt/sources.list 

And comment out the line:

 deb cdrom:[Ubuntu-Server 16.04.5 LTS … 

Figure 6. Correcting a possible error

Next, save the file, as it was done earlier: Ctrl + X, Yes, Enter

After correcting a possible error, rerun the update command and as a result you will need to see something like this:


Figure 7. Update Ubuntu

Installing IBM Cloud Private

Before you install the ICP, you need to edit the hosts file:

 nano /etc/hosts 

Replace line 127.0.1.1 ubuntu with your IP address and any suitable hostname.

Make sure that the host name and alias in the string are separated by a tab or space:

Figure 8. Changing the hosts file

Save the file: Ctrl + X, Yes, Enter

The following 2 commands allow you to upgrade your system and install additional modules.

 apt update apt-get install apt-transport-https ca-certificates curl software-properties-common python-minimal jq 

Trace terminal output for errors:


Figure 9. Installing Optional Components

Increase the maximum virtual memory with the command:

 sysctl -w vm.max_map_count=262144 

K8s will not start with swap enabled. You can disable it with the command:

 swapoff –a 

Note: the two previous commands will work in the current session. When rebooting, these parameters will be lost and you need to register them again.

The use of these parameters when booting the OS can be enabled with the following commands:

 sed -i '/ swap / s/^/#/' /etc/fstab echo "vm.max_map_count=262144" >> /etc/sysctl.conf 

Adding GPG key

Use the following command:

 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - 

Add repository to install Docker

To install the repository, you must run the following 2 commands:

 add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" apt-get update 

Docker installation

Use the following command to install Docker (Docker version 17.12.1, the maximum supported version for IBM Cloud Private 2.1.0.3):

 apt-get install docker-ce=17.12.1~ce-0~ubuntu 

Figure 10. Docker installation process

You can check the Docker operation by the following command:

 docker version 

Figure 11. Docker check

IBM Cloud Private Download

To download the ICP-ce image (ommunity Edition) from the dockerHub resource, you need to use the Docker toolkit. You can do this with the following command:

 docker pull ibmcom/icp-inception:2.1.0.3 

Figure 12. ICP loading

IBM packaged all major components required for installation into one package (note that you can change the version to a newer version if necessary). The pull command will load the image into the local file system (the image will be launched to install the ICP).

Create a directory and go to it:

 mkdir /opt/icp cd /opt/icp 

Copy the ICP package to this directory:

 docker run -e LICENSE=accept -v "$(pwd)":/data ibmcom/icp-inception:2.1.0.3 cp -r cluster /data 

Note: this Docker toolkit command executes the Linux copy (cp) command from the specified directory (-v parameter). This will create a cluster directory in / opt / icp with all the necessary files.

Setting SSH keys

Create new ssh keys in the / opt / icp / cluster directory:

 ssh-keygen -b 4096 -f ~/.ssh/id_rsa -N "" cat ~/.ssh/id_rsa.pub | sudo tee -a ~/.ssh/authorized_keys systemctl restart sshd cp ~/.ssh/id_rsa ./cluster/ssh_key 

Check that the key is created and located in the / opt / icp / cluster folder:

 ls /opt/icp/cluster 

Figure 13. The contents of the cluster folder

ICP Setup

We add the IP address of each node in the cluster to the / opt / icp / cluster / hosts file (in our case, we use the same IP address for each component that we specified in / etc / hosts). We do this with the following command:

 nano /opt/icp/cluster/hosts 

Figure 14. The contents of the hosts file

Save the file: Ctrl + X, Yes, Enter

Detailed information about the installation parameters can be found in the configuration file:

 more /opt/icp/cluster/config.yaml 

We do not recommend changing the default installation settings.

Note: The two most important installation files are hosts and config.yaml, they describe the ICP installation options. In a multi-worker configuration, you must specify a list of IP addresses in the worker section. When you install the current version of ICP in Single Node mode, the IP addresses change only in the hosts file.

ICP installation

Go to the cluster directory and run the installation:

 cd /opt/icp/cluster docker run -e LICENSE=accept --net=host -t -v "$(pwd)":/installer/cluster ibmcom/icp-inception:2.1.0.3 install 

Depending on the resources available, the installation can last from 20 to 60 minutes. If errors appear (red messages), make the appropriate changes to the configuration files. Sometimes the installation may slow down a bit due to attempts to reconnect to components that have not yet started (for example, during the launch of Cloudant or IAM).

The end of the installation should look like this:

Figure 15. The result of the ICP installation

In case of errors, you may need to uninstall ICP:

 docker run -e LICENSE=accept --net=host -t -v "$(pwd)":/installer/cluster ibmcom/icp-inception:2.1.0.3 uninstall 

Before proceeding to the next step, wait about 5 minutes so that all ICP components can start successfully.

By the link indicated after the installation is completed, enter the ICP console (admin / admin):

 https://<ip_address>:8443 

Figure 16. Login page to control panel

ICP Welcome Page:

Figure 17. Web control panel interface

At the top right, select the “Catalog” menu, a list of available applications will appear:

Figure 18. Component Catalog

This directory contains a list of applications and software, the description of which you can study at your leisure, but so far it is not necessary to rush to deploy them.
We have yet to learn how to properly handle the newly installed ICP (not only via the Web interface, but also the console in the form of a CLI - Command Line Interface), connect permanent storage (persistent storage) and make small settings.

As a conclusion

Let's sum up.

We successfully deployed the IBM Cloud Private community edition (version 2.1.0.3) on a single host in a virtual machine running Ubuntu Server 16.04.5. At the moment we have personally tested the installation on VMware and Hyper-V hypervisors, it works exactly on XenServer and there should be no difficulties for VirtualBox and KVM.

To whom one cannot wait to plunge further into the study of ICP, that is, an excellent selection of laboratory work of our colleague here: github.com/phthom/IBMCloudPrivate

From the upcoming steps - after a fresh installation, it remains for us to tweak the ICP a little and begin to fill it with something useful and close to practical tasks.

If the Habr community supports this initiative in a positive way, as our research and experimentation progresses, we will fill the platform core with the necessary components for collecting telemetry, storing data, processing, analyzing, building forecasts, building external program interfaces and creating custom applications. Accordingly, developing a series of such articles.

We are waiting for your feedback and suggestions!

Also, our like-minded people from AIXportal.ru prepared a wonderful video clip on installing IBM Cloud Private on their Youtube channel; it will be more convenient for someone like this: youtu.be/eQwOOTzjM24

Links and useful materials

• Product page in Russian
• Who wants to practice English, the book - “ ICP for Dummies ” 2018. (40 pages)
• Official ICP 2.1.0.3 documentation at the IBM Knowledge Center
• ICP Open Community on IBM developerWorks
• ICP recipes for developers