Hi Habr! I continue the series of articles on how to build your cloud hosting in 5 minutes. In the last article we looked at tools that will help us solve the problem of service discovery. In this part, we will begin to practice, build a cloud and see how these tools behave in real life.

As before, all the work can be done by a regular programmer for 5 minutes, simply by running a set of scripts for Ansible , which I prepared especially for you and uploaded to GitHub .
')
Despite the fact that our cloud has become more complicated and now it uses a larger number of tools, it has become easier to build it. I completely rewrote a set of scenarios from past articles, deleted everything superfluous, simplified the rest as much as possible.

Content

• Part 0: Virtualization
• Part 1: Ansible, Docker, Docker Swarm
• Part 2: Service Discovery
• Part 3: Consul, Registrator, Consul-Template
• ...

Getting started

You must have Ansible and Docker installed on your client machine. There should be 3 servers with authorization by key and Debian 8.1 x64 on board ( you can use any other distribution kit, making minor changes to the script ).

Download a set of scripts or clone the repository:

» git clone https://github.com/vkozlovski/ansible-cloud-hosting » cd ansible-cloud-hosting » git checkout v2.x 

IP addresses

We open the stage file and replace the IP addresses with the IP of our servers in it:

 [dc1-cloud] 192.168.1.1 192.168.1.2 192.168.1.3 

If you want to build a cloud in several data centers, then simply add additional groups with the appropriate IP addresses ( by analogy with how this has already been done ):

Certification Authority

Now we need to generate keys for our certificate authority, which will sign certificates of Docker 'a clients and servers (for more details, see the first article ). To do this, I created a small assistant, so from the root directory of the project we execute the command:

 » make gen-ca 

We answer questions ( there are no specific requirements, you can specify any domain ) and remember the password. The password must be assigned to the variable certs_ca_password in the file group_vars / all.yml .

Certificates

In this step, you need to generate certificates for Consul . To do this, I also created a small assistant, so from the root directory of the project we simply execute the command:

 » make gen-consul-certs 

If you want to learn more about what exactly is happening at this step, you can read a great article on DigitalOcean .

The secret key

Now we need to generate a secret key that Consul will use to encrypt its network traffic. To do this, run the command:

 » docker run --rm --entrypoint "/bin/consul" progrium/consul:latest keygen L+3UkrkFeXHQBT97nTZI/g== 

The key must be assigned to the docker_consul_encrypt variable in the group_vars / cloud.yml file .

Settings for data center

The dc1.yml file in the group_vars directory contains a configuration specific to a particular data center. If you have more than one, you can create dc2.yml , dc3.yml , ... and fill them in by analogy.

 --- # docker-consul # first host in "my_name_dc" DC docker_consul_join: '{{ hostvars[groups["my_name_dc"][0]]["ansible_eth0"]["ipv4"]["address"] }}' docker_consul_dc: 'dc1' # docker-swarm-manager # first host in "my_name_dc" DC docker_swarm_manager_ip: '{{ hostvars[groups["my_name_dc"][0]]["ansible_eth0"]["ipv4"]["address"] }}' 

Consul

If you build a cloud in several data centers, then I have good news for you - Consul supports this out of the box. The only thing you need to do is add one IP from each data center to the docker_consul_start_join_wan variable:

Run

If you have reached this step - you will receive a reward. We launch the assistant:

 » make run 

Now you can "sit back and relax."



Owners of stools - take care of yourself.

After the magic is over, I recommend restarting all the machines.

Done!

Consul UI

Open the browser and go to any of the IP addresses of our machines ( http://192.168.1.1:8500/ ). If you set up multiple data centers, you should see a similar picture:

If you have one data center or you selected it from the list above:

Consul displays the list of services our cloud consists of. Green shows “healthy” services, yellow shows problematic ones ( in the last article I mentioned that Consul can check the health of services ).

Docker swarm

Let's check Docker Swarm ( you can read more about it in the first article ). Docker Swarm Manager is installed on the first IP address of each data center from the list in the stage file. For example from the list:

 [dc1-cloud] 192.168.1.1 192.168.1.2 192.168.1.3 [dc2-cloud] 192.168.2.1 192.168.2.2 192.168.2.3 #--- in all DC ---# # cloud in all DC [cloud:children] dc1-cloud dc2-cloud #--- everything in DC ---# [dc1:children] dc1-cloud [dc2:children] dc2-cloud 

these will be 192.168.1.1 and 192.168.2.1 .

In order to connect to the Docker Swarm Manager you need to run:

 » docker -H tcp://192.168.1.1:8000 --tlsverify=true --tlscacert=certs/ca/ca.pem --tlscert=certs/docker/cert.pem --tlskey=certs/docker/key.pem info 

You should see something similar:

 Containers: 13 Images: 12 Role: primary Strategy: spread Filters: health, port, dependency, affinity, constraint Nodes: 3 debian1: 192.168.1.1:2376 └ Containers: 5 └ Reserved CPUs: 0 / 1 └ Reserved Memory: 0 B / 519.2 MiB └ Labels: executiondriver=native-0.2, kernelversion=3.16.0-4-amd64, operatingsystem=Debian GNU/Linux 8 (jessie), storagedriver=aufs debian2: 192.168.1.2:2376 └ Containers: 4 └ Reserved CPUs: 0 / 1 └ Reserved Memory: 0 B / 519.2 MiB └ Labels: executiondriver=native-0.2, kernelversion=3.16.0-4-amd64, operatingsystem=Debian GNU/Linux 8 (jessie), storagedriver=aufs debian3: 192.168.1.3:2376 └ Containers: 4 └ Reserved CPUs: 0 / 1 └ Reserved Memory: 0 B / 519.2 MiB └ Labels: executiondriver=native-0.2, kernelversion=3.16.0-4-amd64, operatingsystem=Debian GNU/Linux 8 (jessie), storagedriver=aufs CPUs: 3 Total Memory: 1.521 GiB Name: debian1 

If this is so, and it should be that way, then all that remains is to congratulate you. If you have any difficulties - welcome to the comments.

We are testing

It's time to launch something in our steep cloud. And what could it be if not Nginx ? That's it!

Run:

 » docker -H tcp://178.62.232.38:8000 --tlsverify=true --tlscacert=certs/ca/ca.pem --tlscert=certs/docker/cert.pem --tlskey=certs/docker/key.pem run -d -p 80:80 -p 443:443 -e "SERVICE_80_NAME=http" -e "SERVICE_443_NAME=https" nginx 

You can read more about the environment variables that we pass here in the last article .

We look at what machine Nginx was running:

 » docker -H tcp://192.168.1.1:8000 --tlsverify=true --tlscacert=certs/ca/ca.pem --tlscert=certs/docker/cert.pem --tlskey=certs/docker/key.pem ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e96b351a857e nginx "nginx -g 'daemon off" 3 minutes ago Up 3 minutes 192.168.1.2:80->80/tcp, 192.168.1.2:443->443/tcp debian2/fervent_dubinsky ... 

Open in the browser http://192.168.1.2:80/ :


There is a contact. Now let's see if our service appeared in the Consul panel:

There were 2 services ( by the number of ports ): http and https ( we passed their names in the variables SERVICE_80_NAME and SERVICE_443_NAME ).

DNS

Let's now check the work of the DNS service, which Consul kindly provided us. To do this, run a container with Debian on any machine:

 » docker -H tcp://192.168.1.1:8000 --tlsverify=true --tlscacert=certs/ca/ca.pem --tlscert=certs/docker/cert.pem --tlskey=certs/docker/key.pem run -ti debian:testing /bin/bash root@2e68749354b2:/# 

See if our service is http :

 root@2e68749354b2:/# ping http PING http.service.consul (172.17.0.6): 56 data bytes 64 bytes from 172.17.0.6: icmp_seq=0 ttl=64 time=0.076 ms 64 bytes from 172.17.0.6: icmp_seq=1 ttl=64 time=0.118 ms 64 bytes from 172.17.0.6: icmp_seq=2 ttl=64 time=0.075 ms ^C--- http.service.consul ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.075/0.090/0.118/0.000 ms 

The full address of our service is http.service.consul , but we can also access via short http ( because we started Docker with the --dns-search service.consul parameter ). We can also use the longer version of http.service.dc1.consul indicating the data center ( if you want to reach a service from another data center, for example ). For more information about this, you can read in the official documentation .

Let's run a few more copies of nginx . Open another tab in the console ( we will need the container with Debian ) and run the command 2 times:

 » docker -H tcp://178.62.232.38:8000 --tlsverify=true --tlscacert=certs/ca/ca.pem --tlscert=certs/docker/cert.pem --tlskey=certs/docker/key.pem run -d -p 80:80 -p 443:443 -e "SERVICE_80_NAME=http" -e "SERVICE_443_NAME=https" nginx 

Docker Swarm is smart enough to run all 3 services on different machines (it looks like there are free 80 and 443 ports ). And if you try to run more copies of Nginx than your machines, he will report this:

 Error response from daemon: unable to find a node with port 443 available 

Now back to the container with Debian and put the package:

 root@2e68749354b2:/# apt-get update && apt-get install dnsutils --no-install-recommends 

Let's see whether new http services have appeared:

 root@866f410a5f18:/# dig http.service.dc1.consul. ANY ; <<>> DiG 9.9.5-12+b1-Debian <<>> http.service.dc1.consul. ANY ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17731 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;http.service.dc1.consul. IN ANY ;; ANSWER SECTION: http.service.dc1.consul. 0 IN A 192.168.1.1 http.service.dc1.consul. 0 IN A 192.168.1.2 http.service.dc1.consul. 0 IN A 192.168.1.3 ;; Query time: 4 msec ;; SERVER: 172.17.0.1#53(172.17.0.1) ;; WHEN: Thu Nov 26 10:22:41 UTC 2015 ;; MSG SIZE rcvd: 158 

Everything is working.

If you access your service by the name http , then the load will be distributed according to the Round-robin algorithm. If you now stop one of the containers with Nginx and rerun the above command, you will notice that it is no longer in the list.

Thus, the load is distributed only between "live" services. You also have the opportunity to take advantage of the health monitoring provided by Consul , in which case you can distribute the load only between “healthy” services ( do not confuse with just “live” ).

You can add and remove services dynamically and everything will continue to work without your intervention.

Conclusion

In this article, I wanted to tell you how to raise your personal cloud.

If you were attentive, and I am sure that this is so, then you noticed that we did not use the Consul-Template . I decided to open another part of my work for you and describe the process of automatic deployment of projects to our cloud in the next article. It took some time to find a suitable option for this purpose and now it saves us a lot of time.

What services “fill” your cloud is up to you. I worked on this configuration for a long time and did not encounter any problems.

That's all. Thank you all for your attention. Stable clouds and good luck to you!

Follow me on Twitter , I talk about working in a startup, my mistakes and the right decisions, about python and everything related to web development.

PS I'm looking for developers to the company, the details in my profile .