In this article I will tell how you can run the kubernetes master inside the LXC container.

This method works quite well with Proxmox and can be considered as an alternative to the classic deployment of kubernetes with several wizards.

Why proxmox?

Because proxmox out of the box provides many useful features.

Proxmox includes high availability features, migration, automatic backups, access control and everything is available through a simple graphical interface.

You can also deploy the simplest configuration with one master kubernetes, high availability will be ensured by proxmox itself.

This method is perfect for deploying test clusters or small clusters.

For larger deployments, it is recommended that you put etcd in separate containers that will use fast local disks and cluster them.

Nevertheless, the kubernetes-master itself can still remain an ordinary ha-container, it does not need to have fast storage.

In addition, if you are reading this article, I assume that in most cases you already have some kind of infrastructure on Proxmox, and perhaps you want to have a single interface for managing your services.

Why LXC?

Kubernetes will run without problems inside a normal virtual machine. But LXC containers provide the flexibility that is not available when using conventional virtual machines.

In fact, LXC containers do not provide complete isolation of containers from the host; on the contrary, all processes inside containers are started as normal host processes, just in a separate namespace for them.

This method gives you good performance but imposes some limitations in our case.

About these restrictions and how to cope with them I will discuss in this article.

Configuration

Since, by default, containers are not allowed to load kernel modules themselves, you must configure them to load directly on hypervisors.

We will use the overlay driver for the docker, so this is all we need:

 echo overlay >> /etc/modules 

Now we need to add more privileges for our container in order to allow it to run other containers inside, add these lines to your container config:

 lxc.apparmor.profile: unconfined lxc.cap.drop: lxc.cgroup.devices.allow: a lxc.mount.auto: proc:rw sys:rw 

Starting with version v11.0, kubelet requires shared mode for all mounts from the host.

This dirty hack will allow you to achieve this, inside the LXC container, run:

 echo '#!/bin/sh -e mount --make-rshared /' > /etc/rc.local 

This action will add the mount --make-rshared / to /etc/rc.local and will launch it every time the container is loaded.

Also, if you plan to use HA-manager in proxmox, know that at the moment there is an unpleasant bug # 1842 , which forcibly kills container processes during migration, which can spawn zombie processes or even block your storage.

This is not good, fortunately there is a simple solution:

 sed -i 's/forceStop => 1/forceStop => 0/' /usr/share/perl5/PVE/HA/Resources/PVECT.pm 

In addition, you can add the following options for your docker:

 --storage-driver overlay2 --iptables=false --ip-masq=false 

Copy docker.service from /lib to /etc to override its parameters:

 cp /{lib,etc}/systemd/system/docker.service 

Now add these options to the ExecStart section.

That's all, after these steps, the standard kubeadm installation should work without problems.