KlusterKit

KlusterKit: an open source toolkit for simplifying Kubernetes deployments and working in physically isolated local environments

Today we are happy to announce that Platform9 opens the source code for Klusterkit, a set of three tools, under the Apache v2.0 license on GitHub.

Our customers roll out software in private data centers that are often not connected to the Internet (for security reasons or for other reasons). These large companies want to use the advantages of Kubernetes and modernize their applications while rolling them out at different data centers, which often have no connection with the outside world. And here comes Klusterkit, which simplifies the delivery and management of K8s clusters in physically isolated environments.

Klusterkit includes three independent tools that can be used together or separately to manage the life cycle of a Kubernetes production cluster:

1. etcdadm , CLI for simplified cluster management etcd.
2. nodeadm , CLI for administering nodes, which complements kubeadm and deploys dependencies needed by kubeadm.
3. cctl , a cluster lifecycle management tool that accepts the Cluster API from the Kubernetes community and uses nodeadm and etcdadm to easily deliver and maintain high-availability Kubernetes clusters in local and including physically isolated environments.

Together, these three tools perform the following tasks:

• Easy delivery and management of high availability etcd cluster and Kubernetes control panel in physically isolated local environments via the Cluster API.
• Recovery of the cluster control panel after failure using backup etcd.
• Packaging of all artifacts required for the delivery of Kubernetes in physically isolated environments.

Klusterkit features

• Multiple wizard support (HA K8s cluster).
• Supply and manage secure clusters etcd.
• Work in physically isolated environments.
• Support sequential update and rollback.
• Flannel (vxlan) as CNI for the backend; plans to support other CNI.
• Backing up and restoring etcd clusters after quorum loss.
• Protection of the control panel against low memory and CPU time.

Klusterkit Solution Architecture

For fault tolerance and simplicity, Klusterkit uses a single cctl-state.yaml file to store Kubernetes cluster metadata. Through the cctl CLI, you can manage the life cycle of a Kubernetes cluster on any machine that has this state file. This could be an operator’s laptop or any other computer that belongs to the Kubernetes cluster.

Cctl implements and calls the cluster-api interface from upstream as a CRUD operation library on clusters. It uses ssh-provider , open source cluster-api provider on Platform9's “bare iron”, which in turn calls etcdadm and nodeadm to perform operations on the cluster.

How to use Klusterkit and its components:

1 - Any of the three tools can be easily built with the go get command:

go get -u github.com/platform9/cctl go get -u github.com/platform9/nodeadm go get -u github.com/kubernetes-sigs/etcdadm 

2 - Then these executable files can be packaged and copied to the target machines where the high-availability cluster Kubernetes should work. Place the nodeadm and etcdadm files in the version directories:

 cp $GOPATH/bin/nodeadm /var/cache/ssh-provider/nodeadm// cp $GOPATH/bin/etcdadm /var/cache/ssh-provider/etcdadm// 

3 - If it is necessary to orchestrate the Kubernetes cluster locally, in a physically isolated environment, the necessary dependencies can be easily downloaded in advance on a computer with Internet access by the nodeadm and etcdadm download command. Then loaded items (i.e., kubelet and kubelet for systemd unit file, executable CNI files, kubeadm file, all container images, including Kubernetes, keepalived image and systemd file, etcd container image and corresponding configuration files) can be easily copied to physically isolated hosts along with cctl, nodeadm and etcdadm. (See wiki for details.)

4 - When everything is in place, you can create the first cluster Kubernetes with a couple of commands:

- First create credentials for the cluster.

 $GOPATH/bin/cctl create credential --user root --private-key ~/.ssh/id_rsa 

- Then create a cluster object. –Help brings up a list of supported parameters.

 $GOPATH/bin/cctl create cluster --pod-network 192.168.0.0/16 --service-network 192.169.0.0/24 

- Finally, create the first machine in the cluster.

 $GOPATH/bin/cctl create machine --ip $MACHINE_IP --role master 

More documentation on GitHub .