Writing operator for Kubernetes at Golang

Note trans. : Operators (operators) is an auxiliary software for Kubernetes, designed to automate the execution of routine actions on cluster objects at certain events. We have already written about the operators in this article , where they talked about the fundamental ideas and principles of their work. But if that material was more like a view from the exploitation of ready-made components for Kubernetes, then the translation of the new article now proposed is the vision of the developer / DevOps engineer, perplexed by the implementation of the new operator.



This post with a real-life example, I decided to write after my attempts to find documentation on the creation of an operator for Kubernetes that went through the study of code.
')
An example that will be described is as follows: in our Kubernetes cluster, each Namespace represents the sandbox environment of a team, and we wanted to restrict access to them so that teams could only play in their sandboxes.

You can achieve the desired by assigning the user a group that has a RoleBinding to specific Namespace and ClusterRole with the right to edit. The YAML view will look like this:

 --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: kubernetes-team-1 namespace: team-1 subjects: - kind: Group name: kubernetes-team-1 apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: edit apiGroup: rbac.authorization.k8s.io 

( rolebinding.yaml , in raw )

You can create such a RoleBinding manually, but after overcoming the mark in a hundred namespaces, this becomes a tedious task. This is where the Kubernetes operators help - they automate the creation of Kubernetes resources based on changes in resources. In our case, we want to create a RoleBinding when creating a Namespace .

First, we define the main function, which performs the required setup to start the operator and then invokes the operator’s action:

( Note : hereinafter, the comments in the code are translated into Russian. In addition, indents are corrected for spaces instead of the [recommended in Go] tabs solely for the purpose of better readability in the Habr layout. After each listing, links to the original are on GitHub where English-language comments and tabs are saved.)

 func main() { //      STDOUT log.SetOutput(os.Stdout) sigs := make(chan os.Signal, 1) //       stop := make(chan struct{}) //     - //   SIGTERM   sigs signal.Notify(sigs, os.Interrupt, syscall.SIGTERM, syscall.SIGINT) // Goroutines      WaitGroup, //      wg := &sync.WaitGroup{} runOutsideCluster := flag.Bool("run-outside-cluster", false, "Set this flag when running outside of the cluster.") flag.Parse() //  clientset     Kubernetes clientset, err := newClientSet(*runOutsideCluster) if err != nil { panic(err.Error()) } controller.NewNamespaceController(clientset).Run(stop, wg) <-sigs //   (      ) log.Printf("Shutting down...") close(stop) //  goroutines  wg.Wait() // ,    } 

( main.go , raw )

We do the following:

1. We configure the handler of specific operating system signals to cause a correct (graceful) shutdown of the operator.
2. Use WaitGroup to correctly stop all goroutines before terminating the application.
3. We provide access to the cluster by creating a clientset .
4. We start NamespaceController in which all our logic will be located.

Now we need a basis for logic, and in our case it is the NamespaceController mentioned:

 // NamespaceController   Kubernetes API   //      RoleBinding   namespace. type NamespaceController struct { namespaceInformer cache.SharedIndexInformer kclient *kubernetes.Clientset } // NewNamespaceController   NewNamespaceController func NewNamespaceController(kclient *kubernetes.Clientset) *NamespaceController { namespaceWatcher := &NamespaceController{} //      Namespaces namespaceInformer := cache.NewSharedIndexInformer( &cache.ListWatch{ ListFunc: func(options metav1.ListOptions) (runtime.Object, error) { return kclient.Core().Namespaces().List(options) }, WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) { return kclient.Core().Namespaces().Watch(options) }, }, &v1.Namespace{}, 3*time.Minute, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, ) namespaceInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{ AddFunc: namespaceWatcher.createRoleBinding, }) namespaceWatcher.kclient = kclient namespaceWatcher.namespaceInformer = namespaceInformer return namespaceWatcher } 

( controller.go , raw )

Here we set up SharedIndexInformer , which will effectively (using the cache) wait for changes in namespaces (for more information about informers, see the article “ How does the Kubernetes scheduler actually work? ” - approx. Transl. ) . After that, we connect the EventHandler to the informer, so that when the namespace is added, the createRoleBinding function is createRoleBinding .

The next step is to define this createRoleBinding function:

 func (c *NamespaceController) createRoleBinding(obj interface{}) { namespaceObj := obj.(*v1.Namespace) namespaceName := namespaceObj.Name roleBinding := &v1beta1.RoleBinding{ TypeMeta: metav1.TypeMeta{ Kind: "RoleBinding", APIVersion: "rbac.authorization.k8s.io/v1beta1", }, ObjectMeta: metav1.ObjectMeta{ Name: fmt.Sprintf("ad-kubernetes-%s", namespaceName), Namespace: namespaceName, }, Subjects: []v1beta1.Subject{ v1beta1.Subject{ Kind: "Group", Name: fmt.Sprintf("ad-kubernetes-%s", namespaceName), }, }, RoleRef: v1beta1.RoleRef{ APIGroup: "rbac.authorization.k8s.io", Kind: "ClusterRole", Name: "edit", }, } _, err := c.kclient.Rbac().RoleBindings(namespaceName).Create(roleBinding) if err != nil { log.Println(fmt.Sprintf("Failed to create Role Binding: %s", err.Error())) } else { log.Println(fmt.Sprintf("Created AD RoleBinding for Namespace: %s", roleBinding.Name)) } } 

( controller.go , raw )

We get the namespace as obj and convert it to a Namespace object. Then we define the RoleBinding based on the YAML file mentioned at the beginning, using the provided Namespace object and creating the RoleBinding . Finally, we log whether the creation was successful.

The last function you need to define is Run :

 // Run        //       . func (c *NamespaceController) Run(stopCh <-chan struct{}, wg *sync.WaitGroup) { //    ,    defer wg.Done() //  wait group, ..   goroutine wg.Add(1) //  goroutine go c.namespaceInformer.Run(stopCh) //   - <-stopCh } 

( controller.go , raw )

Here we tell WaitGroup to run goroutine and then call the namespaceInformer that was previously defined. When the stop signal arrives, it will terminate the function, tell WaitGroup that it is no longer running, and this function will complete its work.

Information on building and running this operator in the Kubernetes cluster can be found in the repository on GitHub .

On this, the operator that creates the RoleBinding when Namespace appears in the Kubernetes cluster is ready.