How we set up Continuous Delivery at Kubernetes using TFS

We continue our journey to Continuous Delivery (CD) and High Availability (HA), based on redundancy. In the previous series, we translated the mobile API to .NET Core . The next logical step to achieve the CD is to configure the assembly in the Docker container.

Today we will share our getting-started guide on how to configure the build of docker images and deploy to Kubernetes in TFS for .NET developers.

(It is assumed that by this time you have already migrated your ASP.NET application to an ASP.NET Core, and if not, read our previous article ).

Setting up an ASP.NET application to work with Docker

In Visual Studio 2017, right-click on a web project -> Add -> Docker Support;
For VS2015, you need to add an extension .
The Dockerfile file is added to the project folder - this is the config for creating the image of our application.
Read more about Docker here.
A new docker-compose project will be added.
By itself, docker-compose is a utility for managing multi-container applications. For example, you run the application and the DBMS to it.
Files in the project:
a. docker-compose.yml - description of your services / dependencies.
Here is an example of an ASP.NET application in conjunction with SQL Server:

version: '3' services: agentrequests.webapp: image: agentrequests.webapp build: context: . dockerfile: AgentRequests.WebApp/Dockerfile depends_on: - agentrequests-db agentrequests-db: image: microsoft/mssql-server-linux environment: SA_PASSWORD: "<YourStrong!Passw0rd>1" ACCEPT_EULA: "Y" ports: - "1401:1433" volumes: - agent-requests-db-data:/var/opt/mssql volumes: agent-requests-db-data:

Service names (in the example database, agentrequests-db) can be used directly in your application, such as Server Side Service Discovery.

For example, the connection string to the base is "Server = agentrequests-db; Database = AgentRequests; User = sa; Password = <YourStrong! Passw0rd> 1;"

b. docker-compose.override.yml - this file is used during development. Visual Studio merges these files when you press F5.

HINT: Each time docker-compose.yml changes, the application will be assigned a new local port, which quickly becomes boring during debugging. Therefore, in docker-compose.override.yml it is useful to fix the port of your application:

 version: '3' services: agentrequests.webapp: environment: - ASPNETCORE_ENVIRONMENT=Development ports: - "40005:80"

c. docker-compose.ci.build.yml - with this config you can crash your application on CI, and the result will be similar to the local build. This file is needed if you simply deploy ready files, for example, in IIS. We are going to supply ready-made docker images and will use Dockerfile directly.

The intermediate result: we do the docker-compose project starting, we press F5 and we rejoice.
NOTE: The first launch may be long, since the docker needs to download the SQL / ASP.Net Core images.

Also, when debugging, Visual Studio does not create a new docker-image each time the application code changes - in fact, only one container is created from your Dockerfile to which the folder with your sources on the host machine is mounted. Thus, every change, for example, js-file, will instantly affect even the running container.

Build and Deploy Docker Images in TFS

CI assumes that we have a build machine for performing automated builds independently of the developer. Developers upload their changes to the version control system, the build machine takes the latest changes and recompiles the project. Thus, the build machine should have all the necessary tools for building the project. In our case, it must have access to the Docker in order to build Docker images.

There are several options:

We can deliver Docker directly to a build machine or remotely connect to Docker on another machine via the Docker client. Initially, we had a standard .Net development on Windows, so all the build machines were Windows virtual machines with one or more build agents. So that Docker can build Linux containers on a Windows machine, the docker installs a Linux virtual machine. It turns out that we will have several virtual machines nested in each other. But I don’t want to start fussing, and Docker doesn’t officially support this mode.
To connect to Docker on another machine, you need to configure remote access , by default it is turned off. It is also recommended to ensure the security of TLS certificates . There is also a manual from Microsoft , which offers a simplified configuration option using a windows container preloaded with LibreSSL.
The easiest way: you can run the build agent directly in the Docker container, and it will have access to the Docker that is running. Simply select the desired container from the microsoft / vsts-agent repository.

Setting up the build agent

Download build agent.
About the differences in the versions of images and parameters can be found here .
You need a version with docker on board, for example:
docker pull microsoft / vsts-agent: ubuntu-16.04-tfs-2017-u1-docker-17.12.0-ce
Generate Personal Access Token (PAT) on the TFS Security page:
You can add a new Agent Pool for docker assemblies. This is done here:

We start the container:

 docker run \ -e TFS_URL=<YOUR_TFS_URL> \ -e VSTS_TOKEN=<PAT> \ -e VSTS_POOL=<POOL> \ -e VSTS_AGENT=$(hostname)-agent \ -v /var/run/docker.sock:/var/run/docker.sock \ --restart=always \ -it microsoft/vsts-agent:ubuntu-16.04-tfs-2017-u1-docker-17.12.0-ce

CI Setup

In the project in TFS we add a new Build Definition with a template - Container (PREVIEW)
Taska build image:
')
a. Container Registry Type - Container Registry;
b. Docker Registry Connection - here we set up the path to your image registry. You can use the Docker Hub, but we in the company use the Nexus Registry;
c. Docker File - the path to the docker file. It is better to specify the path clearly, without a mask;
d. Use Default Build Context - uncheck;
e. Build Context - the path to the folder where your .sln file is located;
f. Image-name - it is better to set it explicitly, all the characters in lower case. Example: groups / agent-requests: $ (Build.BuildId);
g. You can put include latest tag - the latest tag in the registry will be updated.
Push an image - similar to the second paragraph. The main thing is not to forget to change the image name.
Add a tusk with Publish Build Artifacts template. Since we are planning to deploy to kubernetes, our artifact will be the config for kubectl:

a. Path to Publish - the path to your yaml file with the kubernetes config. You can specify a folder if there are several configs;
b. Artifact Name - to your taste. For example, kubernetes;
c. Artifact Type - Server.

CD and Kubernetes Setup

At first, a small digression. Roughly speaking, docker-compose (swarm) is a competitor to kubernetes. But since VS does not have a convenient tuling for build and debug in kubernetes, we use both options: compose when developing, kubernetes in combat.

The good news is that there is a utility Kompose - it can convert Kubernetes configs to / from docker-compose.yaml files.

However, it is not necessary for a developer to use docker / compose in general - you can configure everything in the old manner and use your hands to change urls / configs or store ten web.config for different environments.

Example service.yaml

 apiVersion: v1 kind: Service metadata: name: webapp spec: type: LoadBalancer selector: app: webapp ports: - port: 80  deployment.yaml: apiVersion: extensions/v1beta1 kind: Deployment metadata: name: webapp spec: replicas: 1 template: metadata: labels: app: webapp spec: imagePullSecrets: - name: <   Docker Registry> containers: - image: webapp name: webapp env: - name: DB_USER valueFrom: secretKeyRef: name: database-secret key: username - name: DB_PASSWORD valueFrom: secretKeyRef: name: database-secret key: password - name: SQLCONNSTR_<  > # SQLCONNSTR_DefaultConnection value: "Server=< SQL >;Initial Catalog=< >;Persist Security Info=False;User ID=$(DB_USER);Password=$(DB_PASSWORD);" ports: - containerPort: 80

To configure the CD, we used the Kubernetes extension for our TFS server, since the standard Deploy to Kubernetes task, which comes out of the box, turned out to be inactive in our version of TFS.

Add connection settings to our Docker Registry in Kubernetes

 kubectl create secret docker-registry\ <   Docker Registry>\ --docker-server=< >\ --docker-username=<>\ --docker-password=<>\ --docker-email=<>

Add login and password to connect to the database

 kubectl create secret generic database-secret\ --from-literal=username=<>\ --from-literal=password=<>

Note: Steps 1 and 2 can also be automated via TFS.

Add Kubernetes endpoint to TFS:
Create a new Release Defenition:

a. Select a project and Build definition
b. Tick Continuous deployment
Add a kasu kubernetes downloader:

a. In the “kubernetes end point” field, select your kubernetes endpoint
b. In the “kubectl download version” field, specify the required version or leave the field blank, in which case the latest version of the client will be installed.
Add the kaspek exec task:

a. In the field “Sub Command” we write: apply
b. In the field “Arguments” we write: -f $ (System.DefaultWorkingDirectory) / <path to the folder with configs> /deployment.yaml
Add the kaspek exec task:

a. In the field “Sub Command” we write: apply
b. In the field “Arguments” we write: image -f $ (System.DefaultWorkingDirectory) / <path to the folder with configs> /service.yaml
Add the kaspek exec task:

a. In the “Sub Command” field we write: set
b. In the field “Arguments” we write: image -f $ (System.DefaultWorkingDirectory) / <path to the folder with configs> /deployment.yaml webapp = webapp: $ (Build.BuildID)