Setup a Kubernetes Cluster with Ansible

Although all large Cloud provider nowadays offer Managed Kubernetes Clusters, I prefer to have access to a local cluster especially during development.

In this post, we will setup a Kubernetes Cluster using Ansible and Kubeadm. The cluster will include a single master node and two (or more) worker nodes.

Most of the work done here is based on a tutorial by bsder1.

Requirements

I will use three Ubuntu 18.04 LTS (Bionic Beaver) servers, each with 4GB RAM and 2 CPUs, you should also be fine with 1GB RAM.

All servers have been updated to the latest packages and a SSH key for access is also deployed.

Setup playbook

All playbooks are available in the GitHub repository2 and should be cloned first.

The inventory file contains the nodes’ hostnames and should match your servers. You can also add additional nodes if you want to build a larger cluster.

A good test to see, if the inventory is configured correctly is ping:

ansible all -i inventory -m ping

This should return a pong from each node.

Install required software

All nodes need a basic set of software, namely docker3, kubelet4 and kubeadm5. Docker is available in the official repository but for Kubernetes we need to add apt.kubernetes.io.

Run the kube-install-software playbook to perform the required steps:

ansible-playbook -i inventory kube-install-software.yml -K
...
PLAY RECAP *********************************************************************
kubernetes                 : ok=5    changed=3    unreachable=0    failed=0
kubernetes-node-1          : ok=5    changed=3    unreachable=0    failed=0
kubernetes-node-2          : ok=5    changed=3    unreachable=0    failed=0

Install Cluster

Now that all nodes fulfill the basic requirements we are ready to setup the cluster. We will use kubeadm here and rely on flannel6 as the network fabric.

All steps described below are also included in the kube-setup-cluster playbook:

ansible-playbook -i inventory kube-setup-cluster.yml -K

It’s surprisingly simple to create a kubernetes cluster with kubeadm:

kubeadm init --pod-network-cidr=10.244.0.0/16

Note: Kubeadm uses the network 10.96.0.0/12 as default but flannel uses 10.244.0.0/16 which we need to pass as option.

The command provides a very detailed explanation what we need to do to get access to your cluster:

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.0.9.88:6443 --token se28o1.ljmh27sev5umdz9x --discovery-token-ca-cert-hash sha256:6c9f49f5fed776e19aabe2b3f8f938c15f3ddb30519d63acded61cd4397e8f85

As we picked flannel we can install it with this command:

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Finally we can add the nodes to the cluster by running the kubeadm join command on each node:

kubeadm join 10.0.9.88:6443 --token se28o1.ljmh27sev5umdz9x --discovery-token-ca-cert-hash sha256:6c9f49f5fed776e19aabe2b3f8f938c15f3ddb30519d63acded61cd4397e8f85

Now we can ssh into our kubernetes master node, become root and check the status of our cluster:

root@kubernetes:~# kubectl get nodes
NAME                STATUS     ROLES    AGE     VERSION
kubernetes          NotReady   master   10m     v1.13.1
kubernetes-node-1   Ready      <none>   9m31s   v1.13.1
kubernetes-node-2   Ready      <none>   9m27s   v1.13.1

After some time the nodes should become ready:

root@kubernetes:~# kubectl get nodes
NAME                STATUS   ROLES    AGE    VERSION
kubernetes          Ready    master   27m    v1.13.1
kubernetes-node-1   Ready    <none>   27s    v1.13.1
kubernetes-node-2   Ready    <none>   5m9s   v1.13.1

The pods should look similar to this:

root@kubernetes:~# kubectl get pods --all-namespaces -owide
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE     IP           NODE                NOMINATED NODE   READINESS GATES
kube-system   coredns-86c58d9df4-8zwph             1/1     Running   0          9m9s    10.244.1.3   kubernetes-node-2   <none>           <none>
kube-system   coredns-86c58d9df4-tr9rh             1/1     Running   0          9m8s    10.244.1.2   kubernetes-node-2   <none>           <none>
kube-system   etcd-kubernetes                      1/1     Running   1          26m     10.0.9.88    kubernetes          <none>           <none>
kube-system   kube-apiserver-kubernetes            1/1     Running   1          26m     10.0.9.88    kubernetes          <none>           <none>
kube-system   kube-controller-manager-kubernetes   1/1     Running   1          26m     10.0.9.88    kubernetes          <none>           <none>
kube-system   kube-flannel-ds-42cht                1/1     Running   0          2m19s   10.0.9.90    kubernetes-node-2   <none>           <none>
kube-system   kube-flannel-ds-gb2rm                1/1     Running   0          2m19s   10.0.9.88    kubernetes          <none>           <none>
kube-system   kube-flannel-ds-mzgxm                1/1     Running   0          50s     10.0.9.89    kubernetes-node-1   <none>           <none>
kube-system   kube-proxy-8rx9z                     1/1     Running   0          5m32s   10.0.9.90    kubernetes-node-2   <none>           <none>
kube-system   kube-proxy-hrz6p                     1/1     Running   0          50s     10.0.9.89    kubernetes-node-1   <none>           <none>
kube-system   kube-proxy-qb5pc                     1/1     Running   1          27m     10.0.9.88    kubernetes          <none>           <none>
kube-system   kube-scheduler-kubernetes            1/1     Running   1          26m     10.0.9.88    kubernetes          <none>           <none>

Install k8s-self-hosted-recovery (optional, recommended)

Since version 1.8 of kubeadm a new limitation for self-hosted kubernetes clusters was introduced. Those clusters do no longer recover from a reboot without manual intervention7.

On github a small script can be found8 which creates a service that performs all required steps so your cluster survives reboots.

The service is included in the kube-self-hosted-recovery playbook:

ansible-playbook -i inventory kube-self-hosted-recovery.yml -K

Summary

Here we go, you should now have a Kubernetes Cluster with two worker nodes deployed and ready for further exploration.

I do not recommend this installation for a public system as it does not contain any security related changes, please consult the official documentation if you want to use it for production.

Footnotes

Related Posts