Disallowing Workloads on Master Nodes

High availability is critical to Kubernetes clusters running production workloads. Key Kubernetes services such as kube-apiserver, kube-scheduler should be available and running smoothly at all times on master nodes. Therefore, it is essential to have dedicated resources for the master nodes, and avoid having other non-critical workloads interfere with the functioning of the master services.

Master nodes must be made available to kube-dns, heapster, and Kubernetes dashboard services, as these are critical to the smooth functioning of a Platform9 Managed Kubernetes cluster. This can be achieved by leveraging the taints and tolerations functionality provided by Kubernetes.

Taints and tolerations are Kubernetes primitives.

A taint enables a node to repel or disallow a pod from associating with a node to which the taint is applied. An exception to this can be made by using a toleration on the pod. A toleration allows a pod to be deployed on a node to which a taint has been applied. A taint is generally required to be applied to a master node, as the master node would dedicate resources to run pods with critical services.

When the cluster admin opts to disallow non-critical workloads on master nodes, Platform9 Managed Kubernetes applies a taint to master nodes of the Kubernetes cluster. Subsequently, a toleration can be added to all critical workloads that need to run on the master nodes.

Disallow Non-critical Workloads on Master Nodes through Platform9 Clarity UI

You can disallow non-critical workloads on master nodes while creating the cluster, by applying a taint to the master nodes. A taint cannot be applied to running clusters.

To taint master nodes, follow the steps given below.

  1. Log in to Platform9 Clarity UI.
  2. Navigate to Infrastructure>Clusters.
  3. Click Add Cluster, enter the details for the Cluster Type and click Next.
  4. Select the Master Node Instance Type, Worker Node Instance Type, Number of Master Nodes, and Number of Worker Nodes.
  5. Deselect the Allow Workloads on Master Nodes to taint master nodes. Disallow Workloads on Master Nodes

  6. Fill in the other details as required and create the cluster.

A cluster with a taint on the master nodes is created.

Note : If you wish to remove a taint applied to master nodes on a cluster, you must first delete the cluster with the taint, and recreate the cluster without a taint.

Apply Toleration to Node

Kubernetes has a lot of add-on services that might be critical to working of the cluster. If required, users can deploy their own services on master nodes by adding the necessary tolerations.

Platform9 Managed Kubernetes master nodes are tainted with following clause.

node-role.kubernetes.io/master=true:NoSchedule

In the aforementioned clause, node-role.kubernetes.io/master is the key, true is the value, and the effect is NoSchedule

You can apply toleration to a pod through the PodSpec, using a tolerations section like below.

tolerations:
  - key: "node-role.kubernetes.io/master"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"

For more details on Kubernetes taints and tolerations, refer to https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/.


March 08, 2018