Create Multi-Master BareOS Cluster

This document describes creation of a multi-master BareOS cluster using PMK. We recommend reading What is BareOS for an understanding of BareOS and BareOS Cluster Architecture before proceeding with this document.

A highly available cluster is composed of at least 3 master nodes, each running a member of the etcd distributed database and all the Kubernetes master components (API, controller manager and scheduler). We choose an odd number of master nodes so that it is possible to establish quorum within the etcd nodes.

Create BareOS Cluster Using CLI or REST API

Need a BareOS Multi-Master Cluster fast? For an advance user, the PMK CLI Cluster Create command can automate the whole deployment process.

You can also create multi-master BareOS Kubernetes clusters through the PMK REST API. Refer to the PMK API documentation for details.

Create BareOS Cluster Using UI

Follow the steps given below to create a BareOS Kubernetes cluster using the PMK UI.

  • Step 1 - Prepare the nodes that are to be included in the cluster.
    If you haven’t installed the pf9ctl CLI, you need to install it on one of your nodes first.
    bash <(curl -sL http://pf9.io/get_cli)
    

    This will download a scriped named cli_setup.sh and invoke the script.

    This process will ask for the following:

    • Your PMK deployment URL
    • Your PMK account username and password

    Once the CLI is installed, run the prep-node command using the CLI to prepare all the nodes to be added to the cluster. Following example run prepares the current node (localhost) and two additional nodes with IP address 150.20.7.65 and 150.20.7.66.

    NOTE - before you can run the CLI on remote notes, you will need to setup ssh access across these nodes.

    pf9ctl cluster prep-node -u testuser -p testpassword -s ~/.ssh/id_rsa -i localhost -i 150.20.7.65 -i 150.20.7.66
    

    Now your nodes are ready to be added to your cluster. They should show up in your PMK UI cluster creation wizard.

  • Step 2 - Click Infrastructure > Clusters > Add Cluster. Provide a name for your cluster.

  • Step 3 - Select one or more nodes that would function as master nodes. If you want to create a multi-master cluster, select at least three nodes that would function as master nodes.

  • Step 4 - Select the “Allow workloads on masters” checkbox if you wish to run workloads on masters as well as workers. For production clusters, we recommend not running workloads on masters. Click Next.

  • Step 5 - Select the nodes that would function as worker nodes. Click Next.

  • Step 6 - Enter the advanced configuration details for your cluster, based on the table below.
Field Description
Virtual IP address for cluster The IP address with which the user accesses the cluster or the high availability floating IP address.
Physical Interface for Virtual IP Association The network interface to which the virtual IP gets associated. Ensure that the virtual IP specified above is accessible on this network interface, and that all master nodes use the same interface name for the interface to be associated with the virtual IP
API FQDN The FQDN (DNS Name) that is to be used to access the Kubernetes cluster API server from outside of the cluster
Containers CIDR The IP range that Kubernetes uses to configure the Pods (Docker containers) deployed by Kubernetes
Services CIDR The IP range that Kubernetes uses to configure services deployed by Kubernetes
Privileged Select the check box to enable the cluster to run privileged containers. Note that being able to run privileged containers within the cluster is a prerequisite if you wish to enable service type loadbalancer using MetalLB. By default a container is not allowed to access any devices on the host, but a “privileged” container is given access to all devices on the host. For more information, see [Privileged Policy Reference](https://kubernetes.io/docs/concepts/policy/pod-security-policy/#privileged)
Advanced API Configuration Select the check box to configure the APIs to be used by the cluster. If you do not have adequate knowledge of Kubernetes APIs, it is recommended to avoid selecting this check box. When this check box is not selected, the GA and beta APIs (that is the stable APIs) for the currently installed Kubernetes version are enabled.
Default API groups and versions This option is visible only if you select the Advanced API Configuration check box. Select the Default API groups and versions option to enable on the cluster, the default APIs based on the Kubernetes installation in your environment
All API groups and versions This option is visible only if you select the Advanced API Configuration check box. Select All API groups and versions option to enable on the cluster, all alpha, beta, and GA versions of Kubernetes APIs that have been published till date.
Custom API groups and versions This option is visible only if you select the Advanced API Configuration check box. Select Custom API groups and versions option to specify one or more API versions that you wish to enable and/or disable. Enter the API versions in the text area following the Custom API groups and versions option. For example, to enable Kubernetes v1 APIs, enter the expression, api/v1=true. Similarly, to disable Kubernetes v2 APIs, enter the expression, api/v2=false. If you want to enable and/or disable multiple versions, you could enter comma-separated expressions, such as, api/v2=false,api/v1=true.

Review the cluster configuration, and then click Create Cluster.

The multi-master BareOS cluster is now created.

You can now deploy your applications on the highly available multi-master Kubernetes cluster.