# Bare Metal Support

<code class="expression">space.vars.product\_name</code> supports creation of Kubernetes clusters using both virtualized as well as physical nodes.

This guide will walk you through the process of creating a Kubernetes cluster using physical (bare metal) machines.

{% hint style="info" %}
**Info**

* Bare metal support in <code class="expression">space.vars.product\_name</code> is currently in `beta`
* <code class="expression">space.vars.product\_name</code> UI currently does not support creation of bare metal clusters. You need to use a separate CLI to create bare metal clusters today.
  {% endhint %}

## Prerequisites

Before starting, make sure you have:

* One or more Ubuntu physical machines that you want to use as Kubernetes nodes
* Network connectivity between all your machines and the <code class="expression">space.vars.product\_name</code> management cluster
* Administrator (sudo/root) access on all machines you plan to use

## Setup Process

Setting up a bare metal Kubernetes cluster involves four main steps:

1. Creating a cluster definition that describes your desired Kubernetes setup
2. Onboarding your machines using the `byohctl` CLI tool
3. Verifying that your cluster is working properly

Let's go through each step in detail.

### Step 1: Onboard Your Machines

On each physical server that you want to add to your Kubernetes cluster as a node, follow these steps:

Download the `byohctl` CLI:

{% tabs %}
{% tab title="Bash" %}

```bash
bash <(curl -s https://byohctl.s3.us-west-2.amazonaws.com/byohctl-setup)
```

{% endtab %}
{% endtabs %}

This command downloads a setup script and runs it immediately. The script will install the `byohctl` CLI on your machine.

Onboard the machine to your <code class="expression">space.vars.product\_name</code> environment:

{% tabs %}
{% tab title="Bash" %}

```bash
sudo ./byohctl onboard \
  -u <URL> \
  -e <USERNAME/EMAIL> \
  -p <PASSWORD> \
  -c <CLIENT_TOKEN> \
  -d <DOMAIN> \
  -t <TENANT>
```

{% endtab %}
{% endtabs %}

Replace the placeholders with your actual information:

`<URL>`: Your <code class="expression">space.vars.product\_name</code> management plane URL (e.g., example.platform9.com)

`<USERNAME/EMAIL>`: Your <code class="expression">space.vars.product\_name</code> username or email address

`<PASSWORD>`: Your <code class="expression">space.vars.product\_name</code> password

`<CLIENT_TOKEN>`: Your <code class="expression">space.vars.product\_name</code> client token (secret)

`<DOMAIN>`: Your <code class="expression">space.vars.product\_name</code> domain (default is "default")

`<TENANT>`: Your <code class="expression">space.vars.product\_name</code> tenant (default is "service")

For example:

{% tabs %}
{% tab title="Bash" %}

```bash
byohctl onboard -u sam-dev.app.dev-pcd.platform9.com -e admin@platform9.com -p your_password -c client_secret -d default -t service
```

{% endtab %}
{% endtabs %}

Verify that the machine has been successfully onboarded:

Check that the required agent service is running:

{% tabs %}
{% tab title="Bash" %}

```bash
systemctl status pf9-byohost-agent.service
```

{% endtab %}
{% endtabs %}

You should see that the service is active and running.

Check the agent logs to confirm successful registration:

{% tabs %}
{% tab title="Bash" %}

```bash
cat /var/log/pf9/byoh/byoh-agent.log
```

{% endtab %}
{% endtabs %}

Look for messages indicating successful registration with the <code class="expression">space.vars.product\_name</code> management plane.

From your management cluster, verify that the host appears as a resource:

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl get byohost -A
```

{% endtab %}
{% endtabs %}

Your machine should appear in the list of available hosts.

### Step 2: Create Your Kubernetes Cluster

Now you'll create a cluster definition that tells <code class="expression">space.vars.product\_name</code> how to build your Kubernetes cluster using the physical servers you've onboarded.

Create a file named `cluster.yaml` with your cluster definition. Here's an explanation of the key components:

* `Cluster`: Defines the basic properties of your Kubernetes cluster
* `ByoCluster`: Configures the infrastructure-specific settings
* `HostedControlPlane`: Sets up the control plane components
* `ByoMachineTemplate`: Defines the template for worker nodes
* `KubeadmConfigTemplate`: Configures how kubeadm will bootstrap nodes
* `MachineDeployment`: Specifies how many worker nodes to create
* `K8sInstallerConfigTemplate`: Determines how Kubernetes will be installed

The sample cluster definition in the guide includes all these components with appropriate settings for a <code class="expression">space.vars.product\_name</code> environment. You'll need to modify the following items to match your specific setup:

* Cluster name (replace `byo-22-13` with your preferred name)
* Namespace (replace `sam-dev-default-service` with your namespace)
* Control plane endpoint hostname (update to match your environment)
* Kubernetes version (adjust if needed)
* Number of replicas (set based on how many worker nodes you want)

{% tabs %}
{% tab title="YAML" %}

```bash
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  labels:
    cas-capi-version: v1.30
    core-addons: enabled
    metrics-server: 0.6.4
    proxy-addons: enabled
  name: byo-22-13
  namespace: sam-dev-default-service
spec:
  clusterNetwork:
    apiServerPort: 443
    pods:
      cidrBlocks:
      - 10.244.0.0/16
    services:
      cidrBlocks:
      - 10.96.0.0/16
  controlPlaneRef:
    apiVersion: controlplane.platform9.io/v1alpha1
    kind: HostedControlPlane
    name: byo-22-13cp
    namespace: sam-dev-default-service
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: ByoCluster
    name: byo-22-13
    namespace: sam-dev-default-service
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: ByoCluster
metadata:
  name: byo-22-13
  namespace: sam-dev-default-service
spec:
  bundleLookupBaseRegistry: quay.io/platform9
  controlPlaneEndpoint:
    host: byo-22-13cp.sam-dev-default-service.app.dev-pcd.platform9.com
    port: 443
---
apiVersion: controlplane.platform9.io/v1alpha1
kind: HostedControlPlane
metadata:
  name: byo-22-13cp
  namespace: sam-dev-default-service
spec:
  addons:
    coreDNS: {}
    konnectivity:
      agent:
        extraArgs:
        - --proxy-server-host=konnectivity-byo-22-13cp.sam-dev-default-service.app.dev-pcd.platform9.com
        - --proxy-server-port=443
        image: registry.k8s.io/kas-network-proxy/proxy-agent
        tolerations:
        - key: CriticalAddonsOnly
          operator: Exists
        version: v0.0.32
      server:
        image: registry.k8s.io/kas-network-proxy/proxy-server
        port: 8132
        version: v0.0.32
  apiServer:
    extraArgs:
    - --cloud-provider=external
    - --cors-allowed-origins=http://localhost:3000,http://localhost:3000/,https://sam-dev-default-service.app.dev-pcd.platform9.com,https://sam-dev-default-service.app.dev-pcd.platform9.com/,https://sam-dev-default-service-scadrial.app.dev-pcd.platform9.com,https://sam-dev-default-service-scadrial.app.dev-pcd.platform9.com/
    - --advertise-address=10.96.0.40
    - --authentication-config=/etc/kubernetes/oidc-auth/oidc.yaml
    extraVolumeMounts:
    - mountPath: /etc/ssl/host-certs
      name: etc-ssl-certs-from-host
      readOnly: true
    - mountPath: /etc/kubernetes/oidc-auth
      name: oidc-auth
      readOnly: true
    resources: {}
  controllerManager:
    extraArgs:
    - --cloud-provider=external
    resources: {}
  extraCertSANs:
  - konnectivity-byo-22-13cp.sam-dev-default-service.app.dev-pcd.platform9.com
  - byo-22-13cp.sam-dev-default-service.app.dev-pcd.platform9.com
  extraVolumes:
  - configMap:
      name: oidc-auth
    name: oidc-auth
  hcpClass: default
  hostname: byo-22-13cp.sam-dev-default-service.app.dev-pcd.platform9.com
  kubelet:
    cgroupfs: systemd
    preferredAddressTypes:
    - InternalIP
    - ExternalIP
    - Hostname
  scheduler:
    resources: {}
  version: 1.31.2
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: ByoMachineTemplate
metadata:
  name: byo-22-13
  namespace: sam-dev-default-service
spec:
  template:
    spec:
      installerRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: K8sInstallerConfigTemplate
        name: byo-22-13
        namespace: sam-dev-default-service
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
  name: byo-22-13
  namespace: sam-dev-default-service
spec:
  template:
    spec: {}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: byo-22-13
  namespace: sam-dev-default-service
spec:
  clusterName: byo-22-13
  replicas: 1
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: byo-22-13
      clusterName: byo-22-13
      failureDomain: nova
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: ByoMachineTemplate
        name: byo-22-13
      version: v1.32.2
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: K8sInstallerConfigTemplate
metadata:
  name: byo-22-13
  namespace: sam-dev-default-service
spec:
  template:
    spec:
      bundleRepo: quay.io/platform9
      bundleType: k8s
```

{% endtab %}
{% endtabs %}

Apply the cluster configuration:

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl apply -f cluster.yaml
```

{% endtab %}
{% endtabs %}

This command sends your cluster definition to the <code class="expression">space.vars.product\_name</code> management plane, which then begins creating your cluster.

### Step 3: Monitor Cluster Creation

You can track the progress of your cluster creation with the following commands:

Check the TenantControlPlane (TCP) status:

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl get tcp -A
```

{% endtab %}
{% endtabs %}

Check the HostedControlPlane (HCP) status:

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl get hcp -A
```

{% endtab %}
{% endtabs %}

Monitor the Machine provisioning:

{% tabs %}
{% tab title="YAML" %}

```bash
kubectl get machine -A
```

{% endtab %}
{% endtabs %}

Once your machines show a status of "`Running`" with `NODENAME` and `PROVIDERID` values set, your cluster nodes have successfully joined the cluster.

### Step 4: Access Your Cluster

Get the `kubeconfig` file for your new cluster:

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl get secret -n <NAMESPACE> <CLUSTER_NAME>cp-admin-kubeconfig -o json | \
  jq '.data["super-admin.conf"]' | xargs | base64 -d > cluster-kubeconfig.yaml

export KUBECONFIG=cluster-kubeconfig.yaml
```

{% endtab %}
{% endtabs %}

Replace `<NAMESPACE>` with your namespace and `<CLUSTER_NAME>` with your cluster name.

Alternatively, you can download the `kubeconfig` file from the <code class="expression">space.vars.product\_name</code> web interface.

Verify that your cluster is working properly:

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl get nodes -A
kubectl get ns
kubectl get pods -A
```

{% endtab %}
{% endtabs %}

These commands show the nodes in your cluster, the namespaces, and all running pods.

## Important Consideration

* The Kubernetes version specified in the `HostedControlPlane` definition must be supported by your management plane.
* The version specified in `MachineDeployment` should match the version available in the <code class="expression">space.vars.product\_name</code> BYOH bundle repository.
* Make sure to use the correct bundle repository in your cluster definition.
* The <code class="expression">space.vars.product\_name</code> service agent logs are stored at `/var/log/pf9/byoh/byoh-agent.log` and are useful for troubleshooting.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.platform9.com/private-cloud-director/2025.4/kubernetes-clusters/byoh--creating-kubernetes-clusters-with-your-own-infrastructure.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.