# Metallb Addon

## Introduction

MetalLB is an open-source Load balancer implementation based on standard routing protocols for bare-metal Kubernetes. It specifically handles the `ServiceType: Loadbalancer` settings, which provides users with a more granular ability to control how IP addresses are assigned.

## Description

In our latest UI, users can modify the configuration for the IP Range at any stage of the cluster lifecycle. It provides:

* **Address Allocation – MetalLB** assigns and unassigning individual addresses to services as required, but only from IPs that are part of its defined pool.
* **External Announcement – After** MetalLB has assigned an external IP address to a service, it ensures the external network outside the cluster is aware that the IP resides within the cluster. MetalLB uses the typical routing protocols like ARP, NDP, or BGP to achieve this..

## Settings

MetalLB is not enabled by default on BareOS clusters. It can be enabled and configured during cluster creation or post cluster creation.

### Enabling MetalLB during cluster creation

During cluster creation, PMK provides the option to enable MetalLB for the cluster in the Cluster Add-ons sections.

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-3acf86708f2c98b370b5c4c711aa5187acb2b4cb%2Fi5ymingjfyfvzk4gcn08cvqtqtdb2o9hxrokplj3t3t5uge0judyxrtge8i79qsq.png?alt=media" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
**MetalLB configuration moved to only Custom Resource Definitions, starting from the version v0.13.0.**

Read MetalLB configuration documentation here: <https://metallb.universe.tf/configuration/>

With PMK 5.10 all MetalLB configurations are moved to the Custom Resources by default. Clusters with older configuration will working, but it is recommended to re-configure MetalLB with CRDs.
{% endhint %}

It can deployed in following modes:

* Layer 2 Mode (Default)
  * IP address pool range needs to provided and it creates the following CR on the clusters.

{% tabs %}
{% tab title="yaml" %}

```yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  creationTimestamp: null
  name: default
  namespace: metallb-system
spec:
  addresses:
    - 192.168.5.0-192.168.6.0
status: {}
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  creationTimestamp: null
  name: l2advertisement1
  namespace: metallb-system
spec:
  ipAddressPools:
    - default
status: {}
```

{% endtab %}
{% endtabs %}

* Advanced Configuration
  * Can be deployed with advanced configurations by providing all the CRs in a single yaml delimited by ---. Please applicable configuration documentaion here: <https://metallb.universe.tf/configuration/>

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-9e482a30113391da29f055266bfe16a3e67618f3%2Fbp248oli178mwxhtq4ab0klbh2ykw6s5qsjfu227e3wicojdk2s4zhbf0s1z6f7o.png?alt=media" alt=""><figcaption></figcaption></figure>

Example:

{% tabs %}
{% tab title="YAML" %}

```yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  creationTimestamp: null
  name: pool1
  namespace: metallb-system
spec:
  addresses:
    -  192.168.5.0-192.168.6.0
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  creationTimestamp: null
  name: pool2
  namespace: metallb-system
spec:
  addresses:
    -  192.168.7.0-192.168.8.0
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  creationTimestamp: null
  name: l2advertisement1
  namespace: metallb-system
spec:
  ipAddressPools:
    - pool1
    - pool2
```

{% endtab %}
{% endtabs %}

### Enabling MetalLB post cluster creation

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-99e31af32724c42be260029d35f0ef31efff3a6f%2Fsfbq0q4vtyuflamf8t5aymaiwfkpajzdd31brt0y88twhb25kbm55n0jeq4fwnce.png?alt=media" alt=""><figcaption></figcaption></figure>

Go to Infrastructure > Cluster > Add-ons tab and click Edit Add-ons.

Under Configurable Add-ons, enable MetalLB.

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-d248cd2939485c6a3ee05c61309c32549ef0569a%2Fl7xhrm1i187vcrzffheyqxcsrm1t70t9kl41kkajmb039vnt9qilxxt785f1aknl.png?alt=media" alt=""><figcaption></figcaption></figure>

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-b1e70f656631bbb8c94b6805c06d39becec3c865%2Ffm4f731f1whpgkaatfz2iu7n4n01mvhpqshg23n3odi6yv30z53uxpkbbu0qfup9.png?alt=media" alt=""><figcaption></figcaption></figure>

Once enabled, you need to also specify the IP address pool(s) that MetalLB can use to allocate IP addresses to services that are created with type `LoadBalancer`. PMK will then deploy MetalLB as a Kubernetes application within the cluster, under the `metallb-system` namespace.

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-03d791dc2366122de0171985cb10c17927777165%2F9cf712uqcm0mq7bwzevap95vspi3il8m473soic2xoad15h9st9czl6f3estwu66.png?alt=media" alt=""><figcaption></figcaption></figure>

You can add multiple IP ranges by clicking the **+ Add Address Pool Range** .

{% hint style="danger" %}
**Note**

MetalLB is currently inoperative when used in BGP mode for IPv6. This is a limitation with MetalLB itself but is under active development. Additionally, In order for MetalLB to function correctly, privileged mode must be enabled for running containers on the cluster.
{% endhint %}

## YAML Specifications

{% tabs %}
{% tab title="YAML" %}

```yaml
apiVersion: sunpike.platform9.com/v1alpha2
kind: ClusterAddon
metadata:
  labels:
    sunpike.pf9.io/cluster: '<clsuuid>'
    type: metallb
  name: <clsuuid>-metallb
  namespace: default
spec:
  clusterID: '<clsuuid>'
  override:
    params:
    - name: base64EncMetallbConfig
      value: 'base64 encoded entire metallb YAML config comprising of CRs '      
  type: metallb
  version: 0.9.7
  watch: true
```

{% endtab %}
{% endtabs %}

The *base64EncMetallbConfig* field can be added to any configuration that MetalLB supports. For example, if two address pools are required, then the following configuration can be applied.

{% tabs %}
{% tab title="Bash" %}

```bash
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  creationTimestamp: null
  name: pool1
  namespace: metallb-system
spec:
  addresses:
    -  192.168.5.0-192.168.6.0
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  creationTimestamp: null
  name: pool2
  namespace: metallb-system
spec:
  addresses:
    -  192.168.7.0-192.168.8.0
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  creationTimestamp: null
  name: l2advertisement1
  namespace: metallb-system
spec:
  ipAddressPools:
    - pool1
    - pool2
```

{% endtab %}
{% endtabs %}

Data shall be encoded as base64 and used as the override parameter, in the cluster Addon sunpike object.

{% tabs %}
{% tab title="YAML" %}

```yaml
override:
    params:
    - name: base64EncMetallbConfig  
      value: "YXBpVmVyc2lvbjogbWV0YWxsYi5pby92MWJldGExCmtpbmQ6IElQQWRkcmVzc1Bvb2wKbWV0YWRhdGE6CiAgY3JlYXRpb25UaW1lc3RhbXA6IG51bGwKICBuYW1lOiBwb29sMQogIG5hbWVzcGFjZTogbWV0YWxsYi1zeXN0ZW0Kc3BlYzoKICBhZGRyZXNzZXM6CiAgICAtICAxOTIuMTY4LjUuMC0xOTIuMTY4LjYuMAotLS0KYXBpVmVyc2lvbjogbWV0YWxsYi5pby92MWJldGExCmtpbmQ6IElQQWRkcmVzc1Bvb2wKbWV0YWRhdGE6CiAgY3JlYXRpb25UaW1lc3RhbXA6IG51bGwKICBuYW1lOiBwb29sMgogIG5hbWVzcGFjZTogbWV0YWxsYi1zeXN0ZW0Kc3BlYzoKICBhZGRyZXNzZXM6CiAgICAtICAxOTIuMTY4LjcuMC0xOTIuMTY4LjguMAotLS0KYXBpVmVyc2lvbjogbWV0YWxsYi5pby92MWJldGExCmtpbmQ6IEwyQWR2ZXJ0aXNlbWVudAptZXRhZGF0YToKICBjcmVhdGlvblRpbWVzdGFtcDogbnVsbAogIG5hbWU6IGwyYWR2ZXJ0aXNlbWVudDEKICBuYW1lc3BhY2U6IG1ldGFsbGItc3lzdGVtCnNwZWM6CiAgaXBBZGRyZXNzUG9vbHM6CiAgICAtIHBvb2wxCiAgICAtIHBvb2wy"
```

{% endtab %}
{% endtabs %}

## Qbert API support

[Qbert cluster create AP](https://platform9.com/docs/qbert/ref#postcreates-a-cluster-using-auto-deploy-or-manual-mode)I supports following parameter from PMK 5.10 onwards to configure MetalLB during cluster creation:

**PMK 5.10 onwards:**

{% tabs %}
{% tab title="YAML" %}

```yaml
curl --request POST \
 --url https://<account>.platform9.net/qbert/v4/{projectId}/clusters \
 --header "X-Auth-Token: {X-Auth-Token}" \
 --data '{
 "enableMetallb": {type: boolean, description: "If true, install MetalLB to support the loadbalancer service-type"},
 base64EncMetallbConfig: {type: string, description: "Base64 encoded string of metallb custom resource yaml"},
```

{% endtab %}
{% endtabs %}

The base64EncMetallbConfig should be a base64 encoded string with MetalLB Custom Resource Configuration yaml eg:

{% tabs %}
{% tab title="YAML" %}

```yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  creationTimestamp: null
  name: default
  namespace: metallb-system
spec:
  addresses:
    - 192.168.5.0-192.168.6.0
status: {}
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  creationTimestamp: null
  name: l2advertisement1
  namespace: metallb-system
spec:
  ipAddressPools:
    - default
status: {}
```

{% endtab %}
{% endtabs %}

Base64 encoded form of above yaml :

{% tabs %}
{% tab title="YAML" %}

```yaml
YXBpVmVyc2lvbjogbWV0YWxsYi5pby92MWJldGExCmtpbmQ6IElQQWRkcmVzc1Bvb2wKbWV0YWRhdGE6CiAgY3JlYXRpb25UaW1lc3RhbXA6IG51bGwKICBuYW1lOiBkZWZhdWx0CiAgbmFtZXNwYWNlOiBtZXRhbGxiLXN5c3RlbQpzcGVjOgogIGFkZHJlc3NlczoKICAgIC0gMTkyLjE2OC41LjAtMTkyLjE2OC42LjAKc3RhdHVzOiB7fQotLS0KYXBpVmVyc2lvbjogbWV0YWxsYi5pby92MWJldGExCmtpbmQ6IEwyQWR2ZXJ0aXNlbWVudAptZXRhZGF0YToKICBjcmVhdGlvblRpbWVzdGFtcDogbnVsbAogIG5hbWU6IGwyYWR2ZXJ0aXNlbWVudDEKICBuYW1lc3BhY2U6IG1ldGFsbGItc3lzdGVtCnNwZWM6CiAgaXBBZGRyZXNzUG9vbHM6CiAgICAtIGRlZmF1bHQKc3RhdHVzOiB7fQ==
```

{% endtab %}
{% endtabs %}

**Before PMK 5.10:**

The following parameter was used instead of the new **base64EncMetallbConfig (Added in PMK 5.10)**

`metallbCidr: {type: string, description: "If enableMetallb is true, specify the comma-separated pools of IPs that MetalLB will manage (for example: A.B.C.D-E.F.G.H, I.J.K.L-M.N.O.P)"},`

Existing clusters with older configuration will keep working when upgraded to PMK 5.10, but it is recommended to migrate to new configuration.

## Migrating MetalLB configuration from configMap to Custom Resources

* Instructions to convert config map to CRs are present here: [https://metallb.universe.tf/configuration/migration\_to\_ crds/](https://metallb.universe.tf/configuration/migration_to_crds/)
* Go to Infrastructure > Desired clusters > Add-ons > Edit Add-ons(Button) > MetalLB > Edit Configuration(Button)
* The existing configMap configuration should appear as below. Replace the configuration with new Custom Resource configuration and click Save Changes.

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-fde7018fe6b4685939c1472c61aecd89ab29943d%2F52p9pcnnswv15nz4hbx4i2qdrq9wlec4lhbnmhv3ftvrw7hstmqt18pg8hf8stfz.png?alt=media" alt="MetalLB with config map type configuration"><figcaption><p>MetalLB with config map type configuration</p></figcaption></figure>

<figure><img src="https://584068516-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtZryvDiHvZIhOZofYOWU%2Fuploads%2Fgit-blob-1ae57e8af9e22d52bc6f819681adb623f2c1b109%2Fv5u5p6f31adjwr43xb3wfrqvujmu338lipu3nxa73uqff4spvakrm1xkhqescmw2.png?alt=media" alt="MetalLB with Custom Resource type configuration"><figcaption><p>MetalLB with Custom Resource type configuration</p></figcaption></figure>

## Conclusion

Add-ons are updated along with the `pf9-kube` version and can be referenced in the [Support Matrix](https://docs.platform9.com/kubernetes/support-matrix) article. For additional assistance, contact our [support team](https://support.platform9.com/). For additional assistance with this addon, contact our [support team](https://support.platform9.com/).