# Cilium Cni

Platform9 Managed Kubernetes (PMK) supports Cilium as the Container Network Interface (CNI) for secure, high-performance cluster networking powered by eBPF. This page explains the architecture, prerequisites, installation options, configuration choices, migration guidance from Calico, and validation/troubleshooting steps aligned with PMK best practices.

For a conceptual introduction to Cilium and Hubble, see the [Cilium introduction](https://docs.cilium.io/en/stable/overview/intro/)

## Support Matrix and Recommendations

| **Category**            | **Supported**                                         | **Recommended** |
| ----------------------- | ----------------------------------------------------- | --------------- |
| OS (hosts)              | Minimum Ubuntu 22.04                                  | Ubuntu 24.04    |
| MTU                     | Align with underlay; validate path MTU                | 9000 (Jumbo)    |
| Encapsulation / Routing | VXLAN or native routing based on network capabilities | VXLAN routing   |

Best practices

* Use eBPF masquerade with kube-proxy replacement for lower latency and reduced conntrack pressure.
* Set MTU to match end-to-end path; if using VXLAN over jumbo-capable underlay, set 9000 on NICs and size Cilium MTU accordingly (account for overlay overhead).
* Ensure nodes have required kernel features for eBPF (modern kernels on Ubuntu 22.04/24.04 meet this by default).

### Prerequisites

In addition to having a Kubernetes cluster provisioned and healthy and administrative kubectl access, there are requirements for using Cilium as the CNI implementation:

* Subnets on a network should not coincide. You cannot have two clusters with container CIDR or service CIDR being the same on one L2 domain or network.
* Outbound connectivity to pull Cilium images and Helm charts (or a mirrored registry).

## Create a Cilium-enabled Cluster

While creating a new Kubernetes cluster using the PMK UI, in the wizard under Network Configuration, select Cilium as the network backend.

## Alternative Manual Installation Options

For manual Cilium installation, choose one path: Helm (flexible, GitOps-friendly) or Cilium CLI (simplified). Ensure any legacy CNI artifacts are removed before applying Cilium.

### Option A: Installation with Helm

1. Add chart repo and update indices.
2. Install cilium into kube-system with PMK-aligned values: VXLAN routing, eBPF masquerade, kube-proxy replacement, MTU sizing.
3. Verify daemonset rollout and agent health across all nodes.

### Option B: Install with Cilium CLI

1. Install cilium CLI on an admin workstation.
2. Apply installation with flags that match PMK recommendations (VXLAN, eBPF masquerade, kube-proxy replacement).
3. Run post-install validation (cilium status, cilium connectivity test).

If the init container fails to copy binaries into /opt/cni/bin, fix permissions on each node:

{% tabs %}
{% tab title="Bash" %}

```bash
sudo chown root:root /opt/cni/bin
sudo chmod 755 /opt/cni/bin
sudo chown -R root:root /opt/cni/bin
sudo chmod -R 755 /opt/cni/bin
```

{% endtab %}
{% endtabs %}

### Post-Install Validation

1. Agent Health: Cilium status should report OK for all subsystems and nodes.

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl -n kube-system exec ds_cilium -- cilium status
```

{% endtab %}
{% endtabs %}

2. Connectivity: Validate pod-to-pod, pod-to-service, DNS, and egress.

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl run test-a --image=busybox --restart=Never -- sleep 3600
kubectl run test-b --image=busybox --restart=Never -- sleep 3600
kubectl exec test-a -- ping -c3 $(kubectl get pod test-b -o jsonpath='{.status.podIP}')

kubectl create deploy web --image=nginx --replicas=2
kubectl expose deploy web --port=80
kubectl exec test-a -- wget -qO- http://web

kubectl run -it --rm --restart=Never --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 dns -- nslookup kubernetes.default
kubectl exec test-a -- wget -qO- http://example.com
```

{% endtab %}
{% endtabs %}

3. NetworkPolicy: Confirm deny-by-default and allow-specific behavior.

{% tabs %}
{% tab title="YAML" %}

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: demo-deny
spec:
podSelector:
    matchLabels:
        app: web
policyTypes:
- Ingress
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="YAML" %}

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-from-test-a
spec:
podSelector:
    matchLabels:
        app: web
ingress:
- from:
    - podSelector:
        matchLabels:
            run: test-a
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Bash" %}

```bash
kubectl label deploy web app=web --overwrite
kubectl apply -f demo-deny.yaml
kubectl apply -f allow-from-test-a.yaml
# Should succeed
kubectl exec test-a -- wget -qO- http://web
# Should fail
kubectl exec test-b -- wget -qO- http://web || echo "blocked as expected"
```

{% endtab %}
{% endtabs %}

## Migration from Calico to Cilium

If your Kubernetes cluster previously used Calico, perform a careful, stepwise migration to avoid stale CRDs, iptables, or CNI config conflicts. Please follow the corresponding [knowledge base article](https://platform9.com/kb/kubernetes/calico-to-cilium-migration-on-pmk-workload-clusters).