Cilium Cni

Platform9 Managed Kubernetes (PMK) supports Cilium as the Container Network Interface (CNI) for secure, high-performance cluster networking powered by eBPF. This page explains the architecture, prerequisites, installation options, configuration choices, migration guidance from Calico, and validation/troubleshooting steps aligned with PMK best practices.

For a conceptual introduction to Cilium and Hubble, see the Cilium introduction

Support Matrix and Recommendations

Category	Supported	Recommended
OS (hosts)	Minimum Ubuntu 22.04	Ubuntu 24.04
MTU	Align with underlay; validate path MTU	9000 (Jumbo)
Encapsulation / Routing	VXLAN or native routing based on network capabilities	VXLAN routing

Best practices

• Use eBPF masquerade with kube-proxy replacement for lower latency and reduced conntrack pressure.

• Set MTU to match end-to-end path; if using VXLAN over jumbo-capable underlay, set 9000 on NICs and size Cilium MTU accordingly (account for overlay overhead).

• Ensure nodes have required kernel features for eBPF (modern kernels on Ubuntu 22.04/24.04 meet this by default).

Prerequisites

In addition to having a Kubernetes cluster provisioned and healthy and administrative kubectl access, there are requirements for using Cilium as the CNI implementation:

• Subnets on a network should not coincide. You cannot have two clusters with container CIDR or service CIDR being the same on one L2 domain or network.

• Outbound connectivity to pull Cilium images and Helm charts (or a mirrored registry).

Create a Cilium-enabled Cluster

While creating a new Kubernetes cluster using the PMK UI, in the wizard under Network Configuration, select Cilium as the network backend.

Alternative Manual Installation Options

For manual Cilium installation, choose one path: Helm (flexible, GitOps-friendly) or Cilium CLI (simplified). Ensure any legacy CNI artifacts are removed before applying Cilium.

Option A: Installation with Helm

1. Add chart repo and update indices.

2. Install cilium into kube-system with PMK-aligned values: VXLAN routing, eBPF masquerade, kube-proxy replacement, MTU sizing.

3. Verify daemonset rollout and agent health across all nodes.

Option B: Install with Cilium CLI

1. Install cilium CLI on an admin workstation.

2. Apply installation with flags that match PMK recommendations (VXLAN, eBPF masquerade, kube-proxy replacement).

3. Run post-install validation (cilium status, cilium connectivity test).

If the init container fails to copy binaries into /opt/cni/bin, fix permissions on each node:

Bash

sudo chown root:root /opt/cni/bin
sudo chmod 755 /opt/cni/bin
sudo chown -R root:root /opt/cni/bin
sudo chmod -R 755 /opt/cni/bin

Post-Install Validation

1. Agent Health: Cilium status should report OK for all subsystems and nodes.

Bash

kubectl -n kube-system exec ds_cilium -- cilium status

1. Connectivity: Validate pod-to-pod, pod-to-service, DNS, and egress.

Bash

kubectl run test-a --image=busybox --restart=Never -- sleep 3600
kubectl run test-b --image=busybox --restart=Never -- sleep 3600
kubectl exec test-a -- ping -c3 $(kubectl get pod test-b -o jsonpath='{.status.podIP}')

kubectl create deploy web --image=nginx --replicas=2
kubectl expose deploy web --port=80
kubectl exec test-a -- wget -qO- http://web

kubectl run -it --rm --restart=Never --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 dns -- nslookup kubernetes.default
kubectl exec test-a -- wget -qO- http://example.com

1. NetworkPolicy: Confirm deny-by-default and allow-specific behavior.

YAML

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: demo-deny
spec:
podSelector:
    matchLabels:
        app: web
policyTypes:
- Ingress

YAML

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-from-test-a
spec:
podSelector:
    matchLabels:
        app: web
ingress:
- from:
    - podSelector:
        matchLabels:
            run: test-a

Bash

kubectl label deploy web app=web --overwrite
kubectl apply -f demo-deny.yaml
kubectl apply -f allow-from-test-a.yaml
# Should succeed
kubectl exec test-a -- wget -qO- http://web
# Should fail
kubectl exec test-b -- wget -qO- http://web || echo "blocked as expected"

Migration from Calico to Cilium

If your Kubernetes cluster previously used Calico, perform a careful, stepwise migration to avoid stale CRDs, iptables, or CNI config conflicts. Please follow the corresponding knowledge base article.