How To Luigi Macvlan

MacVLAN type definition

Create Network Attach-Definition macvlan type

YAML

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf-1
spec:
  config: '{
      "cniVersion": "0.3.0",
      "name": "macvlan-conf-1",
      "type": "macvlan",
      "master": "eth1",
      "mode": "bridge",
      "ipam": {
        "type": "whereabouts",
        "range": "192.168.60.0/24",
        "range_start": "192.168.60.20",
        "range_end": "192.168.60.50",
        "gateway": "192.168.60.1"
      }
    }'

The master key value is the reference to the *_second nic *_in our worker nodes.

Bash

$ kubectl apply -f networkattachdefinition.yaml

Network Attach Definition Validation macvlan type

Let’s validate our work by listing and describing our new Network Attach Definition

Bash

$ $ kubectl get net-attach-def
NAME               AGE
macvlan-conf-1     134m

YAML

# kubectl describe net-attach-def macvlan-conf-1
Name:         macvlan-conf-1
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"k8s.cni.cncf.io/v1","kind":"NetworkAttachmentDefinition","metadata":{"annotations":{},"name":"macvlan-conf-1","namespace":"...
API Version:  k8s.cni.cncf.io/v1
Kind:         NetworkAttachmentDefinition
Metadata:
  Creation Timestamp:  2020-09-18T13:38:22Z
  Generation:          2
  Resource Version:    6958535
  Self Link:           /apis/k8s.cni.cncf.io/v1/namespaces/default/network-attachment-definitions/macvlan-conf-1
  UID:                 99a902e1-8fd1-4f37-81d7-229361029455
Spec:
  Config:  { "cniVersion": "0.3.0", "type": "macvlan", "master": "eth1", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.60.0/24", "range_start": "192.168.60.20", "range_end": "192.168.60.50", "gateway": "192.168.60.1", "routes": [{"dst": "0.0.0.0/0"}], "gateway": "192.168.60.1" } }
Events:    <none>

Pod Definitions with 2 interfaces

In order for pods to be created with an additional nic, and the same principle applies for adding more than two NICs to the pod, the pod definition should be called with network annotations making reference to the Network Attach Definition(s) please use the following pods definitions to create a testbed.

Create Pods with mcvlan interfaces

YAML

apiVersion: v1
kind: Pod
metadata:
  name: pod0-case-01
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf-1
spec:
  containers:
  - name: pod0-case-01
    image: docker.io/centos/tools:latest
    command:
    - /sbin/init

YAML

apiVersion: v1
kind: Pod
metadata:
  name: pod1-case-01
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf-1
spec:
  containers:
  - name: pod1-case-01
    image: docker.io/centos/tools:latest
    command:
    - /sbin/init

Deploy the new pods

Bash

$ kubectl apply -f pod1-case1.yaml
$ kubectl apply -f pod0-case1.yaml

Validate Pods Creation with macvlan interfaces

Let’s validate your work by confirming that the pods got created with an additional interface by doing the following commands:

Bash

$ kubectl get pods -o wide
NAME                                      READY   STATUS    RESTARTS   AGE    IP             NODE            NOMINATED NODE   READINESS GATES
pod0-case-01                              1/1     Running   0          142m   10.135.41.90   192.168.50.16   <none>           <none>
pod1-case-01                              1/1     Running   0          140m   10.135.41.91   192.168.50.16   <none>           <none>

Bash

$ kubectl exec -it pod0-case-01 -- ip -d address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if585: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default
    link/ether 16:66:b2:53:e4:f8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 0
    veth numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    inet 10.135.41.90/24 brd 10.135.41.255 scope global eth0
       valid_lft forever preferred_lft forever
4: net1@if401: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether 02:5b:b8:96:0a:d7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 0
    macvlan mode bridge numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    inet 192.168.60.20/24 brd 192.168.60.255 scope global net1
       valid_lft forever preferred_lft forever

$ kubectl exec -it pod1-case-01 -- ip -d address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if586: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default
    link/ether fa:61:88:fb:53:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 0
    veth numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    inet 10.135.41.91/24 brd 10.135.41.255 scope global eth0
       valid_lft forever preferred_lft forever
4: net1@if401: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether ca:c3:73:0e:f3:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 0
    macvlan mode bridge numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
    inet 192.168.60.21/24 brd 192.168.60.255 scope global net1
       valid_lft forever preferred_lft forever

Connectivity Testing

East-West macvlan traffic

Bash

$ kubectl exec -it pod0-case-01 -- ping -c 5 192.168.60.21
PING 192.168.60.21 (192.168.60.21) 56(84) bytes of data.
64 bytes from 192.168.60.21: icmp_seq=1 ttl=64 time=0.289 ms
64 bytes from 192.168.60.21: icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from 192.168.60.21: icmp_seq=3 ttl=64 time=0.059 ms
64 bytes from 192.168.60.21: icmp_seq=4 ttl=64 time=0.062 ms
64 bytes from 192.168.60.21: icmp_seq=5 ttl=64 time=0.058 ms

--- 192.168.60.21 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 0.058/0.105/0.289/0.092 ms

Bash

$ kubectl exec -it pod1-case-01 -- ping -c 5 192.168.60.20
PING 192.168.60.20 (192.168.60.20) 56(84) bytes of data.
64 bytes from 192.168.60.20: icmp_seq=1 ttl=64 time=0.253 ms
64 bytes from 192.168.60.20: icmp_seq=2 ttl=64 time=0.044 ms
64 bytes from 192.168.60.20: icmp_seq=3 ttl=64 time=0.050 ms
64 bytes from 192.168.60.20: icmp_seq=4 ttl=64 time=0.047 ms
64 bytes from 192.168.60.20: icmp_seq=5 ttl=64 time=0.069 ms

--- 192.168.60.20 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.044/0.092/0.253/0.081 ms

North-South macvlan traffic:

Bash

$ kubectl exec -it pod0-case-01 -- ping -c 5 192.168.60.1
PING 192.168.60.1 (192.168.60.1) 56(84) bytes of data.
64 bytes from 192.168.60.1: icmp_seq=1 ttl=64 time=0.925 ms
64 bytes from 192.168.60.1: icmp_seq=2 ttl=64 time=0.536 ms
64 bytes from 192.168.60.1: icmp_seq=3 ttl=64 time=0.483 ms
64 bytes from 192.168.60.1: icmp_seq=4 ttl=64 time=0.640 ms
64 bytes from 192.168.60.1: icmp_seq=5 ttl=64 time=0.488 ms

Bash

$ kubectl exec -it pod1-case-01 -- ping -c 5 192.168.60.1
PING 192.168.60.1 (192.168.60.1) 56(84) bytes of data.
64 bytes from 192.168.60.1: icmp_seq=1 ttl=64 time=1.31 ms
64 bytes from 192.168.60.1: icmp_seq=2 ttl=64 time=0.567 ms
64 bytes from 192.168.60.1: icmp_seq=3 ttl=64 time=0.473 ms
64 bytes from 192.168.60.1: icmp_seq=4 ttl=64 time=0.605 ms
64 bytes from 192.168.60.1: icmp_seq=5 ttl=64 time=0.647 ms

--- 192.168.60.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4002ms
rtt min/avg/max/mdev = 0.473/0.721/1.313/0.301 ms