General Networking

The section outlines and describes several plugins available for Luigi like Node Feature Discovery and Host Configuration templates.

• HostPlumber

  • A subset of Luigi, an operator to configure/prep networking on the node and retrieve node details.The Hostplumber operator specifies two new CRD types called HostNetworkTemplate and HostNetwork. These are used to configure the host’s networking and view node networking state.

    • See: https://github.com/platform9/luigi/blob/master/hostplumber/README.md

    • Use to create SRIOV VFs, configure OVS, create VLAN interfaces, etc...

    • Recommended unless you have your own tooling to configure nodes

• Multus

  • Almost always required - the only way K8S can support multiple CNIs and networks

  • Perquisite for OVS

• SR-IOV CNI

  • This plugin enables the configuration and usage of SR-IOV VF networks in Kubernetes

• SRIOV Network Device Plugin

  • The SR-IOV Network Device Plugin is Kubernetes device plugin for discovering and advertising networking resources in the form of SR-IOV virtual functions (VFs) and PCI physical functions (PFs) available on a Kubernetes host.OpenVSwitch daemon & CLI tools

• OVS CNI plugin

  • Allows user to define Kubernetes networks on top of Open vSwitch bridges available on nodes.Macvlan, IPvlan

  • Requires Multus to be installed on hosts.

• Whereabouts IPAM driver

  • Required for dynamic IP assignment without an external DHCP service.

• Node Feature Discovery

  • A component of the Luigi operator that discovers features like SRIOV enablement, kernel version, swap, RAM, CPU, disks and other info at the host level. It creates labels on the nodes with those key-values on each node on the cluster.

Configuration

namespace: Each plugin will take in a namespace override to deploy, default namespace otherwise

Image override: Each plugin will take an image override field to use a non-default/stable container image. This is not guaranteed to work, especially if the plugin's CRs have changed. It should only be used for dev-test or bug fixes

imagePullPolicy: By default IfNotPresent

privateRegistryBase: Some airgapped env's may have a custom container registry. If this is specified, it will replace the public container registry URL (docker.io, gcr.io, quay, etc..) with this path

Each plugin may or may not have some further specific configuration. Here are the current options as of release v0.3:

• HostPlumber - none

• Multus - none

• SRIOV - none

• Node-feature-discovery - none

• OVS - none

• Whereabouts

  • ipReconcilerSchedule - specify the CronJob schedule of the whereabouts IP cleanup Job

  • ipReconcilerNodeSelector - specify the nodeSelector Labels on which to schedule the ip-reconciler

NetworkPlugin CRD

In it's current phase, only one instance of the CRD is supported. It will reflect the final, desired state of all plugins to be deployed.

If it is present, Luigi will ensure that the plugin is deployed and upgraded. If missing and re-applied, Luigi will remove the plugin if it was previously managing it.

YAML

apiVersion: plumber.k8s.pf9.io/v1
kind: NetworkPlugins
metadata:
  name: networkplugins-sample11
spec:
  # Add fields here
  #privateRegistryBase: "localhost:5100"
  plugins:
    hostPlumber: {}
    nodeFeatureDiscovery: {}
    multus: {}
    whereabouts:
      ipReconcilerSchedule: "*/1 * * * *"
      ipReconcilerNodeSelector:
         foo: bar
    # SRIOV actually consists of two plugins - the CNI, and the device-plugin
    # Use HostPlumber to create the actual VFs
    sriov: {}
    ovs: {}

The above will deploy all the plugins specified in the default namespace. To override the namespace, and deploy in kube-system:

YAML

apiVersion: plumber.k8s.pf9.io/v1
kind: NetworkPlugins
metadata:
  name: networkplugins-sample11
spec:
  # Add fields here
  #privateRegistryBase: "localhost:5100"
  plugins:
    hostPlumber:
      namespace: "kube-system"
    nodeFeatureDiscovery: {}
    multus:
      namespace: "kube-system"
    whereabouts:
      namespace: "kube-system"
    sriov:
      namespace: "kube-system"

This will create a secondaryCNId and other related plugins deployed. Use the HostPlumber plugin, to prep the nodes before creating Multus networks and assigning them to pods