Platform9 Host Components
PF9 Host Side Services
This is a list of the services that run on the master/worker nodes.
pf9-hostagent
The pf9-hostagent service is required by the hosts for management by Platform9. Its function introduces a method to install and configure the platform9 apps on the host using roles. The hostagent receives requests from the management plane to apply a node role on the host. It then updates and reports the success or failure of the role apply action back to the management plane.
When it receives the request from the management plane to push a role, it downloads the role’s packages, installs them, and finally executes the configuration scripts using the role data that comes as part of the payload. Hostagent also sends a periodic heartbeat to the management plane via the pf9-comms service. It includes CPU, memory, storage, and network information in the heartbeat.
Additionally, the pf9-hostagent monitors currently running services for the applied role and checks configuration matches with the configuration stored on the management plane in the host role mapping database. For Platform9 Managed Kubernetes (PMK), the host role is known as pf9-kube. It keeps the role configuration consistently applied on the host with the information stored in the management plane, e.g.:
The keepalived configuration for VIP. If the VIP network interface is modified in the configuration file on the host and keepalived is restarted, the hostagent will reset the changes back to the original network interface in keepalived during its next check.
It also monitors pf9-comms and restarts it if it is found to be dead.
The pf9-hostagent is also responsible for updating roles on the host. Host role apply, modify and delete (for example ‘pf9-kube’) are directed from the management plane using the pf9-hostagent. Host role updates are received on the host by the hostagent to update pf9 components on the host during an upgrade from Platform9.
pf9-comms
pf9-comms is used as a single point of communication with the management plane for Platform9-managed host services. For host-side Platform9 services like pf9-hostagent to talk to the management plane, Platform9 creates a tunnel from the host to the management plane that channels communication. The only requirement for this to work is an outbound TCP 443 connection from the host to the management plane.
pf9-comms acts as a multiplexer intercepting requests arriving at multiple localhost ports of host pf9 services and sends them over TCP 443 to the management plane. On a PMK cluster, pf9-hostagent and pf9-muster connect to a port on localhost that gets multiplexed in pf9-comms. The requests from pf9-comms are intercepted by an ingress service in the management plane, demultiplexed, and passed on to the intended receiver. pf9-comms uses TLS with trusted certificates to connect with the management plane ingress service.
COMMAND PID USER FD TYPE DEVICE SIZE_OFF NODE NAME
pf9-comms 17951 pf9 37u IPv4 46812429 0t0 TCP mav-3-1:41768->airctl-1.pf9.localnet:https (ESTABLISHED)
pf9-comms 17951 pf9 41u IPv4 46812664 0t0 TCP mav-3-1:41784->airctl-1.pf9.localnet:https (ESTABLISHED)
pf9-comms 17951 pf9 43u IPv4 46812674 0t0 TCP mav-3-1:41788->airctl-1.pf9.localnet:https (ESTABLISHED)
pf9-hostd 17814 pf9 10u IPv6 47533401 0t0 TCP localhost:38486->localhost:amqp (ESTABLISHED)
pf9-muste 28887 pf9 3u IPv4 47534314 0t0 TCP localhost:49962->localhost:amqp (ESTABLISHED)
pf9-muste 28887 pf9 5u IPv4 47538143 0t0 TCP localhost:50316->localhost:amqp (ESTABLISHED)
pf9-comms 96362 pf9 14u IPv4 47532817 0t0 TCP localhost:amqp (LISTEN)
pf9-comms 96362 pf9 15u IPv6 47532818 0t0 TCP localhost:amqp (LISTEN)
pf9-comms 96362 pf9 34u IPv6 47533404 0t0 TCP localhost:amqp->localhost:38486 (ESTABLISHED)
pf9-comms 96362 pf9 36u IPv4 47533407 0t0 TCP localhost:amqp->localhost:49962 (ESTABLISHED)
pf9-comms 96362 pf9 38u IPv4 47538961 0t0 TCP localhost:amqp->localhost:50316 (ESTABLISHED)pf9-sidekick
pf9-sidekick runs parallel to pf9-comms, but independently, and provides a backup channel for some of the hostagent's operations. When communication with pf9-hostagent is lost from the management plane, pf9-sidekick supports bundle upload operations and remote execution to allow diagnosis and recovery.
This service is typically used for debugging and talks to the host, providing a secondary channel to execute commands on managed hosts. It connects with the management plane component named sidekickserver.
pf9-muster
Muster is a monitoring and troubleshooting tool. It sends back host statistics such as memory and load usage. It also exposes a limited API allowing the Platform9 Support team to send whitelisted commands for troubleshooting. Communication is via pf9-comms.
pf9-nodelet
Nodelet comes into action after cluster creation and after the host has been discovered and authorized in the management plane. It refers to YAML files under /etc/pf9/nodelet/ for config options needed to configure the host as a Kubernetes master or worker node. Nodelet writes to /etc/pf9/kube.env, which is used by component phase scripts. It is responsible for starting, restarting, and stopping Kubernetes components in a controlled manner.
Nodelet can take corrective actions such as performing a partial restart or partial rollback of the Kubernetes components if they fail during startup. For example, if docker is not running, nodelet only attempts to restart the chain of components up to the docker configuration phase.
Nodelet continues to monitor the k8s stack once the node has been added to a cluster, invoking status checks every 1 minute. It creates a cgroup named pf9-kube-status to limit the CPU used during these checks.
Nodelet starts pf9 components when the pf9-kube role is pushed to the host and works with hostagent. It configures pf9 Kubernetes components in phases, retries only selective failed phases, and performs a full cleanup and restart of all components from the beginning on every 10th attempt if any component phase fails in the preceding 9 consecutive attempts.
pf9-kubelet
The Kubernetes kubelet runs as a systemd service on Platform9 managed Kubernetes hosts. On master nodes this service starts three master containers (api-server, scheduler, controller) as static pods. On worker nodes, kubelet communicates with the Kubernetes API on the masters to manage the pods on that node.
keepalived
This service is responsible for keeping the Master VIP highly available. Keepalived is installed and started only when VIP and VIP interface fields are chosen during cluster bootstrapping.
Standalone Docker Containers
etcd
etcd is the key-value data store backing all Kubernetes cluster data. In Platform9, etcd runs as a Docker container. pf9 etcd containers run on the master nodes and can be configured as single, three, or five node configurations.
kube-proxy
kube-proxy runs as a Docker container on every node, implementing components of the Kubernetes Service concept. kube-proxy maintains network rules on the nodes that permit network communication to Pods from inside or outside the cluster.
pf9-bouncer
pf9-bouncer runs as a Docker container and receives authentication validation requests from the Kubernetes API server. It is the Kubernetes cluster-level authentication service configured by default on Platform9 clusters, using Keystone as the identity provider. If the Kubernetes API has a valid authentication token in the request, pf9-bouncer passes the request on to the API server.
Pf9 Namespaces and Pods
pf9-olm
packageserver-844d4fb848-zmpxd
OLM internal pod for processing OLM package installation
pf9-olm
packageserver-844d4fb848-cpr4b
OLM internal pod for processing OLM package installation
pf9-olm
platform9-operators-df5bl
OLM repository pod; OLM operator fetches packages from this repo to install operators
pf9-olm
olm-operator-fbd9c955c-j85zb
OLM operator: watches OLM subscription objects and installs operators in response
pf9-olm
catalog-operator-d59cf9dfb-5t7s8
OLM catalog operator: works with olm operator and validates OLM packages before installing
pf9-olm
prometheus-operator-54dd4d9b-kxv9r
Prometheus operator: installed through OLM by creating an OLM subscription object
pf9-operators
monhelper-5c8558c46-p5hw2
Platform9 helper pod created as part of OLM package for Prometheus operators; installs and configures monitoring objects (prometheus, alertmanager, grafana, etc.)
pf9-monitoring
prometheus-system-0
Actual Prometheus instance installed through Prometheus operator
pf9-monitoring
grafana-986c774cf-p8w58
Grafana for UI visualization of Prometheus metrics; pre-configured to talk to Prometheus and includes built-in dashboards
pf9-monitoring
alertmanager-sysalert-0
Alertmanager instance configured with Prometheus to receive alerts; user must configure targets for delivery
pf9-monitoring
node-exporter-llgkj
Node Exporter daemonset, exports metrics for each kube node for Prometheus scraping
pf9-monitoring
node-exporter-nbw5k
Node Exporter daemonset, exports metrics for each kube node for Prometheus scraping
pf9-monitoring
node-exporter-rjtdg
Node Exporter daemonset, exports metrics for each kube node for Prometheus scraping
pf9-monitoring
kube-state-metrics-595cb5cc
kube-state-metrics exporter: exposes Kubernetes cluster metrics for Prometheus
platform9-system
pf9-sentry
pf9 UI queries the HTTP server running inside this pod to get a list of CSI drivers
pf9-addons
pf9-addon-operator-7f9784f867-ktn6z
Addon operator supporting addons such as coredns, metrics-server, dashboard, autoscaler, etc.
Etcd-backup
If etcd-backup is enabled at bootstrap, the etcd-backup directory is configured on the cluster. The default backup frequency is every 24 hours, but this is configurable. The etcd backup file is saved in /etc/pf9/etcd-backup on the master node. This backup can be used to recover cluster state after events such as etcd corruption.
Was this helpful?