# Release Notes ## Release Summary The Platform9 Managed Kubernetes (PMK) version 5.13 release is now generally available with active support for Kubernetes **v1.32**. This release brings new features, enhancements and critical bug fixes to improve overall user experience and stability. ## PMK 5.13.0 Release Highlights (Released 2025-07-18) #### New Features `Added`Added active support for Kubernetes **1.32** `Added` Support for **Cilium CNI** `Added` Support for Pod Security Admission (**PSA**) controller #### Feature Updates `Added` Updated libraries/ module dependencies to fix CVEs `Added` Monitoring support for MetalLB #### Deprecations, Feature Removal and EOL information {% hint style="danger" %} **Kubernetes 1.29 and lower EoL** * All clusters must be upgraded to at **least Kubernetes v1.30** before upgrading from **PMK 5.12.x to PMK 5.13** * **Kubernetes v1.29** and lower versions are marked as *End of Life* on **PMK 5.13** {% endhint %} {% hint style="warning" %} **Kubernetes 1.30 Deprecated** * **Kubernetes v1.30** is marked as deprecated. * New clusters should be created on **Kubernetes v1.31** or above. However, you can continue to create new clusters on Kubernetes v1.30 or above. {% endhint %} * There are no Operating Systems deprecations in PMK **5.13** release. Check the PMK support matrix here: [auto$](https://github.com/platform9/pcd-docs-gitbook/blob/main/kubernetes/support-matrix/README.md) #### Platform9 CLI The *pf9ctl* release 1.33 is now available ([release notes for pf9ctl v1.33](https://github.com/platform9/pf9ctl/releases/tag/1.33)) and can be installed by running the following command {% tabs %} {% tab title="Bash" %} ```bash bash <(curl -sL https://pmkft-assets.s3-us-west-1.amazonaws.com/pf9ctl_setup) ``` {% endtab %} {% endtabs %} ### Bug Fixes `Fixed` To resolve the request of using `etcdutl` instead of the deprecated `etcdctl` for restoring ETCD, which serves as the datastore for Kubernetes clusters, PMK now bundles the `etcdutl` binary alongside `etcdctl` since the upstream open source `etcd` project recommends `etcdutl` for snapshot restore operations going forward. `Fixed` Decommissioning a node did not remove the proxy configuration file from `/etc/systemd/system/containerd.service.d/00-pf9-proxy.conf`. This file prevented the host from being re-added to a cluster. Users were advised to remove the file manually to resolve the issue. `Fixed` When the internal package repository returned HTTP 403 errors through the proxy, `pf9ctl` prep-node failed to install required packages and crashed with a segmentation fault. `Fixed` After an ETCD backup, reattaching a master node to the cluster failed. Detached nodes remained stuck in a prolonged converging state and did not reach the ready state required for successful reattachment. `Fixed` Onboarding a node with `pf9ctl` 1.30 to a tenant on v5.11.0 with Kubernetes 1.30 connected the node to the Management Plane but left the `projectId` in `QbertDB` as `NULL`, which caused the node to appear in all tenants instead of being assigned to the correct one. `Fixed` In PMK versions 5.12.0 and 5.12.1, vouch failed to create new tokens and roles during redeployment when existing vouch data was present in Consul. The system incorrectly skipped creating new entries if previous entries existed, which caused deployment and node onboarding failures. #### Known Issues `Known Issue` All existing and new AWS clusters in PMK must be configured with an `is_update`flag and restricted security group rules. Without this cluster updates(such as AMI updated) and upgrades may fail. Please reach out to Platform9 support for this configuration. `Known Issue` During upgrade of a PMK cluster, uninstallation of pf9-kube package may be incomplete/ stuck, if there are any workloads whose associated containers cannot be cleanly stopped and removed. Contact platform9 support if this is observed. `Known Issue` On Rocky Linux 9 (tested on 9.2 and 9.4), platform9's pf9-kube package installs `iptables-services` as a dependency. With recent updates to the upstream repositories, installation will fail due to a missing dependency on `iptables-legacy-*` packages. `Known Issue` (On Rocky Linux 9) Users will need to install the legacy packages by running `dnf install iptables` or `dnf install iptables-utils` on workload cluster nodes. Since this is a recent upstream change, a solution will be provided in upcoming releases by packaging the required packages along with `pf9-kube` package. `Known Issue` AWS clusters using flannel CNI need to be updated to use port 2379 instead of 4001 from1.22 version onwards. Workaround is to go to the "Edit cluster" option on the UI and clicked on "Update cluster" without making any changes. This adds the 2379 ingress rule to the master ELB. `Known Issue` When a detach operation is performed on a master node in a multi master cluster, it takes approximately 30 minutes to complete all the detach operations and perform cleanup on the node. Therefore, if you want to reattach this node to any other cluster, you need to wait for the nodelet to stop all the phases and perform cleanup before attempting to reattach the node. `Known Issue` In some scenarios, after a node is removed from the qbert clusters, nodelet fails to cleanup the data. Workaround is to check and remove the /var/opt/pf9/kube directory if present, even after the node is deauthorized. `Known Issue` Cluster upgrade attempt is blocked on UI post a cluster upgrade failure due to nodes being in a converging/not converged state. `Known Issue` Kubelet authorization mode is marked set to AlwaysAllow instead of Webhook. `Known Issue` PMK Cloud provider created directly in Sunpike cannot be used to create qbert clusters. Qbert cloud providers will work to create both qbert and sunpike clusters. But cloud providers created directly in sunpike CANNOT be used to create qbert clusters. Please use the appropriate one based on your needs. ### Package Updates #### PMK 5.13 Latest Kubernetes Components List | **Component** | **Kubernetes 1.32** | **Kubernetes 1.31** | **Kubernetes 1.30** | | ------------------------------------------------- | --------------------------- | ---------------------------- | ---------------------------- | | KUBERNETES BUILD VERSION | 1.32.3-pmk.**55 `updated`** | 1.31.9-pmk.**136 `updated`** | 1.30.4-pmk.**185 `updated`** | | CONTAINERD\*\*`updated`\*\* | 1.7.**27** | 1.7.**27** | 1.7.**27** | | RUNC | 1.1.12 | 1.1.12 | 1.1.12 | | CORE-DNS | 1.11.1 | 1.11.1 | 1.11.1 | | METRICS SERVER | 0.6.4 | 0.6.4 | 0.6.4 | | METAL LB | 0.14.9 | 0.14.9 | 0.14.9 | | KUBERNETES DASHBOARD | 2.7.0 | 2.7.0 | 2.7.0 | | CLUSTER AUTO-SCALER AWS | 1.28.0 | 1.28.0 | 1.28.0 | | FLANNEL CNI | 0.24.2 | 0.24.2 | 0.24.2 | | CALICO CNI **`updated`** | 3.27.**5** | 3.27.**5** | 3.27.**5** | | **CILIUM CNI `new`** | 1.17.2 | 1.17.2 | 1.17.2 | | **CILIUM CLI** **`new`** | 0.18.3 | 0.18.3 | 0.18.3 | | ETCD | 3.5.12 | 3.5.12 | 3.5.12 | | CNI PLUGINS | 1.4.0 | 1.4.0 | 1.4.0 | | KUBEVIRT | 1.0.0 | 1.0.0 | 1.0.0 | | KUBEVIRT CDI | 1.57.0 | 1.57.0 | 1.57.0 | | ADVANCED NETWORKING OPERATOR (LUIGI) | 0.5.8 | 0.5.8 | 0.5.8 | | MONITORING - PROMETHEUS OPERATOR\*\*`updated`\*\* | 0.68.**2** | 0.68.**2** | 0.68.**2** | | PROFILE AGENT | 2.0.2 | 2.0.2 | 2.0.2 | | METAL3 | 1.1.1 | 1.1.1 | 1.1.1 | ## PMK 5.13.1 Release Highlights (Released 2025-07-28) ### Bug Fixes `Fixed` Added a Qbert API to update the CNI network plugin and container CIDR post CNI migration. Previously, after successful CNI migration, the console continued to display Calico as the CNI despite Cilium being active, resulting in misaligned visibility of the active networking backend for users and administrators.