Upgrade SMCP 5.6 patch-4 to SMCP 5.9

1. Take a backup of the 5.6 management plane by following the backup guide.

2. Destroy the older1.21 based nodelet cluster using the following commands.

Bash

airctl stop
airctl unconfigure-du --force 
airctl advanced-ddu delete-mgmt

1. Ensure the older/etc/nodelet/airctl-mgmt/certs directory is cleaned up.

2. Download newer 5.9 based artifacts using the download script. Run the install.sh script to extract and place the artifacts under /opt/pf9/airctl directory.

Bash

bash install.sh v-5.9.x-<airctl version>

Refer the SMCP 5.9 release notes for the exact airctl version used by the install.sh script.

For example, when upgrading to SMCP v5.9.0 :

Bash

bash install.sh v-5.9.0-2847602

When upgrading to SMCP v5.9.1 :

Bash

bash install.sh v-5.9.1-3097398

1. Ensure that the management cluster host has configuration of at least 8vCPU, 32GB memory and 250GB disk.

2. Install the pre-requisite container-selinux package if not installed.

3. Update the config files as suggested below and deploy the new1.25 based nodelet management cluster by running the following command.

Bash

airctl advanced-ddu create-mgmt

Ensure that the airctl-config.yaml file is updated to have the new airctl build number.

In the nodelet-bootstrap.yaml file ensure that systemImages param is set to use newer LTS3 kubedu and nodelet image tar files (the release you are upgrading to). And the userImages section points to previous version of kubedu image tars (the release you are upgrading from).

Also add isAirgapped: true in nodelet-bootstrap.yaml if it is offline mode. A snippet of the nodelet-bootstrap.yaml is shown below.

YAML

isAirgapped: true
systemImages:
- /opt/pf9/airctl/imgs/kubedu-imgs-v-5.9.0-2847602.tar.gz
- /opt/pf9/airctl/imgs/nodelet-imgs-v-5.9.0-2847602.tar.gz
userImages:
- /home/centos/patch2/kubedu-imgs-v-5.6.7-2624593.tar.gz
- /home/centos/patch4/kubedu-imgs-v-5.6.7-2658688.tar.gz

1. Run the restore command by following the restore guide for the 5.6 management plane on top of 1.25 nodelet cluster.

2. Ensure that the DU is in ready state by running the following command and waiting for it to be ready. Check the status of hosts restored.

airctl status
airctl host-status

After restore, we should be able to login to UI and view the existing workload clusters.

1. To upgrade the DU components to5.9, follow the upgrade guide. Note: In order to avoid any possible ImagePullBackOff errors during upgrade/upgrade-hosts steps that can happen in some rare cases, refer additional-information

Wait until the DU is back in ready state after upgrade by using following command.

Bash

airctl status
{% endtab %}
{% tab language="none" %}
// Click to edit code

Login to the GUI and ensure that it shows the version as 5.9.

1. At this point the DU components are upgraded to LTS3. We can now push the new images for 1.24 and 1.25 Kubernetes workload clusters to our private docker registry (if using privateRegistryType: DU in airctl-config.yaml) using the following command.

Bash

sudo airctl advanced-du push-images --img-tar [tar file name]

The new images are stored under the /opt/pf9/airctl/imgs/ directory and each file has a k8s prefix. A handy one-liner to push all images is.

Bash

ls -l /opt/pf9/airctl/imgs/k8s* | xargs -I{} sh -c "sudo /opt/pf9/airctl/airctl --config /opt/pf9/airctl/conf/airctl-config.yaml advanced-du push-images --img-tar {}"

1. We can upgrade the host side components, by running the following 2 commands.

Bash

airctl configure-hosts --reset-pf9-managed-docker-conf --skip-docker-install --skip-docker-img-import
airctl upgrade-hosts

After the upgrade-hosts command, wait for DU task state to be in ready state again by running the following command.

Check the host status after upgrading hosts.

Bash

airctl status
airctl host-status

At this point all the components have been upgraded to SMCP 5.9 (LTS3). Now, we can proceed with the actual workload cluster upgrades either using the GUI or API.

Consul Issue:

After restore, errors seen in the consul pod saying 'ACL not found', which are harmless and can be ignored.

So, we are currently skipping the consul upgrade in the upgrade step and installing the upgraded consul in the restore process itself.