# Upgrade Guide: SMCP 5.9 to SMCP 5.10

## Prerequisites

Before beginning the upgrade process, ensure the following requirements are met:

1. [Minimum Requirements](https://github.com/platform9/pcd-docs-gitbook/blob/main/PEC/requirements/README.md)
2. Minimum Kubernetes version for Workload Cluster should be `v1.26`.

## Upgrade Procedure

### 1. Backup Current System

Follow the [Backup Guide](https://github.com/platform9/pcd-docs-gitbook/blob/main/PEC/backup-guide/README.md) to back up the 5.9 management plane. Before proceeding, verify that the backup was created successfully.

### 2. Take etcd Snapshot

Before making any changes to the system, it's crucial to take a snapshot of the etcd cluster. This provides an additional recovery point if needed.

**a**. Check the etcd cluster status and identify the leader node:

{% tabs %}
{% tab title="Bash" %}

```bash
sudo /opt/pf9/pf9-kube/bin/etcdctl endpoint status --cluster -w=table --endpoints=http://127.0.0.1:2379 --cacert="/etc/pf9/kube.d/certs/etcdctl/etcd/ca.crt" --cert="/etc/pf9/kube.d/certs/etcdctl/etcd/request.crt" --key="/etc/pf9/kube.d/certs/etcdctl/etcd/request.key"
```

{% endtab %}
{% endtabs %}

Make a note of which node is the leader.

**b**. Take the etcd cluster backup:

{% tabs %}
{% tab title="None" %}

```none
sudo /opt/pf9/pf9-kube/bin/etcdctl snapshot save <path/backupfile.db> --cert /etc/pf9/kube.d/certs/etcdctl/etcd/request.crt --key /etc/pf9/kube.d/certs/etcdctl/etcd/request.key --cacert /etc/pf9/kube.d/certs/etcdctl/etcd/ca.crt --endpoints=https://<master_IP>:4001
```

{% endtab %}
{% endtabs %}

Replace `<path/backupfile.db>` with the desired path and filename for your backup and `<master_IP>` with the IP address of the master node.

**c**. Validate the etcd backup:

{% tabs %}
{% tab title="None" %}

```none
sudo /opt/pf9/pf9-kube/bin/etcdctl snapshot status --write-out=table <path/backupfile.db>
```

{% endtab %}
{% endtabs %}

Ensure that the snapshot status shows as valid before proceeding.

### 3. Destroy Old Cluster

Remove the existing 1.25-based nodelet cluster:

{% tabs %}
{% tab title="Bash" %}

```bash
airctl stop
# Stops all running services managed by airctl

airctl unconfigure-du --force 
# Removes DU configuration and forcefully cleans up resources

airctl advanced-ddu delete-mgmt
# Deletes the management plane components
```

{% endtab %}
{% endtabs %}

### 4. Clean Up Configuration

On all master nodes, clean up the certificate directory:

{% tabs %}
{% tab title="Bash" %}

```bash
rm -rf /etc/nodelet/airctl-mgmt/certs/*
# This command removes all files in the specified directory
```

{% endtab %}
{% endtabs %}

Verify the directory is empty before proceeding.

### 5. Download and Install New Artifacts

To obtain and install the SMCP 5.10 artifacts:

**a.** Download the download script:

{% tabs %}
{% tab title="None" %}

```none
wget https://raw.githubusercontent.com/platform9/support-locker/master/edge-stack/download.sh
```

{% endtab %}
{% endtabs %}

**b.** Make the script executable:

{% tabs %}
{% tab title="None" %}

```none
chmod u+x download.sh
```

{% endtab %}
{% endtabs %}

**c.** Run the download script:

{% tabs %}
{% tab title="None" %}

```none
./download.sh <PF9-SHARED-SECRET> v-5.10.0-<airctl_version>
```

{% endtab %}
{% endtabs %}

Replace `<PF9-SHARED-SECRET>` with your Platform9 shared secret.

**d.** After downloading the artifacts, verify their integrity using the following command:

{% tabs %}
{% tab title="Bash" %}

```bash
curl --user-agent <secret> https://pf9-airctl.s3-accelerate.amazonaws.com/<version>/md5sum.txt | md5sum -c -
```

{% endtab %}
{% endtabs %}

Replace `<secret>` with the same one used to download the airctl build from S3 and `<version>` with the SMCP version you're installing (e.g., v5.10.0).

**e.** Run the install.sh script to extract and place the artifacts under `/opt/pf9/airctl` directory.

{% tabs %}
{% tab title="Bash" %}

```bash
bash install.sh v-5.10.0-<airctl-version>
```

{% endtab %}
{% endtabs %}

Refer to the [SMCP 5.10 release notes](https://platform9.com/docs/5.10/PEC/smcp-5-10-release-notes) for the exact `airctl` version used by the `install.sh` script.

For example, when upgrading to SMCP v5.10.0 :

{% tabs %}
{% tab title="example" %}

```bash
bash install.sh v-5.10.0-3248001
```

{% endtab %}
{% endtabs %}

### 6. Configure New Management Cluster

Update configuration files before deployment:

1. Edit `airctl-config.yaml`:
   1. Update `hostAgentRepo` to use SMCP 5.10 images.
   2. `dockerRepo` to use SMCP 5.10 images.
2. Edit `nodelet-bootstrap.yaml`:
   1. Update `systemImages` to use SMCP 5.10 image files
   2. Set `userImages` to point to SMCP 5.9 image files
   3. Add `isAirgapped: true` for offline installations

Example `airctl-config.yaml` and `nodelet-bootstrap.yaml` snippet:

{% tabs %}
{% tab title="airctl-config.yaml" %}

```yaml
hostAgentRepo: /opt/pf9/airctl/hostagent-8-v-5.10.0-3248001.tar.gz
dockerRepo: /opt/pf9/airctl/docker-8-v-5.10.0-3248001.tar.gz
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="nodelet-bootstrap.yaml" %}

```yaml
isAirgapped: true
systemImages:
- /opt/pf9/airctl/imgs/kubedu-imgs-v-5.10.0-3248001.tar.gz
- /opt/pf9/airctl/imgs/nodelet-imgs-v-5.10.0-3248001.tar.gz
userImages:
- /home/centos/kubedu-imgs-v-5.9.0-2847602.tar.gz
```

{% endtab %}
{% endtabs %}

### 7. Deploy New Management Cluster

Deploy the new `1.27` based nodelet management cluster:

{% tabs %}
{% tab title="Bash" %}

```bash
airctl advanced-ddu create-mgmt --verbose
# This command creates and deploys the new management plane components
```

{% endtab %}
{% endtabs %}

Initialize the management plane:

{% tabs %}
{% tab title="None" %}

```none
sudo airctl init
```

{% endtab %}
{% endtabs %}

Monitor the deployment progress and check for any errors.

### 8. Restore from Backup

Restore the SMCP 5.9 by following the [restore guide](https://github.com/platform9/pcd-docs-gitbook/blob/main/docs/v5.9/PEC/restore-guide/README.md) for the `5.9` management plane on top of the `1.27` nodelet cluster.

{% hint style="warning" %}
**Consul Issue:**

After restoring, you might see errors in the Consul pod saying `ACL not found`. These errors are harmless and can be ignored.

Therefore, SMCP currently skips the Consul upgrade in the upgrade step and installs the upgraded Consul during the restore process itself.
{% endhint %}

### 9. Verify DU Status

Check that the DU is in a ready state:

{% tabs %}
{% tab title="Bash" %}

```bash
airctl status
# This command displays the current status of the DU
```

{% endtab %}
{% endtabs %}

Wait until all components show as ready before proceeding.

### 10. Upgrade DU Components

To upgrade the DU components to`5.10`, follow the [upgrade guide](https://github.com/platform9/pcd-docs-gitbook/blob/main/PEC/Upgrade-guide/README.md).

{% hint style="info" %}
**NOTE**

To prevent potential ImagePullBackOff errors, refer to [additional-information](https://platform9.com/kb/PEC/while-upgrade-from-lts2-patch2-to-scmp--management-plane-cluster#additional-information)
{% endhint %}

After upgrade, verify DU status again:

{% tabs %}
{% tab title="Bash" %}

```bash
airctl status
{% endtab %}
{% tab language="none" %}
// Click to edit code
```

{% endtab %}
{% endtabs %}

### 11. Verify Upgrade in GUI

Log in to the management UI and confirm the version is displayed as 5.10.

### 12. Push New Kubernetes Images

Push images for Kubernetes **1.27**, **1.28**, and **1.29** to the registry (if applicable):

{% tabs %}
{% tab title="Bash" %}

```bash
sudo airctl advanced-du push-images --img-tar [tar file name]
```

{% endtab %}
{% endtabs %}

To push all new images at once:

{% tabs %}
{% tab title="Bash" %}

```bash
ls -1 /opt/pf9/airctl/imgs/k8s* | xargs -I{} sh -c "sudo /opt/pf9/airctl/airctl --config /opt/pf9/airctl/conf/airctl-config.yaml advanced-du push-images --img-tar {}"
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
**NOTE**

If the **privateRegistryType** value is **custom,** the images must be manually pushed to the custom registry.
{% endhint %}

### 13. Upgrade Host Components

Upgrade the host-side components:

{% tabs %}
{% tab title="Bash" %}

```bash
airctl configure-hosts --reset-pf9-managed-docker-conf --skip-docker-install --skip-docker-img-import --verbose
# This command configures hosts with new settings

airctl hosts-status
# Check the status of hosts before proceeding

airctl upgrade-hosts --verbose
# This command upgrades the host components
```

{% endtab %}
{% endtabs %}

### 14. Verify Status and Version

After upgrading, verify management plane services are ready. At this point all the components have been upgraded to SMCP 5.10 (LTS4). Run the following command to verify.

{% tabs %}
{% tab title="Bash" %}

```bash
airctl status
# Check the status of management plane

airctl version
# The output should be: v-5.10.0-3248001

kubectl get nodes -o wide
# Ensure all the nodes are in Ready state
```

{% endtab %}
{% endtabs %}

Now, proceed with the actual workload cluster upgrades either using the [GUI](https://github.com/platform9/pcd-docs-gitbook/blob/main/kubernetes/upgrade-cluster/README.md) or [API](https://platform9.com/docs/qbert#postupgrade-a-cluster-identified-by-the-uuid).