# Virtual Machine Migration

VM migration in <code class="expression">space.vars.product\_name</code> refers to the process of moving a running or stopped virtual machine (VM) from one compute host (hypervisor) to another within the same <code class="expression">space.vars.product\_name</code> environment. This is a fundamental capability for maintaining system availability, performing infrastructure maintenance, and balancing workloads in data centers.

## Common Use Cases

Following are the common scenarios in which migration is initiated in <code class="expression">space.vars.product\_name</code>.

* **Maintenance mode:** When you invoke [Maintenance Mode](https://docs.platform9.com/private-cloud-director/virtualized-clusters/add-hosts-virtualized-cluster/maintenance-mode) on host, VMs are live migrated to other compatible hosts in the virtualized cluster.
* **Load Balancing:** <code class="expression">space.vars.product\_name</code> [Dynamic Resource Rebalancing (DRR)](https://docs.platform9.com/private-cloud-director/virtualized-clusters/virtualized-cluster/dynamic-resource-rebalancing-drr) uses live migration to distribute VMs more evenly across hosts in a virtualized cluster to optimize resource utilization and prevent individual hosts from becoming overloaded.
* **Host Failure (VM Evacuation):** <code class="expression">space.vars.product\_name</code> [Virtual Machine High Availability (VM HA)](https://docs.platform9.com/private-cloud-director/virtualized-clusters/virtualized-cluster/virtual-machine-high-availability-vm-ha) service uses VM evacuation to recover VMs from a failing or failed host.

## Types of VM Migration

<code class="expression">space.vars.product\_name</code> supports following types of VM migrations:

* Cold migration
* Live migration
* VM evacuation

Let's dive into the specifics of each.

## Cold Migration

Cold migration is a method of moving a virtual machine from one host to another, where the VM is `shut down`` ``(powered off)`or `suspended` during the process. The VM disks and configuration files are transferred to the destination host as part of the cold migration process.

**Downtime:** Cold migration essentially requires VM downtime, as the VM must be powered off or suspended before it can be cold migrated.

### Cold Migration Prerequisites

* VM should be powered off or suspended.
* Cold migration **uses the management network** for copying over virtual machine disk information. Make sure that the management network interface has **sufficient network bandwidth** **and capacity** to support **disk copy operation**.
* All hosts within the virtualized cluster must meet [CPU Model Pre-requisites](https://docs.platform9.com/private-cloud-director/add-hosts-virtualized-cluster/cpu-model#cpu-model-pre-requisites).
* Source and destination hosts must run the same operating system version. For more information, refer to the [Operating System Compatibility Requirements](#operating-system-compatibility-requirements).

#### Using CLI

1. Authenticate [pcdctl CLI](https://docs.platform9.com/private-cloud-director/reference/pcdctl-command-line)
2. Run the command below to perform the cold migration. The command below will cold migrate the VM to a suitable host within the virtualized cluster.

{% tabs %}
{% tab title="Bash" %}

```bash
$ pcdctl server migrate <VM_UUID>
```

{% endtab %}
{% endtabs %}

3. To migrate VM to a specific host use the below command:

{% tabs %}
{% tab title="BASH" %}

```bash
$ pcdctl server migrate --host <HOST_UUID> <VM_UUID>
```

{% endtab %}
{% endtabs %}

#### Using UI

* Log in to the UI and select Virtual Machine menu on the left side nav bar.
* Select the desired VM then click on the "other" option on the action bar and select the `migrate` action.
* Select the target hypervisor host and hit the "Migrate VM" button.

## Live Migration

Live migration is a method of moving a virtual machine from one host to another, where the VM remains **running** throughout the migration process.

The live migration process copies a virtual machine's memory from the source to the destination host while the VM is running. Any memory pages that get modified or "dirtied" on the VM during this time are copied over again. Finally, the VM enters a brief pause period during which its remaining memory and CPU state are copied over to the destination host, and finally, the VM is resumed on the destination host.

**Downtime:** The VM will experience a brief downtime (typically milliseconds to seconds) during the pause period before resuming on the destination host.

Live migration can be classified further by the way it treats virtual machine storage:

* **Live Migration for a VM using Ephemeral Shared Storage:** The virtual machine has an ephemeral root disk that is located on [Ephemeral Shared Storage](https://docs.platform9.com/private-cloud-director/storage/ephemeral-storage#ephemeral-shared-storage).
  * Live migration enables the testing and validation of whether the source and target hosts are utilizing the same underlying shared storage.
  * When validated, live migration is performed without copying over VM disk.
* **Live Migration of a VM using Ephemeral Local Storage:** The Virtual machine has an ephemeral root disk that is located on [Ephemeral Local Storage](https://docs.platform9.com/private-cloud-director/storage/ephemeral-storage#ephemeral-local-storage).
  * Live migration will copy over the entire virtual machine root disk from the source host to the destination host.
* **Live Migration of a VM using Volumes only:** The Virtual machine is using [block storage volumes](https://docs.platform9.com/private-cloud-director/storage/volume), rather than ephemeral disk.
  * Live migration will migrate the VM without performing a storage copy. The same volumes will continue to be attached to the VM once it's migrated to the target host.

### Live Migration Prerequisites

* All hosts within the virtualized cluster must run the same major and minor version of Ubuntu.
* Live migration uses the **management network to copy virtual machine memory** and, optionally, disk information. Make sure that the management network interface has **sufficient network bandwidth** **and capacity** to support **VM memory migration traffic**.
  * Support for adding a separate network interface for live migration is coming in the future.
* When using live migration in a production setup, specifically for features like [Dynamic Resource Rebalancing (DRR)](https://docs.platform9.com/private-cloud-director/virtualized-clusters/virtualized-cluster/dynamic-resource-rebalancing-drr), it is recommended to configure all virtual machines to use **shared storage only** for the best performance of the migration operation.
* When using [Ephemeral Local Storage](https://docs.platform9.com/private-cloud-director/storage/ephemeral-storage#ephemeral-local-storage), live migration requires **sufficient network bandwidth and capacity** to **support virtual machine disk copy**.
  * This is not required when using [Ephemeral Shared Storage](https://docs.platform9.com/private-cloud-director/storage/ephemeral-storage#ephemeral-shared-storage) at the virtualized cluster level or using [Block Storage Volumes](https://docs.platform9.com/private-cloud-director/storage/volume) exclusively for all virtual machines.
* All hosts within the virtualized cluster must meet [CPU Model Pre-requisites](https://docs.platform9.com/private-cloud-director/add-hosts-virtualized-cluster/cpu-model#cpu-model-pre-requisites) for live migration to work. Read more about [CPU Models](https://docs.platform9.com/private-cloud-director/virtualized-clusters/add-hosts-virtualized-cluster/cpu-model).
* Source and destination hosts must run the same operating system version. For more information, refer to the [Operating System Compatibility Requirements](#operating-system-compatibility-requirements).

{% hint style="warning" %}
**WARNING**

Live migration between hosts with different Ubuntu versions (22.04 and 24.04) is not supported and will result in migration failures. Ensure all hosts in your virtualized cluster run the same operating system version.
{% endhint %}

#### Live Migrate Using CLI

1. Authenticate [pcdctl CLI](https://docs.platform9.com/private-cloud-director/reference/pcdctl-command-line)
2. Run the command below to perform the live migration.

{% hint style="info" %}
**NOTE**

Here target host UUID is mandatory.
{% endhint %}

{% tabs %}
{% tab title="Bash" %}

```bash
pcdctl server migrate --live-migration <VMUUID> --host <HOST_UUID>
```

{% endtab %}
{% endtabs %}

#### Live Migrate Using UI

1. Log in to the <code class="expression">space.vars.product\_acronym</code> UI and select **Virtual Machine**.
2. On Virtual Machine search for a specific VM and select the VM.
3. Click **other** tab and then select the **Migrate**.

This will list down the eligible target hypervisors for Live migration.

4. Select the target hypervisor and click **Migrate VM**.

## VM Evacuation

VM Evacuation is specifically designed for disaster recovery. If a host fails or becomes unresponsive, VM evacuation attempts to restart a VM that was running on the failed host onto a healthy host.

### VM Evacuation Prerequisites

* VM evacuation can only be performed for VMs using [Ephemeral Shared Storage](https://docs.platform9.com/private-cloud-director/storage/ephemeral-storage#ephemeral-shared-storage) or [Block Storage Volumes](https://docs.platform9.com/private-cloud-director/storage/volume). If the VM is using [Ephemeral Local Storage](https://docs.platform9.com/private-cloud-director/storage/ephemeral-storage#ephemeral-local-storage), evacuation can not work as the virtual machine disk can not be copied over from the source host to a target host.
* All hosts within the virtualized cluster must meet [CPU Model Pre-requisites](https://docs.platform9.com/private-cloud-director/add-hosts-virtualized-cluster/cpu-model#cpu-model-pre-requisites) for VM evacuation to work. Read more about [CPU Models](https://docs.platform9.com/private-cloud-director/virtualized-clusters/add-hosts-virtualized-cluster/cpu-model).
* Source and destination hosts must run the same operating system version. For more information, refer to the [Operating System Compatibility Requirements](#operating-system-compatibility-requirements).

{% hint style="info" %}
**NOTE**

The evacuation operation will be triggered automatically for VMs on hosts that experience an outage in clusters with VM High Availability enabled. You can perform a VM evacuation operation manually via the command line.
{% endhint %}

#### Using CLI

1. Authenticate [pcdctl CLI](https://docs.platform9.com/private-cloud-director/reference/pcdctl-command-line)
2. Run the command below to perform the live migration.

{% hint style="info" %}
**NOTE**

Here target host UUID is mandatory.
{% endhint %}

{% tabs %}
{% tab title="Bash" %}

```bash
$ pcdctl server evacuate --host <FAILED_HOST UUID>
```

{% endtab %}
{% endtabs %}

### Operating System Compatibility Requirements

{% hint style="info" %}
**NOTE**

All VM migration types (cold, live, and evacuation) require source and destination hosts to run the same operating system version and KVM version.
{% endhint %}

* Hosts running Ubuntu 22.04 can only migrate VMs to other Ubuntu 22.04 hosts.
* Hosts running Ubuntu 24.04 can only migrate VMs to other Ubuntu 24.04 hosts.
* Cross-version migrations (Ubuntu 22.04 ↔ Ubuntu 24.04) are not supported.

**Known Limitation:** While initial migration from Ubuntu 22.04 to Ubuntu 24.04 may appear to succeed, subsequent migrations back to Ubuntu 22.04 may fail. This is due to incompatibilities between KVM versions on different operating systems.

#### Recommended Approach for OS Upgrades

When upgrading your cluster from Ubuntu 22.04 to Ubuntu 24.04 consider the following:

* Plan for one-way migration only (22.04 → 24.04).
* Complete all hosts upgrades before resuming normal migration operations.
* Do not attempt to migrate VMs back to older OS versions.

## Migration Limitations

### Cross-Cluster Live Migration

Cross-cluster live migration of virtual machines is not currently supported. This feature will be introduced with a future release of Private Cloud Director.

### Live Migration of GPU-Enabled VMs

You can live migrate virtual machines configured with [VGPUs](https://docs.platform9.com/private-cloud-director/gpu/gpu-support-pcd/setup-vgpu). VMs configured with GPU passthrough can not be live migrated today.