Dynamic Resource Rebalancing (DRR)

In this document you will learn about Private Cloud Director Dynamic Resource Rebalancing (DRR), an automated feature that optimizes resource utilization of your virtualized clusters. DRR ensures efficient resource utilization via live migration of virtual machines, preventing performance bottlenecks.

Introduction

In any dynamic virtualized environment, workloads fluctuate. Some virtual machines might become resource-intensive, while others remain idle. This can lead to imbalances across the physical hypervisor hosts within a cluster – some hosts become overloaded, causing performance degradation for their VMs, while other hosts sit underutilized. Private Cloud Director addresses this challenge with its integrated Dynamic Resource Rebalancing (DRR) feature. DRR works continuously to ensure that resources like CPU and memory are utilized efficiently and that no single host becomes a performance bottleneck.

Benefits of DRR

Enabling DRR in your Private Cloud Director environment delivers tangible benefits:

  • Consistent VM Performance: Helps prevent performance issues caused by resource contention on overloaded hosts.

  • Optimized Resource Utilization: Ensures that your hardware investment is used more efficiently across the cluster.

  • Proactive Problem Avoidance: Identifies and resolves potential resource bottlenecks before they negatively impact applications.

  • Reduced Operational Overhead: Automates the complex task of monitoring and balancing VM workloads, freeing up administrator time.

DRR Pre-requisites

Configure DRR for a Cluster

  • Scope - DRR is configured as a property at each virtualized cluster level. When you create a new virtualized cluster, you have the option to enable DRR for the cluster.

    • You can also enable DRR on an existing virtualized cluster at a later point.

  • Frequency - When you enable DRR, you are asked to choose a value for frequency at which DRR should run.

    • The default is 20 minutes. Which means every 20 minutes, DRR will evaluate hosts within the virtualized cluster and identify targets for potential rebalancing.

    • You can change this value to 10 minutes or 30 minutes.

    • Note - support for running DRR at 5 minute frequency is coming in the future.

DRR and VM Migration Priority

What Is Migration Priority

Migration priority is a key concept that is part of Private Cloud Director DRR. Migration priority can be assigned at per virtual machine level. It determines:

  • whether DRR should migrate a virtual machine and, if so,

  • the priority in which it should be selected for migration.

Migration Priority Values

Migration priority supports following values:

  • Unset - This is the default state for migration priority value for virtual machines.

  • Normal - Any virtual machines that do not have migration priority explicitly assigned, and that do not have soft affinity rules associated with them, will be treated with migration priority normal. The VMs are chosen for migration after the VMs with high priority value are migrated, but before the VMs with low priority value are migrated.

  • Low - VMs with a priority value of low will be selected last for migration. You may therefore assign this value to virtual machines that incur a heavier tax on migration and that you would only want to live migrate if all other options are exhausted. VMs with soft affinity rules that do not have migration priority explicitly assigned will also be treated with low priority.

  • High - VMs with a priority value of high will be selected first for migration. A good candidate for this category are dev/test VMs that may be smaller in size and / or their performance may not be impacted by live migration.

  • Excluded - You can also set a migration priority value of never to a VM. VMs with this value will be not be migrated by the DRR service. You can use this for virtual machines that you do not want DRR to ever migrate.

Note that the migration priority values are only relevant in the context of DRR. These values are not taken into account by other Private Cloud Director services such as Virtual Machine High Availability (VM HA) that may still evacuate the VM to a different host, if enabled at cluster level. An Administrator can also manually migrate a VM at any point independent of the migration priority value set for that VM.

Important

Migration priority values are only relevant in the context of DRR. They are not honored by other Private Cloud Director services such as VM HA. Administrators can manually migrate a VM independent of the migration priority.

Configure Migration Priority For VMs

You can change the migration priority value for any VM running on a virtualized cluster with DRR enabled.

  • Select the VM in the virtual machines grid view by navigating to the 'virtual machines' menu in the Private Cloud Director UI.

  • From the action bar, choose 'Other actions', then choose 'Migration Priority'.

  • You can now select the appropriate priority value for the VM.

How DRR Works

DRR functions as an ongoing optimization engine for your cluster:

  1. Continuous Monitoring: DRR continuously monitors key resource utilization metrics, specifically CPU and memory utilization, across all active hosts within the virtualized cluster.

  2. Imbalance Detection: The system analyzes CPU and Memory utilization to identify imbalances in resource utilization across hosts.

    1. DRR looks for hosts that are overloaded in the cluster.

    2. Host overload threshold: When DRR finds a host with CPU OR memory utilization of greater than 80%, it determines that the host is overloaded.

    3. DRR then ensures that there are underutilized hosts with sufficient spare capacity in the cluster.

    4. Assuming it finds the required spare capacity, DRR will identify a suitable target host for migration. DRR will sort all compatible hosts in the cluster based on their utilization value, and choose the host with lowest utilization.

    5. DRR then initiates VM migrations from overloaded hosts to other compatible hosts with spare capacity.

  3. Automated Live Migration: DRR first groups VMs from the source host based on the VM Migration Priority.

    1. VMs with no priority value assigned to them are treated to have priority value of default

    2. VMs with priority value of high are chosen for migration first

    3. VMs with priority value of default are chosen next

    4. VMs with priority value of low are chosen last.

    5. VMs with priority value of never are skipped from migration.

    6. DRR will then initiate VM live migrations, one VM at a time. This ensures that there is no downtime for the virtual machine while it is being migrated.

    7. After each VM live migration, DRR re-evaluates if the source host is still overloaded. If true, it will continue with this process.

DRR Interoperation with other services

This section describes how DRR interoperates with other services configured for your cluster.

Host Aggregates

DRR only finds candidate target hosts for a VM that satisfy the specific VM's host aggregate requirement. If it does not find a suitable host, DRR will not migrate the VM.

VM HA

DRR and Virtual Machine High Availability are designed to interoperate well together. It is possible that a host failure event might occur while DRR is actively rebalancing VMs, either from the same host or from other hosts in the cluster. When this happens:

  1. VM HA will detect the host failure and initiate VM evacuations

  2. The VM evacuations may result in cluster imbalance

  3. DRR will then detect the imbalance during either the current run or the next run.

  4. DRR will redistribute the load across the cluster to address the imbalance.

VMs with Hard Affinity or Anti-Affinity Rules

  1. DRR will skip a VM with hard affinity rule from migration.

    1. This behavior may change in the future once DRR has ability to initiate bulk VM migrations.

  2. For a VM with hard anti-affinity rule, DRR will find suitable candidate host that satisfies the anti-affinity rule. If it can not find a suitable host, DRR will not migrate the VM.

For a VM with soft-affinity, DRR will try its best to find a target host that satisfies affinity, if not it will find another target host to migrate the VM. Also if migration priority is not set for the VM, DRR will consider the priority as low for the VM

For a VM with soft-anti-affinity, DRR will try its best to find a target host that satisfies anti-affinity, if not it will find another target host to migrate the VM. Also if migration priority is not set for the VM, DRR will consider the priority as low for the VM

VM States

DRR only live migrates VMs in Active or powered on state. DRR will skip VMs in all other states.

VMs with Special Properties

  1. Limitation - DRR does have the ability to live migrate VMs with Virtual TPM enabled today. This ability is coming soon.

  2. DRR will live migrate VMs that have hot added CPU or memory resources.

Host States

DRR will only operate on hosts in online state. DRR will ignore hosts in offline or error state.

DRR will only operate on hosts with hypervisor role assigned. DRR will ignore all other hosts.

PreviousVirtualized ClusterNextVirtual Machine High Availability (VM HA)

Last updated

Was this helpful?