For the complete documentation index, see llms.txt. This page is also available as Markdown.

Stretched Clusters

In this document, you will learn about Private Cloud Director Stretched Clusters, a feature that lets you split a single virtualized cluster across two sites and give virtual machines a soft preference for one site during normal operation.

Overview

A stretched cluster is a single virtualized cluster whose hosts span two physical sites connected by a low-latency, high-throughput link. Within a stretched cluster, you define two failure domains, one per site, and optionally assign each VM a soft affinity to one of them. During normal operation, PCD keeps a VM on its preferred failure domain. If that domain's entire site fails, Virtual Machine High Availability (VM HA) recovers the VM on the peer domain using that domain's reserved capacity.

Stretched Clusters address whole-site failure. They do not provide disaster recovery or a Site Recovery Manager style solution: there is no orchestrated failover plan, no asynchronous replication, and no separate recovery site. A stretched cluster is one cluster with two halves; the affinity a VM has to a failure domain is a placement preference, not a hard requirement, and it can be temporarily overridden to keep the VM running.

For VMware Administrators

Failure-domain soft affinity is modeled on the VM-Host "should run on hosts in group" affinity rule that vSphere DRS uses in a vSphere Metro Storage Cluster (vMSC). Like a vMSC "should" rule, it expresses a preference that PCD honors during normal operation and sets aside when availability is at stake.

In this guide, you will learn the prerequisites for a stretched cluster, how to create one and configure its failure domains, how VM placement and failover behave, and how the feature interacts with host aggregates and existing VM-to-VM affinity rules.

Prerequisites

Before you create a stretched cluster, make sure the following are in place:

  • A low-latency, high-throughput link between the two sites. PCD does not validate link quality; you are responsible for provisioning a link suitable for synchronous storage replication and live migration.

  • Storage already stretched across both sites, with synchronous replication between the sites (RPO 0). PCD does not stretch storage for you; the customer provides and manages the stretched storage layer.

  • Network already stretched across both sites (Layer 2 stretched, with the same Layer 3 subnet reachable from hosts at either site). PCD does not stretch the network for you.

  • VM High Availability enabled on the cluster. VM HA performs the cross-site recovery when a failure domain goes down, and the cluster-selection step of the wizard blocks you from continuing until VM HA is enabled.

  • No host aggregates on the cluster. A cluster cannot have both host aggregates and failure domains. See Host Aggregates.

  • 50% failover reserve capacity in each half of the cluster (recommended, not required). Each failure domain should keep enough spare vCPU and memory to absorb its peer's entire workload during a full-site failure. PCD warns when this isn't met but does not block you; see Capacity Planning.

Create a Stretched Cluster

Stretched clusters have their own navigation entry, alongside the other cluster-related pages: Infrastructure > Stretched Clusters. The Stretched Clusters list page shows one card per stretched cluster, with the cluster name and its two failure domains displayed side by side. Each domain tile shows a reachability status (online or offline) and a count of hosts online in that domain.

Click + Create Stretched Cluster to start the two-step wizard.

Step 1: Select a Cluster

Choose an existing virtualized cluster from the Cluster dropdown. PCD validates the cluster against the prerequisites above and shows the result of each check:

  • VM High Availability is enabled. If not, the check fails with "Not enabled on this cluster. Enable VM HA on the cluster before creating failure domains." This check blocks you from continuing.

  • 50% failover capacity reserve allocated (recommended). If not met, the check shows a warning: "Not allocated, so a domain may not be able to absorb its peer if a whole site fails. PCD warns but does not block; managing failover capacity is your responsibility." This check does not block you.

  • No host aggregates exist in the cluster. If the cluster has any host aggregates, the check fails with "This cluster has host aggregates. Host aggregates are disallowed in clusters containing failure domains. Remove host aggregates to continue." This check blocks you from continuing.

Next stays disabled until both blocking checks pass. Once they do, click Next.

Step 2: Configure Failure Domains

Name the two failure domains and assign every host in the cluster to one of them:

  1. Enter a name for each domain in the Failure Domain A Name and Failure Domain B Name fields.

  2. In the Cluster Hosts table, use the Domain Assignment selector in each row to assign that host to domain A or B. You can also select multiple hosts with the row checkboxes and use the bulk actions Assign to A Site or Assign to B Site.

  3. Distribute hosts as evenly as possible between the two domains. The UI blocks assignments that skew the split too far toward one domain; hovering over a disabled assignment shows "Cannot assign more hosts to this failure domain. Hosts must be distributed as evenly as possible between the two failure domains."

  4. Click Create Stretched Cluster.

Every host in the cluster must end up assigned to exactly one domain; neither domain can be left with zero hosts.

Manage a Stretched Cluster

View Failure Domain Details

Each failure domain has its own detail page, reached from the Stretched Clusters list or from a domain tile. The page has three tabs:

  • Overview shows a Configuration Overview card (status, cluster, peer site, host count, and the number of VMs preferring this site), vCPU and memory allocation cards with a 50% recommended-reservation marker, and a table of the domain's hosts.

  • Hosts shows the cluster's hosts table scoped to this domain.

  • Virtual Machines shows the VM instances scoped to this domain.

If a domain's allocated capacity climbs too high, the page shows a capacity warning: "<Domain> has allocated NN% of its available vCPU or memory capacity. Ensure sufficient resources remain available to absorb workloads from the peer failure domain during a failover event." See Capacity Planning.

Edit Failure Domains

From a stretched cluster's card menu, choose Edit to rename the domains or change host assignments. The dialog adds an Allocation Balance widget showing each domain's vCPU and memory totals and a split bar for each resource, so you can see the effect of a reassignment before saving.

A few rules govern edits:

  • Reassigning a host that runs VMs with failure-domain affinity is blocked. You see "<host> runs N VMs with failure-domain affinity. Migrate them off or remove their affinity before reassigning." Migrate the affined VMs off the host, or remove their failure-domain affinity, before you can move the host to the other domain.

  • A capacity warning appears, but does not block the change, if the resulting split would leave a domain without enough reserve to absorb its peer: "After this change, a failure domain may not have enough reserve to absorb its peer during a full-site failure. We recommend ensuring enough failover capacity exists in each domain to cover the other."

  • Host-membership changes are blocked while the cluster is failed over. If a domain is down, or VMs are actively failing over, you cannot reassign hosts: "Host-membership changes are blocked while a failure domain is down or if VMs are in the process of failing over. You can make changes once both domains are healthy and VMs have failed back." Renaming the domains is still allowed in this state.

Delete a Stretched Cluster

Deleting a stretched cluster is all-or-nothing: you cannot delete a single failure domain, only both at once. From the card menu, choose Delete.

If any VM in the cluster still has a failure-domain affinity assigned, deletion is blocked: "N VMs still have a failure domain assigned. Deletion is blocked until every VM's failure-domain affinity is removed." Clear the affinity per VM from the Virtual Machines page, or click Remove affinity from all VMs in this cluster to clear it for every VM in the cluster at once.

Expand What happens after stretched cluster deletion? to review the effect before you confirm:

  • The cluster reverts to a normal cluster; both failure domains are removed together.

  • VMs lose domain-based recovery. VM HA reverts to normal cluster-wide recovery.

  • DRR returns to rebalancing across the whole cluster instead of within a domain.

  • Hosts are no longer bound to a failure domain.

  • Host aggregates become allowed on the cluster again.

Check the confirmation box and click Delete Stretched Cluster to proceed. This action is destructive and cannot be undone.

Host Membership

Every host in a cluster configured with failure domains should end up belonging to exactly one domain, including offline hosts and hosts in an error state, and a domain can never be reduced to zero hosts.

Assigning a host to a failure domain is a separate step from onboarding it, and it happens after the fact rather than as part of authorizing the host. The order of operations for a new host is:

  1. Onboard the host and apply the hypervisor role, as usual (see Host).

  2. From the Edit Stretched Cluster dialog, assign the host to one of the cluster's two failure domains.

Between these two steps, a host can be online in a stretched cluster with the hypervisor role applied but no failure domain assigned yet. PCD surfaces this gap rather than silently leaving it unaddressed:

  • The Edit Roles modal shows a warning when you're authorizing a host into a stretched cluster: "One or more Hypervisor Clusters are designated as Stretched Clusters. After the hypervisor role is applied, assign the respective hosts to the appropriate failure domain from the Edit Stretched Cluster page."

  • A host with no failure domain shows a warning on the hosts grid: "Failure domain assignment missing. Please assign this host to a failure domain within the stretched cluster after the Hypervisor role is applied."

  • The stretched cluster itself shows a warning while any of its hosts has no failure domain assigned.

Assigning the failure domain is not currently available as part of applying the hypervisor role; see Limitations.

VM Placement and Failure-Domain Affinity

A VM can have soft affinity to at most one failure domain, or none. Affinity is set on the Customize VM step of VM creation, in a Failure Domain section that appears only when the target cluster has failure domains configured. The Failure Domain dropdown defaults to None (place anywhere in the cluster), with the cluster's two domains as the other options.

A VM with no affinity can be placed on, and migrated to, any host in the cluster. Where a VM happens to land does not create an implicit affinity; only an explicit assignment does.

The VM list gains a Failure Domain Affinity column, showing either "no affinity" or the domain name. The VM's Configuration Overview on its detail page shows the same information.

Assign a Failure Domain to a VM

For a VM with no affinity, the Other Actions menu offers Assign Failure Domain. A VM can only be assigned to the failure domain of the host it is currently running on; the dialog names that domain and explains that to assign the VM to the other domain, you need to live-migrate it to a host in that domain first.

Change or Remove a VM's Failure Domain

For a VM that already has an affinity, Other Actions offers Change Failure Domain instead. Because a VM always belongs to its host's failure domain, changing a VM's domain is a three-step move: remove its current failure domain, live-migrate it to a host in the target domain, then assign it to that domain. The dialog's action button, Remove Failure Domain, starts that sequence.

Normal Operation and Rebalancing

During normal operation, a VM with affinity stays within its failure domain. Dynamic Resource Rebalancing (DRR) rebalances VMs only within a failure domain; it never migrates a VM across domains. See Dynamic Resource Rebalancing (DRR) for how DRR selects rebalancing targets within that scope.

Failure Handling and Recovery

Cross-domain movement happens only when an entire failure domain goes down, and it's carried out by VM HA, not DRR. When a domain fails:

  • VMs with affinity to the failed domain are evacuated to the surviving domain, drawing on that domain's reserved capacity.

  • VMs with no affinity that were running in the failed domain are evacuated to any available host in the cluster.

  • The soft-affinity preference is necessarily violated for the duration of the outage; recovering the VM takes priority over honoring its domain preference.

The target recovery time objective (RTO) for a stretched-cluster failover is 3 to 5 minutes. See Virtual Machine High Availability (VM HA) for the underlying evacuation mechanics.

There is no automatic failback. When the failed domain comes back online, affected VMs keep running in the peer domain until an administrator migrates them back manually. Once the preferred domain recovers, the VM needs to be moved back manually to restore its original failure-domain affinity.

Capacity Planning

Each failure domain should hold enough spare capacity to absorb its peer's entire workload if that peer's site fails. PCD enforces this as a soft ceiling: it warns when a domain's reservation falls short, both in the create wizard and on the domain detail page, but never blocks you from proceeding. Managing failover capacity across the two sites is your responsibility.

Interaction with Other Features

Host Aggregates

Host aggregates and failure domains are mutually exclusive on a cluster. You cannot create a stretched cluster on a cluster that has host aggregates (see Prerequisites), and you cannot create a host aggregate on a cluster that already has failure domains. Attempting the latter from Create Host Aggregate shows: "<Cluster> has failure domains. Host aggregates cannot be created on a cluster configured with failure domains. Choose a different cluster, or remove the failure domains first." and disables the create action. Deleting a stretched cluster's failure domains makes host aggregates available on that cluster again. Outside of failure-domain clusters, host aggregates are unaffected; see Host Aggregate.

VM-to-VM Server Group Policies

Failure-domain affinity is separate from, and fully compatible with, the VM-to-VM affinity and anti-affinity rules described in VM Affinity Anti-Affinity Rules. A VM can have both a server-group policy and a failure-domain affinity at the same time. During an HA event, VM HA evacuates the members of an anti-affinity server group one at a time, so a cross-domain recovery does not violate a hard anti-affinity policy. If the surviving failure domain does not have enough hosts to keep the group's members apart, the members it cannot place land in Error state instead, and you can retry their evacuation.

Existing VMs After an Upgrade

After you upgrade to a release with Stretched Clusters, VMs that existed before the upgrade have no failure-domain association, even on a cluster you configure with failure domains afterward. To bring an existing VM under domain-based placement, migrate it to a host in the failure domain you want, then assign that domain to the VM from Other Actions > Assign Failure Domain.

Where Failure-Domain Information Appears

Once a cluster has failure domains, several other pages surface that information:

  • The Clusters list gains a Failure Domains column, listing the two domain names (or a dash for a normal cluster).

  • The cluster detail page's Configuration Overview gains a Failure Domains entry, and its Cluster Hosts table gains a Failure Domain column.

  • The Cluster Hosts list gains a Failure Domain column.

  • A host's detail page gains a Failure Domain row under Properties.

Limitations

  • Failure domain assignment cannot be combined with host onboarding. You apply the hypervisor role to a host first, then assign it to a failure domain as a separate step from the Edit Stretched Cluster page. A future release is expected to let you assign the failure domain at the same time you apply the hypervisor role.

  • Only soft ("should") affinity is supported. Hard ("must") VM-to-failure-domain affinity is not available in this release.

  • A cluster supports at most two failure domains.

  • Failback after a domain recovers is always manual; there is no automatic failback.

  • Stretched Clusters is VM-to-failure-domain affinity within a single cluster, not a disaster recovery or Site Recovery Manager style solution. There is no failover orchestration and no asynchronous replication.

  • PCD does not stretch storage or networking for you; both must already span the two sites before you create a stretched cluster.

  • You cannot delete a single failure domain. Deletion removes both domains together and reverts the cluster to normal.

Last updated

Was this helpful?