Diagnose Role Assignment Failures

Overview

After a host is authorized in Private Cloud Director, you assign roles (Hypervisor, Networking Service, Image Library Service, Persistent Storage Service) from the UI or API. Role assignment triggers the management plane to push the required configuration to the host agent, which then downloads packages and starts services. When this process fails, the host gets stuck in a converging or error state rather than reaching applied.

The two most common failure patterns are:

1. HTTP 500 error when assigning or removing a role — the management plane API returns an internal server error and the role change is not queued.

2. "Interface None is missing an IP" error — the Networking Service role fails to apply because the host's network interface configuration is incomplete.

In this guide, you will identify which failure pattern applies, verify the host's prerequisites, and resolve the error.

Prerequisites Before Assigning Roles

Before assigning any role, confirm the following on the host:

• The host is in online connection status and unauthorized or applied role status in the UI (Infrastructure > Cluster Hosts).

• The host agent is running: sudo systemctl status pf9-hostagent

• The host can reach the management plane: curl -sk https://<management-plane-fqdn>/resmgr/v1/hosts | head -c 200

• NTP is synchronized: sudo timedatectl status

• Required network interfaces are present and have IP addresses (see below)

HTTP 500 Error on Role Assignment or Removal

Symptom

When you click Edit Roles in the UI and save, the UI displays an error or the role assignment never starts. In some cases, the API call to assign or remove a role returns an HTTP 500 response.

What to Check

1. Host state in the management plane

A 500 error on role assignment often means the management plane has stale or conflicting state for the host — for example, a prior role removal that did not complete cleanly, leaving the host's record in an inconsistent state.

Check the hostagent log on the host for any errors that occurred just before the 500 response:

tail -100 /var/log/pf9/hostagent.log

Look for authentication failures, connection errors, or messages about the host not being found in the management plane.

2. Host registration status

Confirm the host's UUID in the local file matches what the management plane shows:

sudo cat /etc/pf9/host_id.conf

The UUID in this file must match the host UUID shown in the UI under Infrastructure > Cluster Hosts. A mismatch means the host was re-imaged or the identity file was changed without a clean decommission, and the management plane is seeing two conflicting records for the same host.

If there is a mismatch, decommission the stale host record from the management plane and re-onboard the host. See Re-Onboarding Recovery.

3. Stale compute service entries

If the host was previously decommissioned forcefully (using pcdctl decommission-node -r), stale compute service records may remain in the management plane database and block role re-assignment. Remove them:

pcdctl compute service list
pcdctl compute service delete <uuid>

After removing stale entries, restart the host agent:

sudo systemctl restart pf9-hostagent

Then retry the role assignment from the UI.

4. Self-Hosted deployments: check the resmgr pod

Self-Hosted deployments only. If host-level steps do not resolve the 500 error, check the resource manager service logs on the management plane for the specific error:

kubectl logs -n <region-fqdn> -l app=resmgr --tail=200 | grep -i "500\|error\|<host-uuid>"

In SaaS deployments, contact Platform9 Support if the host-level steps do not resolve the error.

Recovery Steps

1. Deauthorize any partially assigned roles from the UI (select the host, click Edit Roles, remove all roles, save).

2. Wait for the host to return to unauthorized status.

3. Verify the hostagent is running and the host is online.

4. Re-assign the roles from the UI.

If the 500 error persists after these steps, contact Platform9 Support with the host UUID, the hostagent log, and the time the error occurred.

"Interface None Is Missing an IP" Error

Symptom

Role assignment starts but the Networking Service role fails to apply. The hostagent log or the UI role status shows an error message containing Interface None is missing an IP or interface <name> has no IP address.

Cause

The Networking Service role requires at least one network interface on the host to have an IP address configured in the host's blueprint. This error means either:

• The interface selected in the host's network configuration (blueprint) has no IP address assigned to it on the host.

• The host blueprint references an interface name that does not exist on the host (for example, the interface was renamed after the blueprint was created).

What to Check

1. Verify the host's actual network interfaces and their IP addresses

ip addr show

Confirm that the interface you intend to use for the management network (or the interface referenced in the cluster blueprint) has an IP address. If an interface shows state DOWN or has no inet entry, it has no IP.

2. Verify the interface name matches what the blueprint expects

Network interface names can change across OS upgrades or hardware changes (for example, eth0 becoming ens3). Check the host configuration stored by the agent:

sudo cat /opt/pf9/data/state/network_config.json 2>/dev/null || echo "file not found"

Compare the interface names in this file against the output of ip addr show.

3. Check the cluster blueprint

In the UI, navigate to Infrastructure > Cluster Blueprint and review the network interface configuration for the cluster the host belongs to. Confirm the interface name and IP assignment match what is present on the host.

Recovery Steps

1. If the interface has no IP, assign an IP address to it:

# Verify with
ip addr show <interface-name>

For a permanent IP assignment, update the Netplan configuration in /etc/netplan/ and apply it:

sudo netplan apply

1. If the interface name in the blueprint does not match the actual interface on the host, update the host's network configuration to use the correct interface name. From the UI, navigate to Infrastructure > Cluster Hosts, select the host, and update the interface assignment.

2. After the interface has an IP and the blueprint references the correct interface name, retry the role assignment from the UI.

3. Monitor the hostagent log to confirm the Networking Service role applies successfully:

tail -f /var/log/pf9/hostagent.log

Look for role pf9-neutron applied or converge successful.

General Role Assignment Checklist

Before retrying a role assignment after any failure, verify:

• Host shows online in Infrastructure > Cluster Hosts

• sudo systemctl status pf9-hostagent is active (running)

• sudo cat /etc/pf9/host_id.conf UUID matches the UI

• All network interfaces referenced by the cluster blueprint have IP addresses (ip addr show)

• No stale compute service entries for this host (pcdctl compute service list)

• Sufficient disk space on /var, /opt, and /tmp (df -h)

If all items pass and role assignment still fails, contact Platform9 Support with the hostagent log and the exact error message.