> For the complete documentation index, see [llms.txt](https://docs.platform9.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.platform9.com/private-cloud-director/gpu/gpu-support-pcd/setup-vgpu.md).
# Set up vGPU
Maximize GPU efficiency with Virtual GPU (vGPU) technology for virtual machines. Configure your vGPU infrastructure seamlessly with our step-by-step guide, ensuring prerequisites.
Virtual GPU (vGPU) allows multiple VMs to share physical GPU resources efficiently. This approach maximizes resource utilization and reduces costs while still providing GPU acceleration for lighter workloads.
## Supported OS and Kernel Versions
vGPU configuration is currently supported on:
| **OS Version** | **Kernel Version** | **Minimum NVIDIA Host Driver** |
| ---------------- | ------------------ | ------------------------------ |
| Ubuntu 22.04 | 5.15, 6.5 | Any supported GRID branch |
| Ubuntu 24.04 | 6.8 | GRID 595.x branch or later |
| Rocky Linux 10.2 | 6.4+ | GRID 595.x branch or later |
## Configure vGPU infrastructure
Set up your vGPU infrastructure following the specific sequence required for virtual GPU functionality.
**Prerequisites**
Before beginning vGPU configuration, ensure your hosts has the required GPU drivers and licensing:
* vGPU functionality requires proper NVIDIA drivers and valid licenses to be installed on the hosts before any configuration steps.
* Ensure that the GPU cards intended for vGPU configuration are unbound and not linked to any other process or device. For more details see [Troubleshooting GPU Support](/private-cloud-director/gpu/gpu-support-pcd/troubleshooting-gpu-support.md)
* Required components:
* **NVIDIA GPU drivers** installed and functioning on the host
* **NVIDIA vGPU licenses** properly configured
* **NVIDIA license server** created and accessible
* **Valid license allocation** for your vGPU usage
* **Driver version:** Ubuntu 22.04 hosts support any GRID branch. Ubuntu 24.04 and Rocky Linux 10.2 hosts require **GRID 595.x branch or later**. See the [Supported OS and Kernel Versions](#supported-os-and-kernel-versions) table above.
* Ensure that VT-d and Virtualization technology (VT) are enabled in the BIOS before proceeding with vGPU configuration. If your GPU supports SR-IOV, also enable SR-IOV in the BIOS. Non-SR-IOV GPU models (for example, Tesla T4, Quadro RTX 6000/8000) do not require SR-IOV BIOS settings.
* Verify GPU drivers are installed: `nvidia-smi`
* On **Ubuntu 24.04** and Rocky 10.2 hosts, vGPU uses the SR-IOV **type-VF** model for SR-IOV hosts. The GPU model must support SR-IOV, and the initial vGPU pre-configuration step must be run **before** the NVIDIA GRID driver is installed. See Ubuntu 24.04 and Rocky 10.2 hosts: configuration sequence.
* Confirm the host OS and kernel version are supported for vGPU. See GPU VM Machine Type and OS Compatibility.
#### Step 1: Onboard vGPU host
Before configuring vGPU, you must first onboard your GPU host using `pcdctl`.
1. Onboard your vGPU host using `pcdctl` .
2. Verify the hosts onboarding completed successfully.
3. Ensure you have administrator access to the onboarded hosts.
#### Step 2: Run initial vGPU configuration
Execute the GPU configuration script to set up vGPU functionality on your host.
{% hint style="warning" %}
**Ubuntu 24.04 and Rocky Linux 10.2 hosts:** Run this pre-configuration step (option 2) **before** installing the NVIDIA GRID driver. Installing the driver first prevents the script from setting the kernel parameters required for SR-IOV type-VF mode, and vGPU configuration will fail.
{% endhint %}
{% hint style="info" %}
**NOTE**
The GPU configuration script is located at `/opt/pf9/gpu/pf9-gpu-configure.sh` on your onboarded host.
{% endhint %}
1. Access your onboarded vGPU host with administrator privileges.
2. Navigate to the GPU script directory by using the following command.
{% tabs %}
{% tab title="Bash" %}
```bash
cd /opt/pf9/gpu
```
{% endtab %}
{% endtabs %}
3. Run the GPU configuration script and enter **option 2** (`vgpu pre configure`).
{% tabs %}
{% tab title="Bash" %}
```bash
sudo ./pf9-gpu-configure.sh
```
{% endtab %}
{% endtabs %}
This configuration will prompt you to reboot at the end.
4. The script will prompt you to update grub and reboot. If you select **N** in the prompt, manually run the following commands:
{% tabs %}
{% tab title="Bash" %}
```bash
sudo update-grub
sudo reboot
```
{% endtab %}
{% endtabs %}
You may require to wait for the host to come back online before proceeding.
5. To verify if your vGPU pre configuration is successful, run the following command.
{% tabs %}
{% tab title="Bash" %}
```bash
sudo ./pf9-gpu-configure.sh
```
{% endtab %}
{% endtabs %}
6. Enter **option 5** (`validate vgpu`) on the terminal.
#### Step 3: Configure SR-IOV for vGPU (SR-IOV Hosts Only)
{% hint style="warning" %}
**Non-SR-IOV GPU hosts: skip this step.**
If your GPU does not support SR-IOV (for example, Tesla T4, Quadro RTX 6000/8000), skip Step 3 entirely and proceed directly to [Step 4: Create Host Configuration and vGPU Cluster](#step-4-create-host-configuration-and-vgpu-cluster). Non-SR-IOV hosts use mediated devices (mdev) directly from the physical function and do not require the SR-IOV configuration step.
{% endhint %}
Configure SR-IOV settings required for vGPU functionality on SR-IOV-capable GPU hosts.
1. Navigate to the GPU script directory using the following command.
{% tabs %}
{% tab title="Bash" %}
```bash
cd /opt/pf9/gpu
```
{% endtab %}
{% endtabs %}
2. Run the GPU configuration script and enter **option 3** (`vGPU SR-IOV configure`).
{% tabs %}
{% tab title="Bash" %}
```bash
sudo ./pf9-gpu-configure.sh
```
{% endtab %}
{% endtabs %}
4. The script will display output similar to the following.
{% tabs %}
{% tab title="Bash" %}
```bash
Step 3: Enable SR-IOV for NVIDIA GPUs
Detecting PCI devices from /sys/bus/pci/devices...
Found the following NVIDIA PCI devices:
Found the following NVIDIA devices: 0000:c1:00.0
Enter the full PCI device IDs (e.g., 0000:17:00.0 0000:18:00.0) to enable sriov, separated by spaces.
Press Enter without input to configure ALL listed NVIDIA GPUs:
No PCI device IDs provided. Configuring all NVIDIA GPUs...
Enabling SR-IOV for 0000:c1:00.0...
Enabling VFs on 0000:c1:00.0
```
{% endtab %}
{% endtabs %}
5. You can either:
* Enter specific PCI device IDs separated by spaces.
* Press Enter without input to configure ALL listed NVIDIA GPUs.
If you encounter an `Cannot obtain unbindLock`error during this step, refer to the [Troubleshooting Gpu Support](/private-cloud-director/gpu/gpu-support-pcd/troubleshooting-gpu-support.md) for resolution steps.
6. Verify vGPU and SR-IOV configurations, by performing the following steps.
* Navigate to the GPU script directory:
{% tabs %}
{% tab title="Bash" %}
```bash
cd /opt/pf9/gpu
```
{% endtab %}
{% endtabs %}
* Run the GPU configuration script to verify vGPU setup.
{% tabs %}
{% tab title="Bash" %}
```bash
sudo ./pf9-gpu-configure.sh
```
{% endtab %}
{% endtabs %}
* Enter **option 5** (`Validate vGPU`) on your terminal and review the verification output to confirm vGPU set up is configured.
7. On the console, view **Infrastructure** > **GPU Hosts** to verify if your GPU host appears on the list. You will see:
* Compatibility mode (vGPU)
* GPU model and device ID
#### Step 4: Create host configuration and vGPU cluster
Create the necessary host configuration and cluster settings for vGPU operation.
1. Navigate to **Infrastructure > Cluster Blueprint > Host Configurations** on the space.vars.product\_acronym console.
2. Select Add Host Configuration to create a new configuration.
3. Configure your new space.vars.product\_acronym host by entering the network and management settings. Each setting controls a specific functionality that determines how the host operates within your space.vars.product\_acronym environment.
| **Field Name** | **Description** |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Name this configuration | Specify the host name to configure. |
| Network Interface | Enter the Physical Interface Name. |
| Physical Network Label | This is an optional entry. Use a descriptive label to identify and organize physical network interfaces. By assigning meaningful names like "Production-Network" or "Management-VLAN" you can filter your search for easier identification and troubleshooting. |
| Management | Enable management functions. |
| VM Console | Enable VM Console access to allow administrators to connect directly to virtual machines running on this host for troubleshooting and management. |
| Image Library I/O | Enable Image Library I/O to allow this host to read from and write to the centralized image repository for VM deployment and updates. |
| Virtual Network Tunnels | Enable Virtual Network Tunnels to allow secure network connectivity between this host and other hosts in the space.vars.product\_acronym environment. |
| Host Liveness Checks | Enable Host Liveness Checks to automatically monitor a specific host health status and trigger alerts when the host is unresponsive. |
4. Name this configuration.
5. Configure the basic host settings with network section configured in the blueprint.
#### Step 5: Create vGPU cluster
1. Navigate to **Infrastructure > Clusters** on the space.vars.product\_acronym console.
2. Select **Add Cluster** to configure the cluster configuration.
3. Select **Enable GPU** and then select the GPU mode: **vGPU** for sharing GPUs across multiple VMs.
4. Select **Save**.
Your host configuration and cluster now support vGPU workloads.
#### Step 6: Authorize vGPU hosts
Authorize your vGPU configured host in your cluster.
1. Navigate to **Infrastructure > Cluster Hosts** in the space.vars.product\_acronym console.
2. Authorize the hosts by assigning:
* Host configuration
* Hypervisor role
* vGPU cluster
You may be required to wait for few minutes for the authorization process to complete.
#### Step 7: Configure GPU host with vGPU profile
Configure each physical GPU on your vGPU host with a vGPU profile.
1. Navigate to **Infrastructure > GPU Hosts** in the PCD console.
2. Select your vGPU host from the list and click **Configure**.
3. In the **GPU Card** dropdown, select a card. Cards are listed as `Card () ()` — for example `Card 1 (NVIDIA A10) (0000:17:00.0)`.
4. Select a vGPU profile for that card. Each profile shows its framebuffer memory and the maximum number of instances the card can host.
5. Repeat steps 3 and 4 for each card you want to serve vGPUs from. The **Selected Profiles** list shows each card and the profile assigned to it.
6. Click **Save Changes**.
7. Navigate to **Infrastructure > Cluster Hosts** in the console, to view vGPU host status.
You may be required to wait for a few minutes for vGPU profile to be configured on the vGPU host.
{% hint style="info" %}
**NOTE**
Only single profile selection is allowed per physical GPU on the host.
{% endhint %}
Your vGPU infrastructure is now ready for creating flavors and deploying VMs. For more details see [Create GPU Enabled Flavors](/private-cloud-director/gpu/gpu-support-pcd/create-gpu-enabled-flavors.md)
### vGPU migration behavior
Understanding live migration behavior for vGPU VMs:
* vGPU VMs can be migrated if the destination host supports the same vGPU profile.
* The system validates compatibility before allowing migration.
* Migration fails if the destination host does not have the required vGPU profile available.
### vGPU best practices
* **Profile selection**: Choose vGPU profiles that match your workload requirements.
* **Resource monitoring**: Monitor vGPU utilization to optimize resource allocation.
* **Driver compatibility**: Ensure vGPU drivers are compatible with your guest operating system
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://docs.platform9.com/private-cloud-director/gpu/gpu-support-pcd/setup-vgpu.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `build a script that syncs our docs to a CMS` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.