Air Gapped Installation

Deploy Platform9 Private Cloud Director (PCD) in an environment without direct internet access.

Overview

Use this guide to deploy Platform9 Private Cloud Director (PCD) in an air-gapped environment, where cluster nodes have no direct internet access. Before you start the installation, download all required packages, container images, and installation artifacts on a host with internet access, then transfer them into your environment.

In an air gapped deployment, the following components must be set up inside your environment before PCD can be installed. You will configure each of these as part of this guide.

Component	Purpose
Private APT repository	Hosts required Ubuntu packages
Private container registry	Hosts required container images
Airctl installation artifacts	Installer components downloaded externally
NTP server	Ensures time synchronization across nodes

Prerequisites

Before you begin, ensure the following conditions are met.

Network requirements

All cluster nodes must have network connectivity to the following internal services:

• Private APT repository

• Private container registry

• NTP server

For management plane host configuration, follow the prerequisites guide, except for package updates and OpenSSL installation. Those steps are covered in this guide.

DNS requirements

Configure DNS resolution for the container registry hostname.

Example:

registry.pf9.io → 10.10.13.145

If DNS is not available, add the registry hostname to /etc/hosts on every node instead.

Example:

10.10.13.145 registry.pf9.io

Step 1: Configure NTP synchronization

All nodes must synchronize time with an NTP server inside your network to prevent clock skew across the cluster. In this step, you configure each node to point to your internal NTP server, restart the time synchronization service, and verify that synchronization is active before proceeding. If NTP is already configured on your nodes, skip this step.

1. Configure the NTP client on each node to point at your internal NTP server.

    sudo mkdir -p /etc/systemd/timesyncd.conf.d
    echo "[Time]
    NTP=<ntp-server-ip-or-fqdn>" | sudo tee /etc/systemd/timesyncd.conf.d/custom.conf

1. Restart the time synchronization service and enable it to start on boot.

bash

    sudo systemctl restart systemd-timesyncd
    sudo systemctl enable systemd-timesyncd

1. Verify that the node is actively syncing with the NTP server before moving on.

bash

    timedatectl status
    timedatectl show-timesync --all

Step 2: Set up a private APT repository

The private APT repository hosts the Ubuntu packages required by the platform. You create this repository on a server with internet access and then make it accessible to the air-gapped cluster. To complete this step, you will download the required scripts and dependency list, download the package dependencies, and then initialize the repository.

1. On a host with internet connectivity, download the repository scripts and the dependency list.

    # Script to create a repo
    curl --user-agent "<TOKEN>" -O https://pf9-airctl.s3.us-west-1.amazonaws.com/latest/sample_scripts/create_apt_repo.sh

    # Script to download package dependencies
    curl --user-agent "<TOKEN>" -O https://pf9-airctl.s3.us-west-1.amazonaws.com/latest/sample_scripts/download_all_deps.sh

    # Dependency list
    curl --user-agent "<TOKEN>" -O https://pf9-airctl.s3.us-west-1.amazonaws.com/latest/dependency_list.txt

1. Make both scripts executable before running them.

    chmod +x create_apt_repo.sh download_all_deps.sh

1. Download all package dependencies required for the installation.

    ./download_all_deps.sh <path-to-dependency-list.txt>

NOTE

If you are using HTTPS with a custom CA, install the CA into /usr/local/share/ca-certificates and run update-ca-certificates before proceeding.

4. Initialize the APT repository. Two options are available: HTTPS and HTTP. HTTPS is recommended because it encrypts package transfers and prevents tampering in transit. Use HTTP only if your environment does not support TLS.

NOTE

Creating the repository typically takes approximately 20 minutes, depending on hardware performance.

Option 1: HTTPS (recommended)

If you do not have an existing certificate, generate a self-signed certificate and initialize the repository.

    sudo ./scripts/create_apt_repo.sh init-https repo.local
    sudo ./create_apt_repo.sh add-bulk ./deb_packages

If you already have a certificate and key, provide the paths directly.

    sudo ./scripts/create_apt_repo.sh init-https repo.example.com /path/to/cert.pem /path/to/key.pem
    sudo ./create_apt_repo.sh add-bulk ./deb_packages

Option 2: HTTP (insecure)

Use this option only if TLS is not available in your environment. Package transfers over HTTP are unencrypted and not verified.

    sudo ./create_apt_repo.sh init
    sudo ./create_apt_repo.sh add-bulk ./deb_packages

Step 3: Configure the APT repository on cluster nodes

Update the APT configuration on all management and compute nodes so they can access the private repository.

1. If you are using a self-signed certificate, distribute the CA to each node and update the certificate store.

    sudo cp /path/to/apt_repo.crt /usr/local/share/ca-certificates/
    sudo update-ca-certificates

1. Back up your existing APT sources.

    sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak
    sudo mkdir -p /etc/apt/sources.list.d.bak
    sudo mv /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d.bak/ 2>/dev/null || true
    sudo rm /etc/apt/sources.list

1. Add the private repository and update the package index.

    echo "deb [trusted=yes] http://<repo-host>/ stable main" | sudo tee /etc/apt/sources.list.d/private-repo.list
    sudo apt update

Step 4: Set up a private container registry

The private registry stores all container images required by PCD. You set up the registry on a host with internet access and then make it reachable from your air-gapped cluster nodes. To complete this step, you will download the registry setup scripts, initialize the registry, and confirm it is ready to receive images.

1. On a host with internet connectivity, download the registry setup scripts and the image list.

    curl --user-agent "<TOKEN>" -O https://pf9-airctl.s3-accelerate.amazonaws.com/latest/pcdv-images.txt
    curl --user-agent "<TOKEN>" -O https://pf9-airctl.s3.us-west-1.amazonaws.com/latest/sample_scripts/setup_registry.sh
    curl --user-agent "<TOKEN>" -O https://pf9-airctl.s3.us-west-1.amazonaws.com/latest/push-images.sh

1. Make the setup script executable and run it to initialize the registry.

    chmod +x setup_registry.sh
    ./setup_registry.sh

1. When prompted, provide the following inputs to configure the registry.

   Prompt	Example	Description
   Registry host IP	10.10.13.145	IP of the registry server
   Registry domain	registry.pf9.io	DNS name used by the cluster
   Username	<registry-user>	Registry login username
   Password	<registry-password>	Registry login password

Example session:

    Enter the registry host IP address: 10.10.13.145
    Enter the registry domain: registry.pf9.io
    Enter the username: admin
    Enter the password: <registryPassword>

Step 5: Push images to the private registry

In this step, you push all required PCD container images to the private registry you created in Step 4. Before Docker can communicate with a registry that uses a self-signed certificate, you must add the registry's CA certificate to Docker's trust store. Without this, Docker rejects the connection and the image push fails.

NOTE

Docker must be installed on the host machine before you can push images.

1. Add the registry's CA certificate to Docker's trust store.

    sudo mkdir -p /etc/docker/certs.d/<registry_url>:443
    sudo cp /usr/local/share/ca-certificates/ca.crt /etc/docker/certs.d/<registry_url>:443/ca-cert.pem

1. Restart the Docker service so the certificate change takes effect.

    sudo systemctl restart docker

1. Push the images to the private registry using the push-images.sh script.

    ./push-images.sh \
      --images-file-path <image_list_path> \
      --private-registry-url <registry_url> \
      --private-registry-username=<registry_username> \
      --private-registry-password=<registry_password>

Step 6: Install the OpenSSL dependency

PCD requires a specific build of OpenSSL that is not available through the standard Ubuntu package repositories. In this step, you download the pre-built package from the Platform9 artifact store, transfer it to each cluster node, and install it. The checksum verification confirms that the package was not corrupted in transit.

1. On a host with internet access, download the OpenSSL package.

    curl --user-agent "<YOUR_USER_AGENT_KEY>" \
      https://pf9-airctl.s3-accelerate.amazonaws.com/openssl-smcp-ubuntu/openssl_3.0.7-1_amd64.deb \
      --output openssl_3.0.7-1_amd64.deb

1. Transfer the downloaded package to each cluster node.

2. On each node, verify the MD5 checksum to confirm the package integrity.

    md5sum openssl_3.0.7-1_amd64.deb | grep 706caf \
      || { echo "MD5 checksum does not match, exiting."; exit 1; }

1. Install the OpenSSL package and configure the library path.

    # Install the OpenSSL package
    sudo dpkg -i openssl_3.0.7-1_amd64.deb \
      || { echo "Failed to install OpenSSL, exiting."; exit 1; }

    # Add the OpenSSL library path
    echo "/usr/local/ssl/lib64" | sudo tee /etc/ld.so.conf.d/openssl-3.0.7.conf

    # Refresh the dynamic linker cache
    sudo ldconfig -v

    # Create a symbolic link to the new OpenSSL binary
    sudo ln -sf /usr/local/ssl/bin/openssl /usr/bin/openssl

1. Verify that the correct version of OpenSSL is active on the node.

    openssl version | grep 3.0.7 \
      || { echo "OpenSSL version does not match, exiting."; exit 1; }

Confirm that the output matches the following before continuing.

    OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)

Step 7: Install required packages

cgroup-tools is a Linux utility that PCD requires to manage and interact with control groups on each cluster node. Run the following command on each node to update the package index and install cgroup-tools in a single operation.

apt-get update -y && apt-get install cgroup-tools -y

Step 8: Deploy PCD

With the private registry and APT repository in place, you can now run the PCD installer. Complete the following steps on a management node.

NOTE

If the registry hostname is not resolvable via DNS in your environment, add the hostname to /etc/hosts on each node before proceeding.

1. On a system with internet connectivity, download the installer artifacts.

    curl --user-agent "<YOUR_USER_AGENT_KEY>" \
      https://pf9-airctl.s3-accelerate.amazonaws.com/latest/index.txt | \
      awk '{print "curl -sS --user-agent \"<YOUR_USER_AGENT_KEY>\" \"https://pf9-airctl.s3-accelerate.amazonaws.com/latest/" $NF "\" -o ${HOME}/" $NF}' | bash

1. Copy all downloaded artifacts to the management node, then make the installer executable.

    chmod +x ./install-pcd.sh

1. Run the installer using the version number from version.txt.

    ./install-pcd.sh $(cat version.txt)

1. Create a symlink so that airctl is available globally on the management node.

    sudo ln -s /opt/pf9/airctl/airctl /usr/bin/airctl

1. Generate the configuration file for your management cluster. Use single-master for a POC environment or multi-master for production.

NOTE

Before running this command, copy the ca.crt file from the registry to the management node. Provide its path using the --custom-registry-ca-cert-path flag.

    airctl configure \
      -4 \
      -f <fqdn> \
      -e <du-public-ip> \
      -i <comma-separated-master-node-IPs> \
      --master-vip4 <vip-for-nodelet-cluster> \
      -v ens3 \
      -r <du-region-name> \
      -k <nfs-host-ip-for-hostpath-provisioner> \
      -n /mnt/gnocchi \
      -p hostpath-provisioner \
      --custom-registry-url https://registry.pf9.io \
      --custom-registry-username <registry_username> \
      --custom-registry-password "<registry_password>" \
      --custom-registry-ca-cert-path <path-to-registry-ca.crt> \
      --custom-registry-path-overrides \
      --enable-pcd-chart-bundle \
      --verbose

Next steps

After completing the air-gapped setup, proceed to Install to deploy the management cluster and install the PCD management plane using airctl.