NVIDIA Run:ai
Cordatus installs NVIDIA Run:ai on your own GPU servers. You do not prepare Kubernetes by hand: Cordatus checks the device, sets up Kubernetes (RKE2), installs Run:ai, opens the Run:ai panel on a public address and lets other devices join the cluster as worker nodes. Once the cluster is up, LLM Launcher → Run:ai Clusters lists it with its nodes, GPU usage and inference endpoints.
Before you start
| Requirement | Detail |
|---|---|
| GPU | An NVIDIA data center GPU (for example H100, H200, B200, B300, A100, L40S, L4). |
| CPU architecture | x86_64 or aarch64. |
| Disk | At least 100 GiB free on the data location; 160 GiB or more is recommended. |
| Device state | The device is Online and you have permission to manage it. |
| Sudo password | Installing, uninstalling and cancelling need root on the device. If no sudo password is saved in Cordatus, you are asked for it; it is used only for that action. |
| Internet access | The device must reach the Run:ai download hosts on port 443. |
- Jetson, GeForce cards and vGPU cannot run Run:ai. These devices can still share their GPU between workloads through Cordatus.
- DGX Spark and workstation cards with 22 GB or more memory can be installed only after you accept a warning: NVIDIA does not support Run:ai on them. On DGX Spark, GPU fractions do not work and every workload gets the whole GPU.
Open the Run:ai window
- Go to Devices.
- Open the ... menu at the end of the device row.
- Click NVIDIA Run:ai.
The window opens with "Checking whether this device can run Run:ai...", and then shows a verdict at the top:
| Verdict | Meaning |
|---|---|
| Run:ai can be installed on this device. | All checks passed. Cordatus sets up Kubernetes (RKE2) first, then installs Run:ai. If Kubernetes is already running, it is reused. |
| This server can run Run:ai once the issues below are fixed. | The hardware is supported, but some checks failed. Each failed check says what to fix. |
| Run:ai does not support this hardware. | Jetson, GeForce and similar devices. |
| This device cannot run Run:ai. | No supported NVIDIA GPU was found. |
On the right, the GPUs, GPU fractions and Checks panels show the details. Failed checks are listed first, then warnings. After you fix something on the device, click Check again.
What is checked
| Check | Fails when | Warns when |
|---|---|---|
| CPU architecture | Not x86_64 or aarch64 | |
| GPU model | Jetson, GeForce, TITAN; workstation cards and DGX Spark (can be accepted) | The card is not on the known data center list |
| GPU virtualization | vGPU (only passthrough is supported) | |
| Operating system | Not Ubuntu 22.04 / 24.04 / 26.04, RHEL / Rocky / Alma 8.8+ / 9 / 10 or RHCOS (DGX OS counts as Ubuntu) | |
| Control groups (cgroup) | cgroup v1 (can be accepted; workloads then get whole GPUs only) | |
| CPU cores / Memory | Fewer than 20 cores / less than 32 GiB | |
| Free disk | Less than 100 GiB free (can be accepted) | Less than 160 GiB free |
| Required ports | One of 80, 443, 2379, 2380, 2381, 6443, 9345, 10250 is in use | |
| Docker networks | A network overlaps 10.42.0.0/16 or 10.43.0.0/16 | |
| firewalld | firewalld is active (RKE2 needs it off) | |
| NVIDIA Container Toolkit | Not installed | |
| Internet access | The download hosts cannot be reached | |
| Kubernetes | k3s, or a Kubernetes version outside 1.35–1.37 |
GPU fractions is information only. Run:ai works without it; to give a workload a share of a GPU the device needs NVIDIA driver 615 or newer, CUDA 13.4 or newer, cgroup v2, the open NVIDIA kernel module, and MIG turned off.
Install Run:ai
Fill in the Installation card:
| Field | Description |
|---|---|
| Panel address | The name of the Run:ai panel's public address, for example my-runai. You reach the panel from anywhere through Cordatus. 4–63 lowercase letters, digits or hyphens; the name is checked while you type. |
| Cluster name | The name of the Run:ai cluster. Lowercase letters, digits and hyphens. |
| Run:ai administrator e-mail | The account you sign in to the Run:ai panel with. |
| Run:ai administrator password | At least 12 characters with upper and lower case letters, a digit and one of ! @ # $ + -. Use Generate for a strong one. |
| Data location | Where Kubernetes images, models and volumes are kept. Internal disks only (no network or USB disks). It cannot be changed after installation. |
| Model folder | Where model weights (the Hugging Face cache) are kept. Defaults to the folder LLM Launcher uses. Uninstalling never deletes it. |
| Worker nodes (optional) | Other devices that join this cluster when the installation finishes. Each one is checked and shows Can join, Can join with your acknowledgement or Cannot join, and gets its own data location and model folder. |
Cordatus does not keep the Run:ai administrator password. Save it before you start; you sign in to the Run:ai panel with it.
If a check failed but can be accepted (disk space, cgroup v1, DGX Spark or workstation GPU), tick the Install anyway box under it. Then click Install Run:ai.
Installation progress
The installation takes 30 to 60 minutes. You can close the window; the installation continues on the device. The steps are shown one by one with a live log:
- Preparing the device
- Installing Kubernetes (RKE2)
- Waiting for Kubernetes to settle
- Setting up storage
- Creating certificates
- Downloading the Run:ai installer
- Run:ai preflight checks
- Installing Run:ai
- Creating the default project
Worker nodes join after that (Preparing the device → Installing Kubernetes (RKE2) → Joining the cluster).
Notify me when the installation ends sends one message when the installation finishes, stops with an error or is cancelled. You can choose your e-mail address or any channel you added under Alarms → Notification Channels (Slack, e-mail, webhook). If you close the window without choosing one, Cordatus asks whether it should let you know.
- Cancel installation (click twice) stops the installation and removes everything it set up.
- If the installation stops with an error, the failed step is shown. Continue installation resumes from where it stopped; Discard installation removes everything installed so far.
When it is done, the window shows Run:ai is installed. with the panel URL, the administrator e-mail, the cluster, the project and the data location. Click the URL and sign in with the password you chose.
Manage the cluster
Open NVIDIA Run:ai on the head device again (or Manage on the Run:ai Clusters page):
- Cluster nodes — every node with its role (head / worker), GPU count and state (Ready, Joining..., Failed, Leaving...). Use Add worker nodes → Add to join more devices, and Remove (click twice) to take a worker out. Its workloads move to other nodes, then Kubernetes is removed from it. A worker needs a saved sudo password and must reach the head node on ports 9345 and 6443.
- Inference endpoints — the inference workloads you deployed in Run:ai. Create public URL gives an endpoint a public address through Cordatus; Remove public URL closes it.
- Uninstall (click twice) removes Run:ai and Kubernetes from the head and all workers. Tick Also delete the stored container images to free their disk space too. Model folders are never deleted.
When you open the window on a worker device, it shows which cluster the device belongs to and offers Leave the cluster.
An endpoint spread over several nodes is marked distributed. To be reachable through a public URL,
its server must be started with --root-path <prefix>; the window shows the prefix.
The Run:ai Clusters page
LLM Launcher → Run:ai Clusters lists every Run:ai cluster Cordatus installed on your devices. When the page opens, it scans your online devices once. Scan devices runs the scan again, for example for a cluster installed before this page existed.
| Column | Meaning |
|---|---|
| Cluster | The cluster name, with the head node under it. |
| Nodes | One badge for the head and each worker. |
| Status | Online, Offline or Removing. |
| Run:ai panel | Link to the panel. |
| Endpoints | How many inference endpoints are ready. |
| Actions | Manage and Delete. |
Click the arrow on a row to expand it:
- Nodes — Node, Role, Device, IP, GPU in use (used / total) and Status, with Remove for worker nodes.
- Inference endpoints — Endpoint, Project, Status, Runs on (the node and the GPU share, for example 0.5 GPU (50%) · 23.6 GB), Address, and Public URL with a copy button and Create public URL / Remove public URL.
Manage opens the same NVIDIA Run:ai window. Delete uninstalls Run:ai from the head and all worker nodes after a confirmation; all workloads and endpoints on the cluster are deleted. Docker and Cordatus containers are not touched. If the head device has no saved sudo password, Cordatus asks you to Open Run:ai window and uninstall from there.
You can also ask the AI Assistant to deploy a model on your Run:ai cluster. It plans how the model is split over the nodes, shows the plan and starts the inference after you confirm.