MIG (GPU Partitioning)
MIG (Multi-Instance GPU) cuts one physical NVIDIA card into several smaller, hardware-isolated GPUs. Each piece — an instance — has its own slice of compute, its own memory and its own video engines, and behaves like a separate card: one container gets one instance, and nothing running on it can disturb what runs on the others.
Cordatus manages this from Devices → your device → Metrics, and the instances it creates then appear everywhere a GPU can be picked.
Fleet-Management/mig-split-card.mp4 — opening the MIG dialog from the GPU strip, reading the card
list, splitting a card into four instances, and the instances appearing in the strip afterwards.
Why split a card
A 96 GB card running one 8 B model wastes most of itself, but sharing it between two containers the ordinary way means they fight over memory and one can starve the other. Splitting turns that single card into, for example, four 24 GB GPUs:
- Isolation — an out-of-memory crash in one instance does not touch the others.
- Density — four small models, four jobs, or four users on one card.
- Predictability — each instance gets a fixed share of compute and memory, not whatever is left over.
The cost is that an instance is smaller than the card, and a workload cannot be spread across two instances of the same card. Split when you want more, smaller GPUs; leave the card whole when you want one big GPU.
Before you start
| Requirement | Detail |
|---|---|
| Device type | x86 devices with NVIDIA cards that report through nvidia-smi. Jetson devices do not offer MIG. |
| Card support | Decided by the driver, not by the card's name. Two cards with the same model name on the same machine can differ — one supports MIG, the other does not. Cordatus asks the device and tells you per card. |
| Connection | The device must be Online. |
| Root | Changing MIG mode needs root on the device, so Cordatus asks for the sudo password and verifies it before doing anything. |
A card in MIG mode stops rendering graphics. If it is the only GPU in a machine with a desktop session, the desktop will go dark. The confirmation screen warns you when the card you picked is driving a display.
Open the MIG dialog
- Go to Devices and open the device's Metrics panel.
- On the right-hand GPU strip, at the end of the cards, there is a dashed MIG tile. Its subtitle reads Configure, or the number of instances already carved out of this device's cards.
- Click it. Cordatus reads the GPU capabilities from the device — "Reading GPU capabilities…" — and lists every card.
The tile only appears when the device reports NVIDIA GPUs.
Reading the card list
Each card is one row: its index, its name, and its state.
| State | Meaning |
|---|---|
| Whole card | Not split, and it can be. Split options are listed underneath. |
| Whole card · MIG not available | The driver says this card cannot do MIG. No options are offered. |
| Split into N instances (+ MIG on chip) | Already split. The only action is Merge back into one card. |
Split a card
Under a splittable card, Cordatus lists how many pieces you can cut it into — not the raw driver profile list. A card typically advertises several entries that all say "24gb" and two of them quietly produce a single instance rather than four, so Cordatus keeps only the real splits and picks the best profile variant for you (the one that keeps graphics APIs and the video engines).
Each option shows what one instance gets:
- N instances · X GB each — the headline.
- SM — compute units.
- decoder / encoder — video engines.
0 decodermatters: without one, a DeepStream pipeline cannot decode a camera stream. - JPEG — the JPEG engine count, when the driver reports it.
- graphics — this variant still exposes the graphics APIs.
On devices with unified memory, MIG splits compute only — memory stays shared and each instance reports no size of its own. The dialog says so above the options instead of printing a GB figure.
Pick an option and Cordatus checks the card before asking you to confirm — "Checking what is using this card…".
The confirmation screen
It tells you the whole cost before anything happens:
| Line | What it means |
|---|---|
| These containers use this card and will be stopped: (names) | Containers pinned to this card, by name. The driver refuses to change MIG mode while a process holds the card, so they are stopped. They are not restarted afterwards — they were pinned to a card that no longer exists; recreate them against an instance. |
| The desktop session on this device is using this card… | Instances cannot be created right now. The layout is saved and applied on the next reboot, before the desktop starts. |
| This card is driving a display… | Graphics stop on this card once it is in MIG mode. |
| No reboot is needed. Cordatus remembers this layout… | The card was free, so the split happens immediately. |
Press Split the card. If the sudo password is not stored yet, Cordatus asks for it and verifies it first.
Why the layout is remembered
The driver forgets both the MIG mode and the instances on every restart — a split card comes back whole. So a split is not a one-off command: Cordatus stores the layout you asked for on the device and rebuilds it at every boot, before the desktop starts. That is also why a split blocked by a desktop session can still be honoured — it simply happens at the next reboot.
What the result says
| Result | Meaning |
|---|---|
| GPU N was split into X instances (profile). | Done, and live now. |
| GPU N is set to X instances, but the desktop session is using the card… | Saved, applied at the next boot. Restart the device to get there. |
| Splitting failed at the "step" step. | The driver refused a step; the card is left as it was. |
| GPU N is one whole card again. | Merged. |
| GPU N is set to become whole again, but the desktop session is using one of the instances… | Saved, the card becomes whole at the next boot. |
| Stopped containers: … | Appended to the message, naming what was stopped. |
The restart offer
When the work is left to the next boot, Cordatus offers to restart the device — but never before answering whether the device will come back:
- Cordatus starts automatically after boot, so the device will come back on its own. — safe.
- This device will not come back online by itself. — with the reason: Cordatus is not set to launch on startup; or it starts from the desktop session and this machine has no automatic login, so someone has to log in at the machine; or it starts from a user service that only runs after login; or it is installed as a system service that is not enabled.
It also tells you how many running containers will stop. Cordatus restores the ones it manages once it is back.
If the answer is will not come back, fix the startup path first (see Device Actions) or restart the machine where you can physically reach it. There is no remote way back from a device that does not start Cordatus on boot.
Merge back into one card
Open the same dialog, find the split card and press Merge back into one card. The confirmation lists the containers holding its instances — they are stopped for the same reason as before — and warns you when the desktop session is sitting inside one of the instances, in which case the card becomes whole at the next boot instead.
Merging also clears the stored layout, so the card stays whole from then on.
Where instances show up afterwards
The Metrics GPU strip
Instances are drawn as their own tiles, in indigo, labelled GPU 1 (MIG 0). The carved-up parent
card is not drawn — nothing can run on it.
The big number on an instance tile is VRAM used, not utilisation. On a card in MIG mode the
driver reports no utilisation figure at all, for the card or its instances, so Cordatus shows memory
occupancy — which is the question you actually ask when picking an instance — and writes — rather
than 0% wherever a value could not be measured. The line underneath is the profile name, the
decoder/encoder counts and the VRAM.
Picking a GPU for a container
In the GPU picker (LLM Launcher application launch, Compose services, AI Platform jobs) whole cards and instances are listed together, and the two are mutually exclusive:
- Either N whole cards, or exactly one MIG instance — picking an instance clears a whole-card selection and vice versa, with "Whole GPUs are selected. Clear them to pick a MIG instance."
- Use all whole GPUs never includes an instance. On a device where every card is split the toggle is disabled — "Every card on this device is in MIG mode — pick an instance instead."
- On screens where instances cannot be used, a note above the grid says so instead of hiding them silently: "This device has N card(s) in MIG mode. They cannot be used on this screen."
- On AI Platform job screens, instances with no video decoder or encoder are hidden, because a DeepStream pipeline cannot be built without them — "N MIG instance(s) are hidden here because they have no video decoder or encoder."
Multi-worker (Dynamo) pipelines
A worker can take one instance — an engine cannot split work across slices of the same card, and a worker cannot mix an instance with a whole GPU. The pipeline editor enforces both and says why. Scale across instances by adding more workers, not by giving one worker more slices.
Dynamo hands each worker its GPUs through CUDA_VISIBLE_DEVICES, and some engine builds reject a
MIG UUID there. If a Dynamo worker on an instance fails to start, run that pipeline on whole cards.
Limits worth knowing
- One instance per container. There is no NVLink or peer path between slices of the same card, so tensor-parallel across two instances is not possible — the engine sees two devices on the same PCI bus and refuses.
- No utilisation metric. MIG mode removes the GPU/memory/encoder/decoder utilisation readings
from
nvidia-smifor that card. Cordatus shows memory instead of inventing a number. - The card's name does not tell you whether it can do MIG. Only the driver does, and Cordatus asks it per card, every time the dialog opens.
- Nothing is restarted for you after a split or a merge. The containers that were stopped were pinned to hardware that no longer exists in that shape.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
| The MIG tile is not in the GPU strip | The device reports no NVIDIA GPU, or it is a Jetson. |
| "No NVIDIA GPU was reported by this device." | nvidia-smi is missing or did not answer. Check the driver on the device. |
| "This card cannot do MIG…" | The driver says no. A card with the same model name elsewhere may still support it. |
| "Changing a GPU into MIG mode needs root on the device." | Enter the device's sudo password when asked; Cordatus verifies it before running anything. |
| The split did not happen and the message mentions the desktop session | Expected on a machine with a desktop. Restart the device — the layout is applied at boot, before the desktop starts. |
| A container will not start after a split | It was pinned to the whole card. Recreate it and pick an instance in the GPU step. |
| The card came back whole after a reboot | The layout was not stored, or the device has no saved sudo password to rebuild it with. Split it again from the dialog. |