Configuration

KAI Scheduler Integration Guide

View as Markdown

KAI Scheduler is an open source Kubernetes scheduler for AI workloads. NVCF uses it for GPU bin-packing, queues, gang scheduling, and topology-aware placement.

Install KAI Scheduler

Use a tested KAI Scheduler release that is compatible with the NVCF compute plane stack.

Set addons.kaiScheduler.enabled in the nvcf-compute-plane Helmfile environment to install KAI Scheduler as release and namespace kai-scheduler. Grove, Dynamo, and topology-aware scheduling require this add-on. Skip the manual installation below when the add-on is enabled.

Use the manual path when KAI is managed outside the compute plane stack.

NVCA expects a parent queue named default-parent-queue and a child queue named default-queue. Other queues may also exist.

Set unlimited (-1) quotas and limits on every queue used for NVCF workloads. This lets NVCA track the complete cluster capacity. If NVCF and non-NVCF workloads share a cluster with limited KAI queues, enable Shared Cluster mode so NVCA excludes non-NVCF nodes from capacity tracking and scheduling.

Create values.yaml with the required default queues:

kai-scheduler-queues.yaml
1scheduler:
2  placementStrategy: binpack
3  plugins:
4    nodeplacement:
5      arguments:
6        gpu: binpack
7        cpu: spread
8  actions:
9    preempt:
10      enabled: false
11    consolidation:
12      enabled: false
13
14defaultQueue:
15  createDefaultQueue: true
16  parentName: default-parent-queue
17  childName: default-queue
18  parentResources:
19    cpu:
20      quota: -1
21      limit: -1
22      overQuotaWeight: 1
23    gpu:
24      quota: -1
25      limit: -1
26      overQuotaWeight: 1
27    memory:
28      quota: -1
29      limit: -1
30      overQuotaWeight: 1
31  childResources:
32    cpu:
33      quota: -1
34      limit: -1
35      overQuotaWeight: 1
36    gpu:
37      quota: -1
38      limit: -1
39      overQuotaWeight: 1
40    memory:
41      quota: -1
42      limit: -1
43      overQuotaWeight: 1
$helm install kai-scheduler oci://ghcr.io/kai-scheduler/kai-scheduler/kai-scheduler -f values.yaml -n kai-scheduler --create-namespace --version v0.14.0

Schedule multi-Pod workloads

KAI can hold a multi-Pod workload until all required members fit. Grove and Dynamo build on this behavior for multi-role inference services. See Gang Scheduling for add-on configuration, workload examples, supported resource types, and troubleshooting.

On NVLink-optimized clusters, KAI can also place the complete gang in one GPU clique. See Topology-Aware Scheduling for GPU DRA prerequisites, topology configuration, Grove bindings, and function examples.