Before You Deploy

Artifact Manifest

View as Markdown

This section provides a comprehensive list of all components required for NVIDIA Cloud Functions (NVCF) Self-Hosted deployment for basic inference. Additional components are needed for Low Latency Streaming (Simulation).

Artifacts Overview

The following tables list all artifacts required for an inference-only self-hosted NVCF deployment, organized by category, with their container images, Helm charts, and other resources.

Early Access (EA) Version Policy

During Early Access, artifact versions are updated frequently. The versions shown for Infrastructure Components are stable references, but all other components should use the latest published version from NGC.

To find the latest versions:

First, ensure you have the latest version of the NGC CLI installed and configured.

$# List available versions for any container image
$ngc registry image list "0833294136851237/nvcf-ncp-staging/<artifact-name>:*"
$
$# For Helm charts (OCI-compliant charts are stored in the container registry)
$ngc registry image list "0833294136851237/nvcf-ncp-staging/<chart-name>:*"

Helm chart types

Rows marked Chart (OCI) are OCI-compliant charts stored in the NGC container registry. This means:

  • Charts are pulled using oci:// URLs: helm pull oci://nvcr.io/0833294136851237/nvcf-ncp-staging/<chart-name> --version <version>
  • Charts are listed using the image registry command: ngc registry image list
  • When mirroring to private registries (e.g., ECR), use container image tools like skopeo or helm push/pull with OCI support

Rows marked Chart (HTTP) are traditional Helm repository charts, not OCI URLs. In this manifest, https://helm.ngc.nvidia.com/nvidia/omniverse/ddcs:5.0.0 means the chart ddcs in the omniverse Helm repository (https://helm.ngc.nvidia.com/nvidia/omniverse), at version 5.0.0. Add the Helm repository and pull the chart by name and version, for example:

$helm repo add omniverse https://helm.ngc.nvidia.com/nvidia/omniverse
$helm repo update
$helm pull omniverse/ddcs --version 5.0.0

Some supporting components such as the GPU Operator, OpenBao, NATS, Cassandra, etc. can alternatively be pulled directly from public NGC Catalog or other public opensource repositories if desired.

Artifact Registry Paths

Infrastructure Components

Core infrastructure services including NATS for messaging, NATS auth callout support, Cassandra for data storage, and OpenBao for secret management.

TypeComponent NameFull Path
Imagenats-boxnvcr.io/0833294136851237/nvcf-ncp-staging/nats-box:0.19.2-nonroot
Imagenats-servernvcr.io/0833294136851237/nvcf-ncp-staging/nats-server:2.11.10-alpine3.22
Imagenats-server-config-reloadernvcr.io/0833294136851237/nvcf-ncp-staging/nats-server-config-reloader:0.20.0
Chart (OCI)helm-nvcf-natsnvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-nats:0.6.0
Imagenvcf-nats-auth-callout-servicenvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-nats-auth-callout-service:0.3.3
Chart (OCI)helm-nvcf-nats-auth-callout-servicenvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-nats-auth-callout-service:1.0.1
Imagebitnami-cassandranvcr.io/0833294136851237/nvcf-ncp-staging/bitnami-cassandra:5.0.6-nv-1
Imagenvcf-cassandra-migrationsnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-cassandra-migrations:0.6.1
Chart (OCI)helm-nvcf-cassandranvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-cassandra:0.14.1
Imagenvcf-openbaonvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-openbao:2.2.2-nv-1
Imagenvcf-openbaonvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-openbao:2.5.1-nv-1.2.1
Imagenvcf-openbao-migrationsnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-openbao-migrations:0.11.0
Chart (OCI)helm-nvcf-openbao-servernvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-openbao-server:0.30.4
Imageoss-vault-k8snvcr.io/0833294136851237/nvcf-ncp-staging/oss-vault-k8s:1.6.0

Control Plane Components

Services that manage the NVCF platform including API gateway, deployment orchestration, invocation handling, LLM routing, and security services.

TypeComponent NameFull Path
Imagespotnvcr.io/0833294136851237/nvcf-ncp-staging/spot:1.556.0-v3474d670
Imagestrapnvcr.io/0833294136851237/nvcf-ncp-staging/strap:2.242.2
Chart (OCI)helm-nvcf-apinvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-api:1.18.1
Chart (OCI)helm-nvcf-sisnvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-sis:1.13.6
Imagenvcf-grpc-proxynvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-grpc-proxy:1.27.0
Chart (OCI)helm-nvcf-grpc-proxynvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-grpc-proxy:1.6.2
Imagenvcf-invocation-servicenvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-invocation-service:0.5.2
Chart (OCI)helm-nvcf-invocation-servicenvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-invocation-service:1.5.2
Imageess-apinvcr.io/0833294136851237/nvcf-ncp-staging/ess-api:v0.57.3
Chart (OCI)helm-nvcf-ess-apinvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-ess-api:1.5.2
Imagenotary-servicenvcr.io/0833294136851237/nvcf-ncp-staging/notary-service:1.9.4
Chart (OCI)helm-nvcf-notary-servicenvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-notary-service:1.3.1
Imagereval-servernvcr.io/0833294136851237/nvcf-ncp-staging/reval-server:0.15.0
Chart (OCI)helm-revalnvcr.io/0833294136851237/nvcf-ncp-staging/helm-reval:1.3.0
Imagenv-api-keysnvcr.io/0833294136851237/nvcf-ncp-staging/nv-api-keys:0.0.7
Chart (OCI)helm-nvcf-api-keysnvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-api-keys:1.4.0
Imagenvct-service-ossnvcr.io/0833294136851237/nvcf-ncp-staging/nvct-service-oss:1.2.11
Chart (OCI)helm-nvct-apinvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvct-api:1.0.2
Imagellm-api-gatewaynvcr.io/0833294136851237/nvcf-ncp-staging/llm-api-gateway:0.3.0
Imagellm-request-routernvcr.io/0833294136851237/nvcf-ncp-staging/stargate:0.2.0
Chart (OCI)helm-nvcf-llm-api-gatewaynvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-llm-api-gateway:1.1.0
Chart (OCI)helm-nvcf-llm-request-routernvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-llm-request-router:1.2.1

GPU Workload Components

Components that run on GPU nodes to manage function execution, including the NVCA operator and supporting containers.

TypeComponent NameFull Path
Imagenvcanvcr.io/0833294136851237/nvcf-ncp-staging/nvca:3.0.0-rc.13
Imagenvca-operatornvcr.io/0833294136851237/nvcf-ncp-staging/nvca-operator:3.0.0-rc.13
Chart (OCI)helm-nvca-operatornvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvca-operator:1.11.1
Imagenvcf_worker_utilsnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf_worker_utils:2.101.0
Imagenvcf_worker_initnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf_worker_init:2.102.0
Imagenvcf_worker_nicllsnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf_worker_niclls:2.105.7
Imageess-agentnvcr.io/0833294136851237/nvcf-ncp-staging/ess-agent:1.0.5
Imagenvcf-image-credential-helpernvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-image-credential-helper:0.5.1

Supporting Components

Additional utilities and helper services required for the platform, including the NVIDIA GPU Operator for GPU node management.

TypeComponent NameFull Path
Imagealpine-k8snvcr.io/0833294136851237/nvcf-ncp-staging/alpine-k8s:1.30.12
Imageload_tester_supremenvcr.io/0833294136851237/nvcf-ncp-staging/load_tester_supreme:0.0.8
Chart (HTTP)gpu-operatorPublic NGC Helm repo
Imagegpu-operator-validatorPublic NGC
Imagek8s-device-pluginPublic NGC
Chart (HTTP)ebs-csi-driverhttps://kubernetes-sigs.github.io/aws-ebs-csi-driver
Chart (HTTP)csi-driver-smbhttps://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts

Reference Architecture Components

Optional components for the reference deployment architecture.

TypeComponent NameFull Path
Chart (OCI)nvcf-gateway-routesnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-gateway-routes:1.10.0
Imageadmin-token-issuer-proxynvcr.io/0833294136851237/nvcf-ncp-staging/admin-token-issuer-proxy:1.0.1
Chart (OCI)helm-admin-token-issuer-proxynvcr.io/0833294136851237/nvcf-ncp-staging/helm-admin-token-issuer-proxy:1.3.1

Observability Components

Optional example components for monitoring and observability. These are provided as reference implementations only and are not intended for production use. See self-hosted-example-dashboards for deployment instructions.

TypeComponent NameFull Path
Chart (OCI)nvcf-observability-reference-stacknvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-observability-reference-stack:1.7.0
Chart (OCI)nvcf-example-dashboardsnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-example-dashboards:1.6.0
Chart (OCI)helm-nvcf-state-metricsnvcr.io/0833294136851237/nvcf-ncp-staging/helm-nvcf-state-metrics:1.0.1

Container Caching Components

Optional components for accelerating container image pulls across all workload types.

TypeComponent NameFull Path
Imagenvcf-container-cachenvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-container-cache:v1.1.31
Chart (OCI)helm-nvcf-container-cachenvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-container-cache:0.25.6
Imagenvcf-proxy-tls-certsnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-proxy-tls-certs:1.2.0

Simulation Caching Components

Optional caching components for Low Latency Streaming (LLS) and simulation workloads, including shader caching, derived data caching, and USD content caching.

TypeComponent NameFull Path
Imagegxcache-webhooknvcr.io/0833294136851237/nvcf-ncp-staging/gxcache-webhook:59bd8ec5
Imagegxcache-initnvcr.io/0833294136851237/nvcf-ncp-staging/gxcache-init:1e47f722
Imagegxcache-servicenvcr.io/0833294136851237/nvcf-ncp-staging/gxcache-service:b206ce39
Chart (OCI)helm-gxcachenvcr.io/0833294136851237/nvcf-ncp-staging/gxcache:0.8.2
Imageddcs-dist-kvnvcr.io/nvidia/omniverse/ddcs-dist-kv:5.0.0
Imageusd-content-cachenvcr.io/nvidia/omniverse/usd-content-cache:3.0.1
Chart (HTTP)ddcshttps://helm.ngc.nvidia.com/nvidia/omniverse/ddcs:5.0.0
Chart (HTTP)usd-content-cachehttps://helm.ngc.nvidia.com/nvidia/omniverse/usd-content-cache:3.0.3

Storage API Components

Optional components for USD Storage API functionality used in simulation workloads.

TypeComponent NameFull Path
Imagestorage-servicenvcr.io/nvidia/omniverse/storage-service:1.0.2
Imagesimple-nginxnvcr.io/nvidia/omniverse/simple-nginx:1.0.2
Chart (HTTP)storage-servicehttps://helm.ngc.nvidia.com/nvidia/omniverse/storage-service:1.0.2
Chart (HTTP)discovery-servicehttps://helm.ngc.nvidia.com/nvidia/omniverse/discovery-service:2.3.8

Low Latency Streaming (LLS) Components

Components for Low Latency Streaming functionality.

TypeComponent NameFull Path
Imagestreaming-proxynvcr.io/0833294136851237/nvcf-ncp-staging/streaming-proxy:2.0.1
Chart (OCI)gdn-streamingnvcr.io/0833294136851237/nvcf-ncp-staging/gdn-streaming:2.0.1

Other Published Components

Additional components present in the current stack artifact manifest.

TypeComponent NameFull Path
Imagenvcf-service-ossnvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-service-oss:1.1.7

Deployment Resources

Helmfile and CLI resources for deployment.

TypeComponent NameFull Path
Resourcenvcf-self-managed-stacknvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-self-managed-stack:0.6.0-rc.20
Resourcenvcf-clinvcr.io/0833294136851237/nvcf-ncp-staging/nvcf-cli:0.0.30

Component Descriptions

Infrastructure Components

Component NameDescription
nats-boxNATS utility container for debugging and administration
nats-serverPub Sub Messages, used for Function Invocation and Deployment
nats-server-config-reloaderConfiguration reloader for NATS server
helm-nvcf-natsHelm chart for NATS deployment
nvcf-nats-auth-callout-serviceAuth callout service for NATS authorization
helm-nvcf-nats-auth-callout-serviceHelm chart for the NATS auth callout service
bitnami-cassandraDatabase for Account, Function and Cluster Management
nvcf-cassandra-migrationsDatabase migration scripts for Cassandra
helm-nvcf-cassandraHelm chart for Cassandra deployment
nvcf-openbaoSecret management (OpenBao/Vault)
nvcf-openbao-migrationsMigration scripts for OpenBao
helm-nvcf-openbao-serverOpenBao Helm chart
oss-vault-k8sKubernetes integration for secret management

Control Plane Components

Component NameDescription
spotSpot Instance Service (SIS) - Manages deployments, cluster and instance state
strapNVCF API service, refer to self-hosted-api for full API specification
helm-nvcf-apiHelm chart for NVCF API service
helm-nvcf-sisHelm chart for Spot Instance Service
nvcf-grpc-proxyUsed for bi-directional communication and state management
helm-nvcf-grpc-proxyHelm chart for GRPC Proxy deployment
nvcf-invocation-serviceHandles stateless HTTP Function invocation requests
helm-nvcf-invocation-serviceHelm chart for Invocation Service
ess-apiEncrypted Secrets Service - Used for application secret injection
helm-nvcf-ess-apiHelm chart for ESS API
notary-serviceUsed to sign and validate Functions and nodes
helm-nvcf-notary-serviceHelm chart for Notary Service
reval-serverReval (re-validation) service - Handles background re-validation of function state
helm-revalHelm chart for Reval service
nv-api-keysAPI Key generation and management
helm-nvcf-api-keysHelm chart for API Keys service
llm-api-gatewayGateway service for OpenAI-compatible LLM requests
llm-request-routerRequest routing service backed by the Stargate image
helm-nvcf-llm-api-gatewayHelm chart for LLM API gateway services
helm-nvcf-llm-request-routerHelm chart for LLM request routing services

GPU Workload Components

Component NameDescription
nvcaPerforms the registration of the cluster and deployment orchestration in-cluster
helm-nvca-operator (chart)Helm chart for NVCA operator deployment (current chart name, versions 1.4.0+)
nvcf_worker_utilsActs as a proxy to NATS from the user’s application
nvcf_worker_initSetup & Resource loading on deployment for the users application
nvcf_worker_nicllsNIC LLS worker component for low latency streaming workloads
ess-agentInjects User Secrets
nvcf-image-credential-helperHelper for managing container image credentials

Supporting Components

Component NameDescription
alpine-k8sKubernetes utility container
gpu-operatorNVIDIA GPU Operator for dynamic GPU discovery - Can also pull directly from public NGC Catalog
gpu-operator-validatorGPU Operator validation component
k8s-device-pluginKubernetes device plugin for GPU support
ebs-csi-driverAWS EBS CSI Driver for persistent volume provisioning on EKS
csi-driver-smbCSI Driver for SMB/CIFS file shares

Reference Architecture Components

Component NameDescription
nvcf-gateway-routesGateway routing configuration for reference architecture
admin-token-issuer-proxyAdmin token management proxy
helm-admin-token-issuer-proxyHelm chart for admin token issuer proxy

Observability Components

Component NameDescription
nvcf-observability-reference-stackReference observability backend (Prometheus, Grafana, Loki, Tempo, OpenTelemetry Collector)
nvcf-example-dashboardsPre-configured Grafana dashboards for NVCF control-plane metrics
helm-nvcf-state-metricsHelm chart for NVCF state metrics service

Container Caching Components

Component NameDescription
nvcf-container-cacheAccelerates container image pulls by caching layers locally on nodes
helm-nvcf-container-cacheHelm chart for container cache deployment
nvcf-proxy-tls-certsTLS certificate management for container cache proxy

Simulation Caching Components

Component NameDescription
gxcache-webhookShader cache webhook for intercepting and caching shader compilation requests
gxcache-initInit container for shader cache setup
gxcache-serviceBackend service for shader cache storage and retrieval
helm-gxcacheHelm chart for deploying the complete shader cache stack
ddcs-dist-kvDerived Data Cache Service - caches computed/derived data for simulation workloads
ddcsHelm chart for DDCS deployment
usd-content-cacheUSD Content Cache - caches Universal Scene Description assets for streaming
usd-content-cache (chart)Helm chart for USD Content Cache deployment

Storage API Components

Component NameDescription
storage-serviceUSD Storage Service for managing assets in simulation workloads
storage-service (chart)Helm chart for Storage Service deployment
simple-nginxSimple NGINX container for Storage API routing
discovery-serviceHelm chart for Storage API Discovery Service

Low Latency Streaming (LLS) Components

Component NameDescription
streaming-proxyLLS Streaming Proxy Container
gdn-streamingLLS Self-Hosted Helm Chart

Deployment Resources

Component NameDescription
nvcf-self-managed-stackHelmfile bundle for self-managed stack deployment
nvcf-cliCommand-line interface for managing functions and deployments