Gateway Routing and DNS

This guide explains how NVCF self-hosted deployments route traffic through the Kubernetes Gateway API, and how to configure DNS and HTTPS for production environments.

Overview

The NVCF self-hosted deployment uses the Kubernetes Gateway API for ingress traffic management. This provides:

• Hostname-based routing for HTTP services (API Keys, NVCF API, Invocation)
• Port-based routing for gRPC services
• Single load balancer for all NVCF services
• Cross-namespace routing via ReferenceGrants

The Gateway API is a Kubernetes standard with multiple implementations. The examples on this page use Envoy Gateway, but you can use any Gateway API-compliant controller that supports the requirements below.

Gateway quickstart

Use this quickstart before any remote deployment path that needs NVCF services reachable through Gateway API, including one-click CLI, Helmfile, and standalone Helm chart installation.

Skip this section for local k3d flows that already create the local Gateway and route hostnames.

Install Gateway API CRDs

Install the Kubernetes Gateway API CRDs v1.2.0:

$
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/experimental-install.yaml

If you replace v1.2.0 with another version, verify compatibility with the GatewayClass and Gateway resources that you create.

Create namespaces and labels

Create the namespaces used by the Gateway and NVCF services, then label the route-owning namespaces so the Gateway can accept cross-namespace routes:

$
for namespace in envoy-gateway-system envoy-gateway api-keys ess sis nvcf; do
$
  kubectl create namespace "$namespace" --dry-run=client -o yaml | kubectl apply -f -
$
done
$
$
for namespace in envoy-gateway api-keys ess sis nvcf; do
$
  kubectl label namespace "$namespace" nvcf/platform=true --overwrite
$
done

Install Envoy Gateway

Install Envoy Gateway as the Gateway API controller:

$
helm upgrade --install eg oci://docker.io/envoyproxy/gateway-helm \
>
  --version v1.1.3 \
>
  -n envoy-gateway-system

Verify the controller pod is running:

$
kubectl get pods -n envoy-gateway-system

Create GatewayClass

Create the GatewayClass resource:

$
kubectl apply -f - <<EOF
$
apiVersion: gateway.networking.k8s.io/v1
$
kind: GatewayClass
$
metadata:
$
  name: eg
$
spec:
$
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
$
EOF

Create Gateway

Create the Gateway resource with an HTTP listener on port 80 and a TCP listener on port 10081 for gRPC.

$
kubectl apply -f - <<EOF
$
apiVersion: gateway.networking.k8s.io/v1
$
kind: Gateway
$
metadata:
$
  name: nvcf-gateway
$
  namespace: envoy-gateway
$
  annotations:
$
    # AWS (EKS)
$
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
$
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
$
    # GCP (GKE) example:
$
    # cloud.google.com/load-balancer-type: "External"
$
    # Azure (AKS) example:
$
    # service.beta.kubernetes.io/azure-load-balancer-internal: "false"
$
spec:
$
  gatewayClassName: eg
$
  listeners:
$
  - name: http
$
    protocol: HTTP
$
    port: 80
$
    allowedRoutes:
$
      namespaces:
$
        from: Selector
$
        selector:
$
          matchLabels:
$
            nvcf/platform: "true"
$
  - name: tcp
$
    protocol: TCP
$
    port: 10081
$
    allowedRoutes:
$
      namespaces:
$
        from: Selector
$
        selector:
$
          matchLabels:
$
            nvcf/platform: "true"
$
EOF

Capture Gateway values

Wait for the Gateway to be programmed and export the values used by the install guides:

$
export HTTP_GATEWAY_NAMESPACE="envoy-gateway"
$
export HTTP_GATEWAY_NAME="nvcf-gateway"
$
export GRPC_GATEWAY_NAMESPACE="envoy-gateway"
$
export GRPC_GATEWAY_NAME="nvcf-gateway"
$
$
kubectl -n "$HTTP_GATEWAY_NAMESPACE" wait "gateway/$HTTP_GATEWAY_NAME" \
>
  --for=condition=Programmed=True --timeout=10m
$
kubectl -n "$GRPC_GATEWAY_NAMESPACE" wait "gateway/$GRPC_GATEWAY_NAME" \
>
  --for=condition=Programmed=True --timeout=10m
$
$
export GATEWAY_ADDR="$(kubectl -n "$HTTP_GATEWAY_NAMESPACE" get "gateway/$HTTP_GATEWAY_NAME" \
>
  -o jsonpath='{.status.addresses[0].value}')"
$
export GRPC_GATEWAY_ADDR="$(kubectl -n "$GRPC_GATEWAY_NAMESPACE" get "gateway/$GRPC_GATEWAY_NAME" \
>
  -o jsonpath='{.status.addresses[0].value}')"
$
$
test -n "$GATEWAY_ADDR"
$
test -n "$GRPC_GATEWAY_ADDR"

Use GATEWAY_ADDR as the route hostname suffix for test environments without production DNS. Use your production domain instead when you configure DNS and HTTPS.

Use Gateway values with install paths

Configure the CLI for one-click

For one-click installs on remote clusters, configure the CLI before running nvcf-cli self-hosted up. The command installs the control plane and then immediately calls API, API Keys, invocation, and gRPC endpoints during health and cluster registration phases.

$
export CLUSTER_NAME="nvcf-remote"
$
export NCA_ID="nvcf-default"
$
export REGION="us-west-1"
$
export STACK_DOMAIN="$GATEWAY_ADDR"
$
$
cat > .nvcf-cli.yaml <<EOF
$
base_http_url: "http://${GATEWAY_ADDR}"
$
invoke_url: "http://${GATEWAY_ADDR}"
$
base_grpc_url: "${GRPC_GATEWAY_ADDR}:10081"
$
api_keys_service_url: "http://${GATEWAY_ADDR}"
$
$
api_keys_host: "api-keys.${STACK_DOMAIN}"
$
api_host: "api.${STACK_DOMAIN}"
$
invoke_host: "invocation.${STACK_DOMAIN}"
$
$
api_keys_service_id: "nvidia-cloud-functions-ncp-service-id-aketm"
$
api_keys_issuer_service: "nvcf-api"
$
api_keys_owner_id: "svc@nvcf-api.local"
$
$
client_id: "${NCA_ID}"
$
EOF

Do not leave literal shell variables in the YAML. If you use production DNS and HTTPS, set STACK_DOMAIN to your production domain and update the URL schemes and ports accordingly.

Gateway API Implementations

The nvcf-gateway-routes chart creates standard Kubernetes Gateway API resources (HTTPRoute, TCPRoute) that work with any Gateway API-compliant controller. You are not locked into a specific implementation.

Popular implementations include Envoy Gateway (used in our examples), Istio, Traefik, Kong, Contour, and cloud-native options like GKE Gateway Controller.

Minimum Requirements

Any Gateway API implementation you choose must support:

1. HTTPRoute - For HTTP/HTTPS routing with hostname matching
2. TCPRoute - For gRPC routing (requires experimental Gateway API CRDs)
3. Cross-namespace routing - Routes in one namespace referencing services in another

Using a Different Implementation

To use a different Gateway API implementation instead of Envoy Gateway:

1. Install your chosen controller following its documentation

2. Create namespaces with nvcf/platform=true labels as shown in Gateway quickstart

3. Create a GatewayClass for your controller

4. Create a Gateway with http (port 80) and tcp (port 10081) listeners

5. Update your install configuration to reference your Gateway:

   1
   ingress:
   2
     gatewayApi:
   3
       enabled: true
   4
       gateways:
   5
         shared:
   6
           name: your-gateway-name      # Your Gateway resource name
   7
           namespace: your-namespace     # Namespace where Gateway exists
   8
           listenerName: http            # HTTP listener name
   9
         grpc:
   10
           name: your-gateway-name       # Can be same Gateway with different listener
   11
           namespace: your-namespace
   12
           listenerName: tcp             # TCP listener name for gRPC

The nvcf-gateway-routes chart will create HTTPRoutes and TCPRoutes that attach to your specified Gateway.

Not Using Gateway API

While technically possible to bypass the Gateway API entirely, this is not recommended:

• The nvcf-gateway-routes chart specifically creates Gateway API resources
• You would need to manually create and maintain all routing configuration
• Traditional Kubernetes Ingress does not support TCPRoute (required for gRPC)
• Multiple LoadBalancer services would require multiple external IPs

If you have a specific requirement that prevents using Gateway API, you would need to:

1. Disable nvcf-gateway-routes in your helmfile
2. Create your own Ingress or Service resources for each NVCF service
3. Configure hostname routing manually
4. Set up a separate TCP load balancer for gRPC on port 10081

Gateway Architecture

Components

The gateway architecture consists of two layers:

User-configured

These resources must be created manually before deploying the control plane:

• Namespaces with nvcf/platform=true labels
• Gateway API controller installation (Envoy Gateway, Istio, Traefik, etc.)
• GatewayClass resource
• Gateway resource with http (port 80) and tcp (port 10081) listeners

Auto-Created by nvcf-gateway-routes

When you deploy the control plane via helmfile, the nvcf-gateway-routes chart automatically creates:

• HTTPRoutes for API Keys, NVCF API, and Invocation services
• Optional LLM invocation HTTPRoute when the llmInvocation route is enabled
• TCPRoute for gRPC
• ReferenceGrants for cross-namespace routing permissions

These routes attach to the Gateway you prepared in Gateway quickstart.

Route Configuration

How Routing Works

1. The Gateway’s LoadBalancer service exposes ports 80 (HTTP) and 10081 (gRPC) externally.
2. HTTP requests arrive at port 80. The Gateway inspects the Host header and matches it against HTTPRoute hostnames.
3. The matching HTTPRoute forwards the request to the appropriate backend service (e.g., api-keys service on port 8080).
4. gRPC requests arrive at port 10081. The TCPRoute forwards all traffic directly to the grpc service—no hostname matching required.

Verifying Gateway Configuration

After deploying the control plane, use these commands to verify your gateway configuration.

Get the Gateway Load Balancer Address

$
# Get the gateway's external address (hostname or IP)
>
export GATEWAY_ADDR=$(kubectl get gateway nvcf-gateway -n envoy-gateway \
>
  -o jsonpath='{.status.addresses[0].value}')
>
echo "Gateway Address: $GATEWAY_ADDR"

Verify HTTPRoute Hostnames

The gateway routes requests based on the Host header. Check what hostnames are configured:

$
# List all HTTPRoutes and their hostnames
$
kubectl get httproute -A -o jsonpath='{range .items[*]}{.metadata.name}: {.spec.hostnames[*]}{"\n"}{end}'
$
$
# Example output:
$
# api-keys: api-keys.a1b2c3d4.us-west-2.elb.amazonaws.com
$
# nvcf-api: api.a1b2c3d4.us-west-2.elb.amazonaws.com
$
# invocation-service: *.invocation.a1b2c3d4.us-west-2.elb.amazonaws.com invocation.a1b2c3d4.us-west-2.elb.amazonaws.com

Verify gRPC TCPRoute

$
# Check gRPC routing is configured
$
kubectl get tcproute -A
$
# Expected output:
$
# NAMESPACE       NAME   AGE
$
# envoy-gateway   grpc   19h
$
$
# Verify the gateway exposes port 10081
$
kubectl get svc -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-name=nvcf-gateway \
>
  -o jsonpath='{.items[0].spec.ports[*].port}'
$
# Expected output includes: 80 10081

Test Connectivity

$
# Test HTTP endpoints with Host header
$
curl -H "Host: api-keys.$GATEWAY_ADDR" http://$GATEWAY_ADDR/health
$
curl -H "Host: api.$GATEWAY_ADDR" http://$GATEWAY_ADDR/health
$
$
# Test gRPC endpoint (requires grpcurl)
$
grpcurl -plaintext $GATEWAY_ADDR:10081 grpc.health.v1.Health/Check

Development: Host Header Routing

For development and testing when you don’t have DNS configured, you can use Host header overrides to route requests through the gateway.

Why Host Headers Are Needed

The Envoy Gateway uses hostname-based routing to direct traffic to different backend services through a single load balancer. When you send a request to the raw load balancer address (e.g., http://a1b2c3d4.elb.amazonaws.com), the gateway needs to know which service to route to.

Without the correct Host header, the gateway cannot match the request to an HTTPRoute and returns 404.

Configuring Host Headers

When using tools that support custom Host headers (like the NVCF CLI or curl), specify the expected hostname:

$
# Example: curl with Host header override
$
curl -H "Host: api-keys.a1b2c3d4.us-west-2.elb.amazonaws.com" \
>
  http://a1b2c3d4.us-west-2.elb.amazonaws.com/v1/admin/keys

For the NVCF CLI, configure the *_host settings in your configuration file:

1
# Endpoints point to load balancer
2
base_http_url: "http://a1b2c3d4.us-west-2.elb.amazonaws.com"
3
invoke_url: "http://a1b2c3d4.us-west-2.elb.amazonaws.com"
4
api_keys_service_url: "http://a1b2c3d4.us-west-2.elb.amazonaws.com"
5
base_grpc_url: "a1b2c3d4.us-west-2.elb.amazonaws.com:10081"
6
7
# Host header overrides for routing
8
api_keys_host: "api-keys.a1b2c3d4.us-west-2.elb.amazonaws.com"
9
api_host: "api.a1b2c3d4.us-west-2.elb.amazonaws.com"
10
invoke_host: "invocation.a1b2c3d4.us-west-2.elb.amazonaws.com"

See cli-configuration for complete CLI configuration documentation.

Production: DNS and HTTPS

For production deployments, configure proper DNS and TLS to eliminate the need for Host header overrides.

Benefits

With proper DNS and HTTPS:

• DNS records resolve service hostnames (e.g., api-keys.nvcf.example.com) to your Gateway’s load balancer
• TLS certificates secure all traffic
• Clients use simple URLs without Host header overrides
• Browsers and other clients can access services directly

Step 1: Choose a Domain

Select a domain you control for your NVCF deployment:

# Example domain structure
nvcf.example.com                    # Base domain
├── api-keys.nvcf.example.com       # API Keys service
├── api.nvcf.example.com            # NVCF API
├── invocation.nvcf.example.com     # Invocation service
├── *.invocation.nvcf.example.com   # Wildcard for function routing
└── grpc.nvcf.example.com           # gRPC endpoint (optional, for documentation)

Step 2: Create DNS Records

Create DNS records pointing to your Gateway’s load balancer address:

# A/CNAME records (replace with your load balancer address)
api-keys.nvcf.example.com     → a1b2c3d4.us-west-2.elb.amazonaws.com
api.nvcf.example.com          → a1b2c3d4.us-west-2.elb.amazonaws.com
invocation.nvcf.example.com   → a1b2c3d4.us-west-2.elb.amazonaws.com
*.invocation.nvcf.example.com → a1b2c3d4.us-west-2.elb.amazonaws.com

AWS Route 53 Example:

$
# Get your Gateway load balancer address
$
GATEWAY_ADDR=$(kubectl get gateway nvcf-gateway -n envoy-gateway \
>
  -o jsonpath='{.status.addresses[0].value}')
$
$
# Create CNAME records (example using AWS CLI)
$
aws route53 change-resource-record-sets --hosted-zone-id YOUR_ZONE_ID --change-batch '{
>
  "Changes": [
>
    {"Action": "CREATE", "ResourceRecordSet": {"Name": "api-keys.nvcf.example.com", "Type": "CNAME", "TTL": 300, "ResourceRecords": [{"Value": "'$GATEWAY_ADDR'"}]}},
>
    {"Action": "CREATE", "ResourceRecordSet": {"Name": "api.nvcf.example.com", "Type": "CNAME", "TTL": 300, "ResourceRecords": [{"Value": "'$GATEWAY_ADDR'"}]}},
>
    {"Action": "CREATE", "ResourceRecordSet": {"Name": "invocation.nvcf.example.com", "Type": "CNAME", "TTL": 300, "ResourceRecords": [{"Value": "'$GATEWAY_ADDR'"}]}}
>
  ]
>
}'

Step 3: Update HTTPRoute Hostnames

Update your helmfile environment to use your custom domain instead of the load balancer address:

1
# Use your custom domain
2
domain: "nvcf.example.com"
3
4
# Gateway configuration remains the same
5
ingress:
6
  gatewayApi:
7
    enabled: true
8
    gateways:
9
      shared:
10
        name: nvcf-gateway
11
        namespace: envoy-gateway
12
        listenerName: http
13
      grpc:
14
        name: nvcf-gateway
15
        namespace: envoy-gateway
16
        listenerName: tcp

Redeploy to update the HTTPRoute hostnames:

$
helmfile -e <env> apply

Verify the routes updated:

$
kubectl get httproute -A -o jsonpath='{range .items[*]}{.metadata.name}: {.spec.hostnames[*]}{"\n"}{end}'
$
# Expected:
$
# api-keys: api-keys.nvcf.example.com
$
# nvcf-api: api.nvcf.example.com
$
# invocation-service: *.invocation.nvcf.example.com invocation.nvcf.example.com

Step 4: Configure TLS (HTTPS)

For TLS, you have two main options:

Option A: TLS at the Load Balancer (Recommended for AWS)

Terminate TLS at the AWS NLB using ACM certificates:

1. Request a certificate in AWS Certificate Manager for *.nvcf.example.com
2. Update the Gateway to use HTTPS listeners

1
# Update Gateway listeners for TLS passthrough or termination
2
# This varies by cloud provider - consult your provider's documentation

Option B: TLS at the Gateway with cert-manager

Use cert-manager to automatically provision Let’s Encrypt certificates:

$
# Install cert-manager
$
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.0/cert-manager.yaml
$
$
# Create a ClusterIssuer for Let's Encrypt
>
kubectl apply -f - <<EOF
>
apiVersion: cert-manager.io/v1
>
kind: ClusterIssuer
>
metadata:
>
  name: letsencrypt-prod
>
spec:
>
  acme:
>
    server: https://acme-v02.api.letsencrypt.org/directory
>
    email: your-email@example.com
>
    privateKeySecretRef:
>
      name: letsencrypt-prod
>
    solvers:
>
    - http01:
>
        gatewayHTTPRoute:
>
          parentRefs:
>
          - name: nvcf-gateway
>
            namespace: envoy-gateway
>
EOF

Then update your Gateway to use HTTPS listeners with the certificate secret.

Step 5: Update Client Configuration

With DNS and HTTPS configured, client configurations simplify significantly:

1
# Simple URLs using your domain - no Host header overrides needed!
2
base_http_url: "https://api.nvcf.example.com"
3
invoke_url: "https://invocation.nvcf.example.com"
4
base_grpc_url: "grpc.nvcf.example.com:443"
5
api_keys_service_url: "https://api-keys.nvcf.example.com"
6
7
# No host header overrides required - DNS handles routing
8
# api_keys_host: ""  # Not needed
9
# api_host: ""       # Not needed
10
# invoke_host: ""    # Not needed

Development vs Production Comparison

Troubleshooting

404 Errors

If you receive 404 errors when accessing services:

1. Verify Host header matches HTTPRoute hostname:

   $
   kubectl get httproute api-keys -n envoy-gateway -o jsonpath='{.spec.hostnames}'

2. Confirm the gateway is programmed:

   $
   kubectl get gateway nvcf-gateway -n envoy-gateway -o jsonpath='{.status.conditions}'

3. Check route attachment:

   $
   kubectl describe httproute api-keys -n envoy-gateway | grep -A 5 "Parents"

Routes Not Attaching

If routes show 0 attached in gateway status:

1. Verify namespace labels:

   $
   kubectl get ns -l nvcf/platform=true

2. Check ReferenceGrants exist:

   $
   kubectl get referencegrants -A

3. Review gateway listener configuration:

   $
   kubectl get gateway nvcf-gateway -n envoy-gateway -o yaml | grep -A 20 listeners

gRPC Connection Issues

For gRPC connection problems:

1. Verify port 10081 is exposed:

   $
   kubectl get svc -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-name=nvcf-gateway

2. Test with grpcurl:

   $
   grpcurl -plaintext $GATEWAY_ADDR:10081 list

3. Check TCPRoute status:

   $
   kubectl describe tcproute grpc -n envoy-gateway

Related Documentation

• helmfile-installation - Helmfile values that consume Gateway quickstart outputs
• cli-configuration - CLI configuration including Host header settings
• Kubernetes Gateway API
• Envoy Gateway