Multi-GPU
Lambda runbook · model rank #14

Stand up Nemotron 3 Super 120B-A12B
on Lambda.

Fits one high-memory node, but needs tensor/expert parallel configuration. Straightforward dedicated NVIDIA instances and full multi-GPU nodes.

Open Lambda ↗ Read the runbook
Multi-GPU: Fits one high-memory node, but needs tensor/expert parallel configuration.
Provider walkthrough

Lambda setup, in the order that matters

An On-Demand Cloud instance, or a 1-Click Cluster for larger deployments. Use the provider console for infrastructure and the pinned runtime block for the model server.

  1. 01

    Add an SSH key and choose capacity

    In On-Demand Cloud, launch 2x H100/A100 80GB-class GPUs. For more than one node, use a 1-Click Cluster. Attach at least 200GB of persistent storage if you want cached weights to survive instance replacement.

  2. 02

    Connect to the instance

    Use the console-provided SSH command. Lambda opens only port 22 by default, so keep port 8000 private or add a narrowly scoped firewall rule.

  3. 03

    Start the runtime

    Run the container command below with model ID nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8. Keep the first context limit at 262,144 tokens.

  4. 04

    Tunnel, test, and terminate

    Use an SSH tunnel for the first smoke test. Terminate from the Lambda console or API—shutting down Linux does not stop billing.

Runtime seedvLLM · v0.18.1
docker run --rm --gpus all --ipc=host \
  -p 8000:8000 \
  -e HUGGING_FACE_HUB_TOKEN="$HF_TOKEN" \
  -v "$PWD/.hf-cache:/root/.cache/huggingface" \
  vllm/vllm-openai:v0.18.1 \
  --model nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --served-model-name nemotron-3-super-120b-a12b \
  --enable-expert-parallel

This is a reproducible starting block. Cluster paths still need the model author's distributed recipe and the provider's orchestration layer.

Smoke testOpenAI-compatible request
curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nemotron-3-super-120b-a12b",
    "messages": [{"role": "user", "content": "Reply with: deployment healthy"}],
    "max_tokens": 32
  }'

Run this only after logs show the model loaded. A successful HTTP response is not a latency or quality benchmark.

Before you launch

Quota, license, and storage

  • Confirm 2x H100/A100 80GB-class GPUs is actually available in the selected region.
  • Read the NVIDIA Nemotron Open Model License terms and accept any gated-model conditions.
  • Budget at least 200GB for weights, cache, and container layers.
  • Keep HF_TOKEN in the provider secret store—not in scripts or templates.
Before real traffic

Memory, format, and shutdown

  • Record idle/free VRAM after the model loads and after a representative prompt.
  • Validate the official chat template, reasoning parser, and tool-call parser.
  • Add authentication and TLS in front of port 8000.
  • Verify the provider's stop/delete action actually ends compute billing.
Agent-assisted setup

Hand this runbook to Claude Code or Codex

Open a terminal in the repository where you want the deployment files, start claude or codex, then paste this prompt. It asks the agent to verify sources and stop before it creates billable infrastructure.

Claude Code quickstart ↗ Codex docs ↗
Deployment prompt 
Deploy Nemotron 3 Super 120B-A12B (nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8) on Lambda. Use this guide as the starting context: https://www.getflops.ai/models/nemotron-3-super-120b-a12b/lambda. Use a planning floor of 2x 80GB GPUs, 200GB storage, vllm v0.18.1, and an initial context limit of 262144 tokens. Open every linked primary source and flag any mismatch instead of guessing. Create a deployment folder containing README.md, .env.example with no secrets, a pinned start script or infrastructure manifest, and smoke-test.sh. Make the endpoint OpenAI-compatible where the runtime supports it. Run local/static validation, estimate the billable resources, and stop before provisioning paid infrastructure until I approve.
Guardrails included No secrets in files · verify primary docs · approval before spend
Research trail

The two sources to check first

NVIDIA model card Model size, context, license, chat format, and engine notes ↗ Lambda On-Demand Cloud docs An On-Demand Cloud instance, or a 1-Click Cluster for larger deployments ↗

Checked 2026-07-27. This page separates sourced facts from the planning baseline; verify current runtime support and capacity before spending.

Same model, another cloud

Compare Nemotron 3 Super 120B-A12B elsewhere

RunPod Multi-GPU 2x H100/A100 80GB-class GPUs Vast.ai Multi-GPU 2x H100/A100 80GB-class GPUs AWS Multi-GPU SageMaker/HyperPod GPU capacity with at least 2x H100/A100 80GB-class accelerators Google Cloud Multi-GPU Vertex AI GPU deployment with at least 2x H100/A100 80GB-class accelerators Azure Multi-GPU Azure ML GPU deployment with at least 2x H100/A100 80GB-class accelerators Oracle Cloud Multi-GPU OCI GPU shape with at least 2x H100/A100 80GB-class accelerators