NVIDIA's 120B-A12B reasoning and agent model with official FP8 weights and a documented two-H100 minimum.
Use vLLM 0.18.1 or newer and keep the initial context at 256K or lower before enabling the 1M override.
Every card opens a setup sequence tailored to this model's runtime and memory floor.
Fits one high-memory node, but needs tensor/expert parallel configuration.
Serverless vLLM for smaller models; dedicated Pods for multi-GPU models Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
A Docker template attached to a matching GPU marketplace offer Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
An On-Demand Cloud instance, or a 1-Click Cluster for larger deployments Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
SageMaker HyperPod with a Hugging Face model source and vLLM/SGLang worker Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
A custom vLLM container in Vertex AI, backed by Artifact Registry Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
Azure Machine Learning online endpoint with a bring-your-own vLLM/SGLang container Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
OCI Generative AI imported models when supported; OKE vLLM otherwise Open runbook →Start at 262,144 tokens, confirm all GPUs are visible, load the official chat template, and send a short request. Increase context and concurrency only after recording free VRAM.
Rank comes from observed OpenRouter ordering. Parameters, context, license, and runtime notes come from the linked model card. The 2×80GB hardware figure is a planning floor with runtime and KV-cache headroom.
Open a terminal in the repository where you want the deployment files,
start claude or codex, then paste this prompt.
It asks the agent to verify sources and stop before it creates billable infrastructure.
Prepare a reproducible deployment project for Nemotron 3 Super 120B-A12B (nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8); ask me to choose a provider before writing provider-specific infrastructure. Use this guide as the starting context: https://www.getflops.ai/models/nemotron-3-super-120b-a12b. Use a planning floor of 2x 80GB GPUs, 200GB storage, vllm v0.18.1, and an initial context limit of 262144 tokens. Open every linked primary source and flag any mismatch instead of guessing. Create a deployment folder containing README.md, .env.example with no secrets, a pinned start script or infrastructure manifest, and smoke-test.sh. Make the endpoint OpenAI-compatible where the runtime supports it. Run local/static validation, estimate the billable resources, and stop before provisioning paid infrastructure until I approve.Research checked 2026-07-27. Provider inventory and inference-engine support can change faster than this page.