Z.ai's 753B-A40B coding and reasoning model with official vLLM and SGLang launch recipes.
The official vLLM recipe uses tensor parallel size 8 plus GLM-specific tool and reasoning parsers.
Every card opens a setup sequence tailored to this model's runtime and memory floor.
Fits one high-memory node, but needs tensor/expert parallel configuration.
Serverless vLLM for smaller models; dedicated Pods for multi-GPU models Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
A Docker template attached to a matching GPU marketplace offer Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
An On-Demand Cloud instance, or a 1-Click Cluster for larger deployments Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
SageMaker HyperPod with a Hugging Face model source and vLLM/SGLang worker Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
A custom vLLM container in Vertex AI, backed by Artifact Registry Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
Azure Machine Learning online endpoint with a bring-your-own vLLM/SGLang container Open runbook →Fits one high-memory node, but needs tensor/expert parallel configuration.
OCI Generative AI imported models when supported; OKE vLLM otherwise Open runbook →Start at 32,768 tokens, confirm all GPUs are visible, load the official chat template, and send a short request. Increase context and concurrency only after recording free VRAM.
Rank comes from observed OpenRouter ordering. Parameters, context, license, and runtime notes come from the linked model card. The 8×96GB hardware figure is a planning floor with runtime and KV-cache headroom.
Open a terminal in the repository where you want the deployment files,
start claude or codex, then paste this prompt.
It asks the agent to verify sources and stop before it creates billable infrastructure.
Prepare a reproducible deployment project for GLM-5 (zai-org/GLM-5); ask me to choose a provider before writing provider-specific infrastructure. Use this guide as the starting context: https://www.getflops.ai/models/glm-5. Use a planning floor of 8x 96GB GPUs, 1000GB storage, vllm v0.19.0, and an initial context limit of 32768 tokens. Open every linked primary source and flag any mismatch instead of guessing. Create a deployment folder containing README.md, .env.example with no secrets, a pinned start script or infrastructure manifest, and smoke-test.sh. Make the endpoint OpenAI-compatible where the runtime supports it. Run local/static validation, estimate the billable resources, and stop before provisioning paid infrastructure until I approve.Research checked 2026-07-27. Provider inventory and inference-engine support can change faster than this page.