Multi-GPU
Google Cloud runbook · model rank #10

Stand up MiMo-V2.5-Pro
on Google Cloud.

Fits one high-memory node, but needs tensor/expert parallel configuration. Teams standardizing model serving in Vertex AI with custom containers and raw prediction.

Open Google Cloud ↗ Read the runbook
Multi-GPU: Fits one high-memory node, but needs tensor/expert parallel configuration.
Provider walkthrough

Google Cloud setup, in the order that matters

A custom vLLM container in Vertex AI, backed by Artifact Registry. Use the provider console for infrastructure and the pinned runtime block for the model server.

  1. 01

    Prepare the project

    Enable Vertex AI, Artifact Registry, Cloud Build, and the required GPU quota in one region. Create a dedicated service account for deployment.

  2. 02

    Build the custom runtime

    Start from the official Google vLLM custom-container sample, pin the sglang image/version shown below, and push it to Artifact Registry.

  3. 03

    Upload and deploy the model

    Register XiaomiMiMo/MiMo-V2.5-Pro, select Vertex AI GPU deployment with at least 8x H200-class accelerators, provide HF_TOKEN through a secret, and start with a 32,768-token limit.

  4. 04

    Use raw prediction

    vLLM's OpenAI protocol does not match Vertex's normal prediction envelope; call the documented raw prediction method and verify a short chat request first.

Runtime seedSGLang · latest
docker run --rm --gpus all --ipc=host \
  -p 8000:8000 \
  -e HUGGING_FACE_HUB_TOKEN="$HF_TOKEN" \
  -v "$PWD/.hf-cache:/root/.cache/huggingface" \
  --entrypoint python3 lmsysorg/sglang:latest \
  -m sglang.launch_server \
  --model-path XiaomiMiMo/MiMo-V2.5-Pro \
  --tp-size 8 \
  --context-length 32768 \
  --trust-remote-code \
  --served-model-name mimo-v2.5-pro \
  --reasoning-parser mimo \
  --tool-call-parser mimo

This is a reproducible starting block. Cluster paths still need the model author's distributed recipe and the provider's orchestration layer.

Smoke testOpenAI-compatible request
curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mimo-v2.5-pro",
    "messages": [{"role": "user", "content": "Reply with: deployment healthy"}],
    "max_tokens": 32
  }'

Run this only after logs show the model loaded. A successful HTTP response is not a latency or quality benchmark.

Before you launch

Quota, license, and storage

  • Confirm Vertex AI GPU deployment with at least 8x H200-class accelerators is actually available in the selected region.
  • Read the MIT terms and accept any gated-model conditions.
  • Budget at least 1400GB for weights, cache, and container layers.
  • Keep HF_TOKEN in the provider secret store—not in scripts or templates.
Before real traffic

Memory, format, and shutdown

  • Record idle/free VRAM after the model loads and after a representative prompt.
  • Validate the official chat template, reasoning parser, and tool-call parser.
  • Add authentication and TLS in front of port 8000.
  • Verify the provider's stop/delete action actually ends compute billing.
Agent-assisted setup

Hand this runbook to Claude Code or Codex

Open a terminal in the repository where you want the deployment files, start claude or codex, then paste this prompt. It asks the agent to verify sources and stop before it creates billable infrastructure.

Claude Code quickstart ↗ Codex docs ↗
Deployment prompt 
Deploy MiMo-V2.5-Pro (XiaomiMiMo/MiMo-V2.5-Pro) on Google Cloud. Use this guide as the starting context: https://www.getflops.ai/models/mimo-v2.5-pro/gcp. Use a planning floor of 8x 141GB GPUs, 1400GB storage, sglang latest, and an initial context limit of 32768 tokens. Open every linked primary source and flag any mismatch instead of guessing. Create a deployment folder containing README.md, .env.example with no secrets, a pinned start script or infrastructure manifest, and smoke-test.sh. Make the endpoint OpenAI-compatible where the runtime supports it. Run local/static validation, estimate the billable resources, and stop before provisioning paid infrastructure until I approve.
Guardrails included No secrets in files · verify primary docs · approval before spend
Research trail

The two sources to check first

Xiaomi model card Model size, context, license, chat format, and engine notes ↗ Google custom vLLM deployment docs A custom vLLM container in Vertex AI, backed by Artifact Registry ↗

Checked 2026-07-27. This page separates sourced facts from the planning baseline; verify current runtime support and capacity before spending.

Same model, another cloud

Compare MiMo-V2.5-Pro elsewhere

RunPod Multi-GPU 8x H200-class GPUs Vast.ai Multi-GPU 8x H200-class GPUs Lambda Multi-GPU 8x H200-class GPUs AWS Multi-GPU SageMaker/HyperPod GPU capacity with at least 8x H200-class accelerators Azure Multi-GPU Azure ML GPU deployment with at least 8x H200-class accelerators Oracle Cloud Multi-GPU OCI GPU shape with at least 8x H200-class accelerators