vLLM v0.33.0rc3 drops CUDA 12.9, adds PyTorch 2.14, new models and speculative decoding gains
vLLM v0.33.0rc3 upgrades to PyTorch 2.14, moves to CUDA 13.0-only artifacts, adds 11 new models, and delivers speculative decoding and decode-speed improvements across NVIDIA and AMD hardware.
vLLM v0.33.0rc3 ships 927 commits from 389 contributors and introduces changes operators need to plan for, starting with the toolchain and artifact shifts.
Toolchain and artifact changes
The release upgrades the PyTorch dependency to 2.14.0 with torchvision 0.29.0 and Triton 3.8.0. CUDA 12.9 wheels and the -cu129 / -cu129-ubuntu2404 Docker images are no longer published; the default CUDA 13.0 artifacts should be used instead. The CUDA wheel is now ABI stable with libtorch. A new vllm/vllm-openai:v0.33.0-efa image (x86_64 and aarch64) bundles AWS EFA libraries and the EFA build of the Mooncake transfer engine for KV transfer and multi-node serving on EFA-enabled EC2 instances. Every release image is also published under the shorter vllm/vllm name, for example vllm/vllm:v0.33.0, vllm/vllm:v0.33.0-efa, and vllm/vllm:v0.33.0-rocm.
On AMD, the default ROCm image and wheels move to ROCm 10.0, with ROCm 7.2 still published as -rocm72.
New model support
Eleven new models receive day-0 support: Reflection Beam-1.0 (501B MoE, 23B active) with its DFlash2 drafter, Yandex AliceAI-Foundation-80B-A3B, Recursive Flint (40B MoE, 2.5B active), Muse Spark Open, Step-5 Preview, NVIDIA Alpamayo 2 Super, Nemotron-Labs Dory1.0-4B, Nemotron Parse 3.5, MiniCPM-OCR 1.0, Domyn Edge, and the ColUltraVBERT visual document retriever. Additional models include LFM2.5-Embedding-350M, Kolibri 1, Decision 3.0 models, and Cloudflare Clef-Flash.
DeepSeek-V4 and V4.1 gain length-aware decode indexer block selection (about 850 µs saved per step at 1M context), MegaMoE experts loaded directly in the DeepGEMM layout, fused wqa_wkv split by token across TP on SM90, native small-head FlashMLA sparse prefill on SM90, and multiple correctness fixes for Engram, PP, and compressed-KV writes. GLM-5.3-Flash defaults to an FP8 KV cache (9.8% higher end-to-end throughput in the PR’s benchmark) and gets mHC TileLang decode kernels on Blackwell. Kimi-K3 receives KDA RecoverSSM CuTe DSL verify/commit kernels on SM10x with verify records moved out of the KV cache so KV connectors work. Qwen3.8-Flash-Next adds FP8 QSA KV cache on Ampere and packed NVFP4 PLE embeddings.
Speculative decoding and decode performance
Speculative decoding adds the XPress refiner for block-diffusion drafters such as DFlash and DSpark, FLy (an opt-in lossy verification policy with 1.04–1.31x decode throughput on MI355X and B300), and removal of draft-slot budget reservation for DFlash/DSpark (taking DeepSeek-V4.1-Flash + DSpark on GB300 from 1373 to 3390 output tok/s in the PR’s benchmark). Quantized draft lm_head (draft_lm_head_quantization) adds about 18% decode at concurrency 1. Lookahead block hashes keep EAGLE/MTP prefix-cache hits on their last block, enabled by default. MTP drafts are now supported in the Transformers modeling backend.
On NVIDIA, opt-in latency-specialized decode on GB300 delivers 1.5–2x for Kimi-K3 at TP8/TP16 and 18.3% faster decode step for DeepSeek-V4.1-Flash at batch 1. DeepSeek-V4/V4.1 MXFP4 experts run as block FP8 through Humming on Hopper (+9–21% decode on 4x H200). FA4 sparse MLA decode lands on Blackwell. FlashInfer DCP prefill on trtllm-gen shows 39–48% lower mean TTFT at 100k input on 8x B200. Decode-only full CUDA graphs under MRV2 prefill context parallelism achieve 2.6x over eager.
On AMD, MonoKernel decode via --monokernel gives 2.32x P50 interactivity at concurrency 1 for DeepSeek-V4.1-Flash on MI300X/MI325X and every MI355X layer, and 16.6% lower TPOT for MiniMax-M3-MXFP4 on MI355X. The ROCM_OCTAVE 3–4 bit KV cache provides 4.6x the KV tokens of fp16 on RX 7900 XTX. gfx1250 enablement covers gpt-oss-120b, DeepSeek-R1, DeepSeek-V4-Flash, and MiniMax-M3.
API, serving, and breaking changes
The OpenAI-compatible /v1/decisions endpoint is on by default for supported generation models. Token embeddings with optional late chunking arrive on /v1/embeddings. output_mode="derender" is added to /inference/v1/generate. A large Rust frontend parity push eliminates HTTP 500 on long conversations and makes unsupported flags fail at startup. Concurrent worker startup with TP > 1 saves about 50 seconds at TP8. A watchdog dumps stack traces from stuck engine and worker processes.
Breaking changes: Medusa and the MLP speculator are removed; the legacy Ray compiled-graph executor is removed; --mamba-cache-mode is deprecated; requests sent while generation is paused now receive HTTP 503; the NIXL connector requires NIXL 1.5 or newer; LM Format Enforcer and --ir-op-priority are deprecated.