vLLM splits multimodal encoding into an independently scaled serving tier

vLLM has expanded encoder/prefill/decode, or EPD, disaggregation so the multimodal encoder can run as a separately deployed tier instead of sharing the language-model serving path. The project says the core capabilities, including MiMo and XPU support, are included in vLLM 0.32.0, while metadata-only support for DeepSeek V4.1 Flash remains on the main branch (vLLM engineering post).

What changed

A multimodal request normally encodes images, video or audio before prefill and decode. vLLM now supports E+PD deployments, where encoding is separate from a combined prefill/decode tier, and E/P/D deployments, where all three stages are independent. Operators can therefore add encoder capacity for media-heavy traffic without adding decode capacity, or choose different hardware for each role (vLLM engineering post).

The key handoff is a metadata-only request. The encoder sends structural details such as image grids, feature lengths and placeholder ranges downstream while embeddings travel through an encoder-connector data path. That avoids making the consumer fetch, decode or preprocess the original media again. vLLM reports support across selected image, video and audio models, with model-specific metadata contracts (vLLM engineering post).

The project also added a dedicated encoder-only runner, optional vision-transformer CUDA Graph capture and startup I/O reductions that skip language-model checkpoint shards where the layout can be classified safely. Cross-node embeddings can move through NIXL-backed CPU transport or Mooncake, while Mooncake Store can reuse outputs across encoder instances after a local cache miss (vLLM engineering post).

The operational consequence

Disaggregation creates an explicit distributed-systems contract. Producers and consumers must agree on tensor shapes and protocol versions; vLLM notes that its NIXL protocol update requires both sides to be upgraded together. A metadata-only consumer also cannot fall back to local encoding without the original media, so failed transfers now fail the affected request rather than taking an unsafe recovery path (vLLM engineering post).

Routing support now spans the vLLM Router, SMG and llm-d. In the Kubernetes-oriented llm-d path, encoder, prefill and decode roles run as separate pod tiers, and text-only requests bypass the encoder. The project recommends starting with E+PD to validate the encoder handoff before adding a separate decode tier (vLLM documentation, llm-d guide).

Read the benchmarks carefully

The post reports substantial but workload-specific gains. In an audio test, metadata-only forwarding raised throughput from 62.21 to 73.13 requests per second and reduced mean latency from 980 to 853 milliseconds, while measured accuracy moved from 75.90% to 75.65%, a difference the authors say remains under investigation. In a DeepSeek V4.1 Flash comparison at concurrency 64–128, EPD produced 6.6–9.5% more throughput per GPU and 44–47% lower P90 time to first token, counting all 12 EPD GPUs. At lower concurrency, however, per-GPU throughput was worse or roughly break-even (vLLM engineering post).

That makes EPD an architectural option, not a universal speed switch. Its value depends on media mix, cache-hit behavior, latency targets and whether independent scaling offsets the cost of extra transport, routing and hardware.