[deps][LLM] Upgrade vLLM to 0.14.0 #60253

nrghosh · 2026-01-17T00:29:19Z

Summary

Upgrade vLLM dependency from 0.13.0 to ~~0.14.0rc1~~ 0.14.0 for testing

Fixes

PoolingParams.normalize → use_activation (python/ray/llm/tests/batch/gpu/stages/test_vllm_engine_stage.py) - vllm#32243
Multi-GPU DP tests switched to MoE models (doc/source/llm/doc_code/serve/multi_gpu/dp_basic_example.py, dp_pd_example.py) - vLLM now makes DP ranks independent for dense models - vllm#30739

Dependency Changes

PyTorch 2.9.1 now required (default wheel compiled against CUDA 12.9)
compressed-tensors ≥0.13.0 for updated quantization support
CUDA 12.9 default (up from 12.4 in 0.13.0)
Protobuf >= 6.33.2 ([grpc] Support gRPC server entrypoint vllm-project/vllm#30190)

Testing

gemini-code-assist

Code Review

This pull request upgrades the vLLM dependency to version 0.14.0rc1. The changes include updating the version in requirements.txt, setup.py, and the Dockerfile. A detailed analysis document is also added, which is a great addition. My review focuses on ensuring the accuracy of this analysis document. I've found a couple of inconsistencies in the analysis document that should be addressed for clarity and correctness. Otherwise, the changes look good.

VLLM_0.14.0_UPGRADE_ANALYSIS.md

aslonnie · 2026-01-17T03:17:43Z

python/requirements/llm/llm-requirements.txt

 # Those pins for the sake of workarounds should not be advertised as constraints
 # on future releases in setup.py.
-vllm[audio]>=0.13.0
+vllm[audio] @ git+https://github.com/vllm-project/[email protected]


why are we upgrading to an rc release?

This is to get ahead of testing - so we can be ready - they haven't released 0.14.0 just yet

nrghosh

Running llm release tests - cpu/gpu llm tests unblocked
main blocker it seems is the protobuf upgrade conflict + vllm 0.14.0 requiring a torch upgrade to torch==2.9.1+cpu

cc @aslonnie @elliot-barn

nrghosh

multi-gpu test regression is fixed (running locally with vllm0.14.0) but is now OOMing on CI https://buildkite.com/ray-project/premerge/builds/58312/steps/table?sid=019be30d-ed6f-4ed6-94c7-6d9c87068347

cc @eicherseiji if we want to request them to be bumped from T4 -> L4 iirc or fix it on the config side

doc/source/llm/doc_code/serve/multi_gpu/dp_pd_example.py

Signed-off-by: Nikhil Ghosh <[email protected]>

Signed-off-by: elliot-barn <[email protected]>

Signed-off-by: Nikhil Ghosh <[email protected]>

…ctivation Signed-off-by: Nikhil Ghosh <[email protected]>

- Use a MoE model (Deepseek-V2-Lite) because vllm-project/vllm#30739 changes how vLLM handles DP ranks - overrides dp_size=1 and dp_rank=0 if non-MoE model - Fixes doc/source/llm/doc_code/serve/multi_gpu/dp_basic_example.py and doc/source/llm/doc_code/serve/multi_gpu/dp_pd_example.py - vLLM 0.14.0 commit bd877162e optimizes DP for dense models by making each rank independent and only preserving DP coordination for MoE models where it's needed for expert - Impact: Ray's DPServer DP coordination (rank assignment, stats addresses) was ignored for dense models like Qwen2.5-0.5B-Instruct, causing cascading assertion failures - Fix: The tests now use an MoE model where vLLM's DP coordination is preserved. Outside of this test, dense model deployments should use Ray Serve replicas (num_replicas) instead of vLLM's data_parallel_size. Signed-off-by: Nikhil Ghosh <[email protected]>

Signed-off-by: Nikhil Ghosh <[email protected]>

duyleekun · 2026-01-29T14:16:22Z

https://github.com/vllm-project/vllm/releases/tag/v0.15.0 released, just saying :)

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale · 2026-01-30T01:26:16Z

ci/raydepsets/pre_hooks/remove-compiled-headers.sh

+# Remove the GPU constraints, numpy pin, and scipy pin (LLM requires numpy>=2 and compatible scipy)
 cp "python/${FILENAME}" "/tmp/ray-deps/${FILENAME}"
-sed -e '/^--extra-index-url /d' -e '/^--find-links /d' "/tmp/ray-deps/${FILENAME}" > "/tmp/ray-deps/${FILENAME}.tmp"
+sed -e '/^--extra-index-url /d' -e '/^--find-links /d' -e '/^numpy==/d' -e '/^scipy==/d' "/tmp/ray-deps/${FILENAME}" > "/tmp/ray-deps/${FILENAME}.tmp"


This is modified by Claude. We'll see if we need this.

jeffreywang-anyscale · 2026-01-30T01:28:08Z

python/requirements/llm/llm-requirements.txt

 # Those pins for the sake of workarounds should not be advertised as constraints
 # on future releases in setup.py.
-vllm[audio]>=0.14.0
+vllm[audio] @ git+https://github.com/vllm-project/[email protected].0


0.15.0 is somehow still unavailable. Will check again later.

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale · 2026-01-30T02:59:43Z

Ran the following locally and everything succeeded. Trying to wrap my head around why premerge fails.

bash ci/ci.sh compile_pip_dependencies
bash ci/compile_llm_requirements.sh
bazel run //ci/raydepsets:raydepsets -- build --all-configs

Signed-off-by: Jeffrey Wang <[email protected]>

…ck locally Signed-off-by: Jeffrey Wang <[email protected]>

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale · 2026-01-31T06:38:15Z

ci/raydepsets/configs/llm_release_tests.depsets.yaml

    - --python-version=3.11
    - --unsafe-package ray
-    - --python-platform=linux
+    # Use manylinux_2_31 for vllm 0.15.0 wheel compatibility


hint: Wheels are available for `vllm` (v0.15.0) on the following platforms: `manylinux_2_31_aarch64`, `manylinux_2_31_x86_64`

linux defaults to manylinux_2_28_x86_64 which vllm 0.15.0 does not support

Signed-off-by: Jeffrey Wang <[email protected]>

duyleekun · 2026-01-31T10:26:10Z

What's current ray policy on vllm version support? Since 0.15 produces a lot of breaking changes and some might want to mix vllm versions between ray apps.

gemini-code-assist bot reviewed Jan 17, 2026

View reviewed changes

VLLM_0.14.0_UPGRADE_ANALYSIS.md Outdated Show resolved Hide resolved

VLLM_0.14.0_UPGRADE_ANALYSIS.md Outdated Show resolved Hide resolved

nrghosh force-pushed the nrghosh/vllm-0.14.0-rc branch from e3d235b to 01d9154 Compare January 17, 2026 00:35

eicherseiji added the go add ONLY when ready to merge, run all tests label Jan 17, 2026

aslonnie reviewed Jan 17, 2026

View reviewed changes

nrghosh force-pushed the nrghosh/vllm-0.14.0-rc branch from 261437a to 8cc3ce8 Compare January 21, 2026 19:53

nrghosh changed the title ~~[LLM] Upgrade vLLM to 0.14.0~~ [deps][LLM] Upgrade vLLM to 0.14.0 Jan 21, 2026

nrghosh force-pushed the nrghosh/vllm-0.14.0-rc branch from cf7f2be to b766902 Compare January 22, 2026 00:11

nrghosh commented Jan 22, 2026

View reviewed changes

eicherseiji reviewed Jan 22, 2026

View reviewed changes

doc/source/llm/doc_code/serve/multi_gpu/dp_pd_example.py Outdated Show resolved Hide resolved

nrghosh and others added 8 commits January 26, 2026 15:58

[LLM] Point vLLM dependency to v0.14.0rc1 for upgrade testing

cd095ee

Signed-off-by: Nikhil Ghosh <[email protected]>

overriding vllm torch requirement

758b2c9

Signed-off-by: elliot-barn <[email protected]>

including torch and vllm

923dabc

Signed-off-by: elliot-barn <[email protected]>

[LLM] Update vLLM from rc1 to 0.14.0 release

fc6b087

Signed-off-by: Nikhil Ghosh <[email protected]>

wip - compile deps

c1c11e7

Signed-off-by: Nikhil Ghosh <[email protected]>

wip - fix test - PoolingParams.normalize deprecated in favor of use_a…

d18c71c

…ctivation Signed-off-by: Nikhil Ghosh <[email protected]>

use smaller MoE model to avoid OOms in CI

3190ad1

Signed-off-by: Nikhil Ghosh <[email protected]>

jeffreywang-anyscale force-pushed the nrghosh/vllm-0.14.0-rc branch from 43094cc to ee57de3 Compare January 26, 2026 23:58

jeffreywang-anyscale force-pushed the nrghosh/vllm-0.14.0-rc branch from ee57de3 to 10801be Compare January 29, 2026 18:18

Upgrade to vLLM 0.15.0 take 2

eca9898

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale force-pushed the nrghosh/vllm-0.14.0-rc branch from 10801be to eca9898 Compare January 30, 2026 01:23

jeffreywang-anyscale reviewed Jan 30, 2026

View reviewed changes

Take 3

24e1d99

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale added 2 commits January 29, 2026 19:08

Remove extra-index-url

231de88

Signed-off-by: Jeffrey Wang <[email protected]>

Attempt to fix CUDA version incompatibility

04eb5d2

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale force-pushed the nrghosh/vllm-0.14.0-rc branch from d706411 to 04eb5d2 Compare January 30, 2026 18:46

jeffreywang-anyscale added 6 commits January 30, 2026 12:17

OpenAI protocol adjustment

4cd9ecc

Signed-off-by: Jeffrey Wang <[email protected]>

linter

0d9957c

Signed-off-by: Jeffrey Wang <[email protected]>

Trim

b749a9f

Signed-off-by: Jeffrey Wang <[email protected]>

Run bazel run //ci/raydepsets:raydepsets -- build --all-configs --che…

9f9958c

…ck locally Signed-off-by: Jeffrey Wang <[email protected]>

Use parse_chat_messages_async

8248ad7

Signed-off-by: Jeffrey Wang <[email protected]>

Clean up

6711055

Signed-off-by: Jeffrey Wang <[email protected]>

jeffreywang-anyscale reviewed Jan 31, 2026

View reviewed changes

Recompile locally

cda414a

Signed-off-by: Jeffrey Wang <[email protected]>

[deps][LLM] Upgrade vLLM to 0.14.0 #60253

Are you sure you want to change the base?

[deps][LLM] Upgrade vLLM to 0.14.0 #60253

Conversation

nrghosh commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Fixes

Dependency Changes

Testing

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

aslonnie Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

nrghosh Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

nrghosh left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nrghosh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

duyleekun commented Jan 29, 2026

Uh oh!

jeffreywang-anyscale Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

jeffreywang-anyscale Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

jeffreywang-anyscale commented Jan 30, 2026

Uh oh!

jeffreywang-anyscale Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

jeffreywang-anyscale Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

duyleekun commented Jan 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

nrghosh commented Jan 17, 2026 •

edited

Loading

nrghosh left a comment •

edited

Loading

jeffreywang-anyscale Jan 31, 2026 •

edited

Loading