Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 11 18 #286

Merged
merged 105 commits into from
Nov 19, 2024
Merged

Upstream merge 24 11 18 #286

merged 105 commits into from
Nov 19, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Nov 19, 2024

No description provided.

HarryWu99 and others added 30 commits November 12, 2024 00:17
@HarryWu99 @robertgshaw2-redhat @DarkLight1337
[Metrics] add more metrics (vllm-project#4464)
874f551 
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@yyccli
[Doc] fix doc string typo in block_manager swap_out function (vllm-…
36fc439 
…project#10212)
@youkaichao
[core][distributed] add stateless process group (vllm-project#10216)
e6de978 
Signed-off-by: youkaichao <[email protected]>
@dependabot
Bump actions/setup-python from 5.2.0 to 5.3.0 (vllm-project#10209)
25144ce 
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@WoosukKwon
[V1] Fix detokenizer ports (vllm-project#10224)
f9dadfb 
Signed-off-by: Woosuk Kwon <[email protected]>
@WoosukKwon
[V1] Do not use inductor for piecewise CUDA graphs (vllm-project#10225)
d7a4f22 
Signed-off-by: Woosuk Kwon <[email protected]>
@youkaichao @WoosukKwon
[v1][torch.compile] support managing cudagraph buffer (vllm-project#1…
330e82d 
…0203)

Signed-off-by: youkaichao <[email protected]>
Co-authored-by: Woosuk Kwon <[email protected]>
@WoosukKwon
[V1] Use custom ops for piecewise CUDA graphs (vllm-project#10227)
fe15729 
Signed-off-by: Woosuk Kwon <[email protected]>
@terrytangyuan @russellb
Add docs on serving with Llama Stack (vllm-project#10183)
4800339 
Signed-off-by: Yuan Tang <[email protected]>
Co-authored-by: Russell Bryant <[email protected]>
@youkaichao
[misc][distributed] auto port selection and disable tests (vllm-proje…
8a7fe47 
…ct#10226)

Signed-off-by: youkaichao <[email protected]>
@WoosukKwon
[V1] Enable custom ops with piecewise CUDA graphs (vllm-project#10228)
9d5b4e4 
Signed-off-by: Woosuk Kwon <[email protected]>
@shcheglovnd
Make shutil rename in python_only_dev (vllm-project#10233)
08f93e7 
Signed-off-by: shcheglovnd <[email protected]>
@robertgshaw2-redhat @njhill @tlrmchlsmth
[V1] AsyncLLM Implementation (vllm-project#9826)
6ace6fb 
Signed-off-by: Nick Hill <[email protected]>
Signed-off-by: [email protected] <[email protected]>
Signed-off-by: Nick Hill <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@youkaichao
[doc] update debugging guide (vllm-project#10236)
d1c6799 
Signed-off-by: youkaichao <[email protected]>
@russellb
[Doc] Update help text for --distributed-executor-backend (vllm-pro…
9cdba96 
…ject#10231)

Signed-off-by: Russell Bryant <[email protected]>
@youkaichao
[1/N] torch.compile user interface design (vllm-project#10237)
eea55cc 
Signed-off-by: youkaichao <[email protected]>
@jeejeelee
[Misc][LoRA] Replace hardcoded cuda device with configurable argument (…
7f5edb5 
…vllm-project#10223)

Signed-off-by: Jee Jee Li <[email protected]>
@maleksan85
Splitting attention kernel file (vllm-project#10091)
812c981 
Signed-off-by: maleksan85 <[email protected]>
Co-authored-by: Aleksandr Malyshev <[email protected]>
@youkaichao
[doc] explain the class hierarchy in vLLM (vllm-project#10240)
3a28f18 
Signed-off-by: youkaichao <[email protected]>
@zhouyuan
[CI][CPU]refactor CPU tests to allow to bind with different cores (vl…
d201d41 
…lm-project#10222)

Signed-off-by: Yuan Zhou <[email protected]>
@gcalmettes
[BugFix] Do not raise a ValueError when tool_choice is set to the…
36c513a 
… supported `none` option and `tools` are not defined. (vllm-project#10000)

Signed-off-by: Guillaume Calmettes <[email protected]>
@jeejeelee
[Misc]Fix Idefics3Model argument (vllm-project#10255)
a838ba7 
Signed-off-by: Jee Jee Li <[email protected]>
@DamonFool
[Bugfix] Fix QwenModel argument (vllm-project#10262)
176fcb1 
Signed-off-by: Jie Fu <[email protected]>
@zifeitong
[Frontend] Add per-request number of cached token stats (vllm-project…
47db6ec 
…#10174)
@WoosukKwon
[V1] Use pickle for serializing EngineCoreRequest & Add multimodal in…
7c65527 
…puts to EngineCoreRequest (vllm-project#10245)

Signed-off-by: Woosuk Kwon <[email protected]>
@sroy745
[Encoder Decoder] Update Mllama to run with both FlashAttention and X…
b41fb9d 
…Formers (vllm-project#9982)

Signed-off-by: Sourashis Roy <[email protected]>
@followumesh
[LoRA] Adds support for bias in LoRA (vllm-project#5733)
8a06428 
Signed-off-by: Umesh Deshpande <[email protected]>
Co-authored-by: Umesh Deshpande <[email protected]>
@WoosukKwon
[V1] Enable Inductor when using piecewise CUDA graphs (vllm-project#1…
1f55e05 
…0268)

Signed-off-by: Woosuk Kwon <[email protected]>
@youkaichao
[doc] fix location of runllm widget (vllm-project#10266)
96ae0ea 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[doc] improve debugging doc (vllm-project#10270)
1808145 
Signed-off-by: youkaichao <[email protected]>
youkaichao and others added 25 commits November 16, 2024 01:23
@youkaichao
[misc][plugin] improve log messages (vllm-project#10386)
2f427c2 
Signed-off-by: youkaichao <[email protected]>
@rasmith
[BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (vllm-pro…
1d75472 
…ject#10385)

Signed-off-by: Randall Smith <[email protected]>
@kakao-steve-ai
[Misc] Update benchmark to support image_url file or http (vllm-proje…
8b6725b 
…ct#10287)

Signed-off-by: rbbang <[email protected]>
@skylee-01
[Misc] Medusa supports custom bias (vllm-project#10361)
b98d89e
@imkero
[Bugfix] Fix M-RoPE position calculation when chunked prefill is enab…
361c29e 
…led (vllm-project#10388)

Signed-off-by: imkero <[email protected]>
@WoosukKwon
[V1] Add code owners for V1 (vllm-project#10397)
661a34f 
Signed-off-by: Woosuk Kwon <[email protected]>
@youkaichao
[2/N][torch.compile] make compilation cfg part of vllm cfg (vllm-proj…
4fd9375 
…ect#10383)

Signed-off-by: youkaichao <[email protected]>
@ywang96
[V1] Refactor model executable interface for all text-only language m…
643ecf7 
…odels (vllm-project#10374)

Signed-off-by: Roger Wang <[email protected]>
@xuechendi
[CI/Build] Fix IDC hpu [Device not found] issue (vllm-project#10384)
905d0f0 
Signed-off-by: Chendi Xue <[email protected]>
@Isotr0py
[Bugfix][CPU] Fix CPU embedding runner with tensor parallel (vllm-pro…
cf349c4 
…ject#10394)

Signed-off-by: Isotr0py <[email protected]>
@youkaichao
[platforms] refactor cpu code (vllm-project#10402)
8d74b5a 
Signed-off-by: youkaichao <[email protected]>
@jikunshang
[Hardware] [HPU]add mark_step for hpu (vllm-project#10239)
76aab90 
Signed-off-by: Kunshang Ji <[email protected]>
@imkero
[Bugfix] Fix mrope_position_delta in non-last prefill chunk (vllm-pro…
80d85c5 
…ject#10403)

Signed-off-by: imkero <[email protected]>
@wchen61
[Misc] Enhance offline_inference to support user-configurable paramet… (
d1557e6 
vllm-project#10392)

Signed-off-by: wchen61 <[email protected]>
@Isotr0py
[Misc] Add uninitialized params tracking for AutoWeightsLoader (vll…
c4e4643 
…m-project#10327)

Signed-off-by: Isotr0py <[email protected]>
@HollowMan6
[Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (…
47826ca 
…vllm-project#10375)

Signed-off-by: Hollow Man <[email protected]>
@youkaichao
[4/N][torch.compile] clean up set_torch_compile_backend (vllm-project…
51bb12d 
…#10401)

Signed-off-by: youkaichao <[email protected]>
@lk-chen
[VLM] Report multi_modal_placeholders in output (vllm-project#10407)
c7dec92 
Signed-off-by: Linkun Chen <[email protected]>
@Maybewuss
[Model] Remove redundant softmax when using PoolingType.STEP (vllm-pr…
01aae1c 
…oject#10415)
@B-201
[Model][LoRA]LoRA support added for glm-4v (vllm-project#10418)
5be4e52 
Signed-off-by: B-201 <[email protected]>
@Isotr0py
[Model] Remove transformers attention porting in VITs (vllm-project#1…
e7ebb66 
…0414)

Signed-off-by: Isotr0py <[email protected]>
@B-201
[Doc] Update doc for LoRA support in GLM-4V (vllm-project#10425)
4186be8 
Signed-off-by: B-201 <[email protected]>
@youkaichao
[5/N][torch.compile] torch.jit.script --> torch.compile (vllm-project…
7851b45 
…#10406)

Signed-off-by: youkaichao <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main' into develop
9540837
@gshtras
Vision model ROCm fix
bd1cc77 
Formatting
@gshtras gshtras force-pushed the upstream_merge_24_11_18 branch from d889855 to bd1cc77 Compare November 19, 2024 16:12
@gshtras gshtras merged commit 62334b5 into develop Nov 19, 2024
8 of 9 checks passed
@gshtras gshtras deleted the upstream_merge_24_11_18 branch November 19, 2024 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.