Skip to content

Pull requests: vllm-project/vllm

New pull request New
263 Open 1,794 Closed
263 Open 1,794 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[Bugfix] Fix illegal memory access for lora
#5169 opened May 31, 2024 by sfc-gh-zhwang • Draft
1
[Bugfix] Fix KeyError: 1 When Using LoRA adapters
#5164 opened May 31, 2024 by BlackBird-Coding Loading…
1
1
[Kernel] Pass a device pointer into the quantize kernel for the scales
#5159 opened May 31, 2024 by tlrmchlsmth Loading…
4
[Core] Optimize gpu_memory_utilization parameter to be applied on the GPU memory available after peak_memory
#5158 opened May 31, 2024 by alexm-neuralmagic Loading…
9
[Kernel] Add GPU architecture guards to the CUTLASS w8a8 kernels to reduce binary size
#5157 opened May 31, 2024 by tlrmchlsmth • Draft
[Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py
#5151 opened May 31, 2024 by dashanji Loading…
1
[KERNEL] int8 quantization kernel refactoring & optimization WIP
#5146 opened May 31, 2024 by ZelboK Loading…
6
add gptq_marlin test for bug report https://github.com/vllm-project/vllm/issues/5088
#5145 opened May 30, 2024 by alexm-neuralmagic Loading…
[Kernel] Update Cutlass fp8 configs
#5144 opened May 30, 2024 by varun-sundar-rabindranath Loading…
25
[Bugfix] Fix KV head calculation for MPT models when using GQA
#5142 opened May 30, 2024 by bfontain Loading…
1
[CI/Build] Test buildkite monorepo plugin
#5140 opened May 30, 2024 by dgoupil Loading…
1
5
[Frontend]token_ids are useless param sent to the logit_bias_logits_processor.
#5139 opened May 30, 2024 by Etelis Loading…
3
[Core] Remove unnecessary copies in flash attn backend
#5138 opened May 30, 2024 by Yard1 Loading…
2
[Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU
#5137 opened May 30, 2024 by tlrmchlsmth Loading…
8
[Feature][Frontend]: Add support for stream_options in ChatCompletionRequest
#5135 opened May 30, 2024 by Etelis Loading…
1
1
[Speculative Decoding 1/2 ] Add typical acceptance sampling as one of the sampling techniques in the verifier
#5131 opened May 30, 2024 by sroy745 Loading…
1
[Misc] Simplify code and fix type annotations in conftest.py
#5118 opened May 30, 2024 by DarkLight1337 Loading…
@rkooo567
1
[Kernel] Add w4a16 support for compressed_tensors models
#5116 opened May 30, 2024 by dsikka Loading…
3
New CI template on AWS stack
#5110 opened May 29, 2024 by khluu Loading…
1
[Speculative Decoding] Enable arbitrary model inputs
#5101 opened May 29, 2024 by abhigoyal1997 • Draft
1 of 8 tasks
2
[CI/Build] Simplify OpenAI server setup in tests
#5100 opened May 29, 2024 by DarkLight1337 • Draft
[Misc] Add vLLM version getter to utils
#5098 opened May 29, 2024 by DarkLight1337 Loading…
4
New vllm CLI
#5090 opened May 28, 2024 by EthanqX Loading…
1
[Core][CUDA Graph] add output buffer for cudagraph to reduce memory footprint
#5074 opened May 27, 2024 by youkaichao Loading…
5
[CI/Build][Misc] Add scripts that performs a fair comparison between vLLM and alternatives (TGI and TRT)
#5073 opened May 27, 2024 by KuntaiDu Loading…
1 of 4 tasks
6
ProTip! Adding no:label will show everything without a label.