Hardware Pluginhelp wantedroadmap
Repository metrics
- Stars
- (4,990 stars)
- PR merge metrics
- (PR metrics pending)
Description
Motivation.
Model coverage:
- Toward >80% of all models at https://docs.vllm.ai/projects/vllm-omni/en/latest/models/supported_models/
- exclude only because:
- memory usage can't be held with 8 * Intel ARC Pro B70
- 3rd modeling with CUDA hard-code
Features to enable on Intel GPU(XPU)
- Sequence Parallel
- HSDP / FSDP
- torch compile - https://github.com/vllm-project/vllm-omni/pull/3113
- XPU Graph
- sleep mode - pt2.11 - https://github.com/vllm-project/vllm-omni/issues/2545
- lora support
- Large Scale Deployment - follow https://github.com/vllm-project/vllm-omni/issues/2136 (3) Large Scale Deployment (Disaggregated Serving))
- EPLB (TBD)
- Prefix Cache (TBD)
Quantization
- https://github.com/vllm-project/vllm-omni/issues/1325
- fp8 KV cache support - https://github.com/vllm-project/vllm-omni/pull/1413
Kernel list
- Sparse ATTN - https://github.com/vllm-project/vllm-omni/issues/2233
Performance optimization models
- Wan 2.2 - https://github.com/vllm-project/vllm-omni/issues/1355
- I2V
- T2V
- S2V - https://github.com/vllm-project/vllm-omni/pull/2751
- Flux-dev (T2I)
Memory Management
- expand layerwise offloading model coverage
- multi stage weight sharing (Hunyuan as example)
UX
- Streaming input/output
- All supported models with xpu validated stage_config / document
misc:
- vllm omni xpu - model triage Agent skill
Proposed Change.
Please provide the detailed design document of the RFC using the template.
Feedback Period.
No response
CC List.
@hsliuustc0106 @Gaohan123
cc: @pi314ever @Joshna-Medisetty @yiliu30 @yma11
Any Other Things.
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.