vllm-project/vllm-omni

[RFC]: Continuous Quantization Support for NPU

Open

#2,438 opened on Apr 2, 2026

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (1,067 forks)github user discovery
NPUdiffusiongood first issuehelp wantedhigh priority

Repository metrics

Stars
 (4,990 stars)
PR merge metrics
 (PR metrics pending)

Description

Motivation.

  1. support DiT FA online FP8 quantization --- P0: https://github.com/vllm-project/vllm-omni/issues/2236
  2. support DiT MM online mxFP8 quantization ---P1: https://github.com/vllm-project/vllm-omni/issues/2592
  3. support MOE online mxFP8 quantization ---P1

Wan2.2: 1+2 HunyuanVideo1.5: 1+2 HunyuanImage3.0: 1+2+3

Proposed Change.

Please provide the detailed design document of the RFC using the template.

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Contributor guide

Research direction
Analyze the existing quantization infrastructure in the vllm omni project, focusing on FP8 and mxFP8 implementations. Investigate how continuous quantization can be achieved on NPUs for DiT and MOE models. Review the linked issues for more context.
Tech stack
pythonpytorch
Domain
machine learningaibackend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
PythonPyTorchQuantization concepts
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
15