[RFC]: Support Wan2.2-I2V-A14B Model in vllm-omni Multimodal Generation Framework

Motivation.

1、 Problem Statement vllm-omni is a high-performance multimodal generation framework that currently supports mainstream text-to-image, speech, and LLM models. However, it does not yet support Wan2.2-I2V-A14B, a large-scale text-to-video model. Wan2.2-I2V-A14B has a unique video generation architecture, including specific video encoder/decoder structures, frame generation logic, and tensor parallel strategies. The current vllm-omni codebase cannot load or run this model natively, preventing users from leveraging vLLM’s high-performance inference optimizations (e.g., PagedAttention, dynamic batching, continuous batching) for this model. 2、Motivation Industry demand: Wan2.2-I2V-A14B is a widely used 14B-parameter text-to-video model for production-grade video generation. Ecosystem expansion: Adding support will extend vllm-omni’s coverage into video generation. Performance improvement: Native inference of Wan2.2-I2V-A14B suffers from high GPU memory usage and low throughput. vllm-omni can drastically improve efficiency.

Proposed Change.

Add a dedicated adaptation layer for Wan2.2-I2V-A14B within vllm-omni’s multimodal architecture. 4.1 Architecture Changes 4.1.1 Model Config Parsing Add Wan22I2VA14BConfig inheriting from MultiModalConfig. Support parsing video-related config: frame size, number of frames, latent dimensions, text encoder hidden size, etc. 4.1.2 Weight Loading Support loading from Hugging Face format and local checkpoints. Adapt tensor parallel sharding for the video decoder and text encoder. 4.1.3 Multimodal Modules Add Wan22I2VTextEncoder for text embedding. Add Wan22I2VVideoDecoder for video latent generation and frame decoding. Integrate into the existing MultiModalPipeline. 4.1.4 Inference Pipeline Support full workflow: text input → text encoding → video latent generation → video frame decoding → output. Optimize memory management for large video tensors.

Feedback Period.

please feedback ASAP。

CC List.

@gcanlin @hsliuustc0106 @Fishermanykx

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.