[Performance]: High VRAM overhead in Diffusion Stage and ineffectiveness of `gpu_memory_utilization`

Proposal to improve performance

No response

Report of performance regression

I am observing unexpectedly high GPU memory usage when launching Qwen-Image (20B) using vllm-omni. Despite setting gpu_memory_utilization to a very low value (0.2), the VRAM consumption remains significantly higher than the model weights alone would suggest.

Model: Qwen-Image (approx. 20B parameters).

Expected Weight Footprint: At bf16, the weights are ~40GB total. With tensor_parallel_size=2, I expect ~20GB per GPU for weights.

Actual Usage: Each GPU is consuming ~40GB, meaning there is an additional 20GB of overhead per GPU that is not being controlled by the utilization parameter.

It has been mentioned in developer discussions that gpu_memory_utilization might not be functional for the diffusion stage yet due to the lack of a profile run. I would like to better understand the breakdown of this memory usage and how to optimize it.

Environment / Command: I am using the following command to launch the service:

python -m vllm_omni.entrypoints.cli.serve serve \
    /path/to/Qwen-Image/ \
    --dtype bfloat16 \
    --load_format safetensors \
    --distributed_executor_backend mp \
    --max_model_len 512 \
    --max_num_seqs 4 \
    --enable_chunked_prefill \
    --max_num_batched_tokens 8192 \
    --enable_prefix_caching \
    --enable_sleep_mode \
    --logprobs_mode processed_logprobs \
    --enforce_eager \
    --gpu_memory_utilization 0.2 \
    --tensor_parallel_size 2 \
    --compilation_config '{"cudagraph_mode":"NONE"}'

And it occupies 45GB of memory on both cards

Wonder what factors determine the final GPU memory usage and how can I reduce it?

Misc discussion on performance

No response

Your current environment (if you think it is necessary)

The output of `python collect_env.py`

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.