[RFC]: [UX][Metrics] Improve vLLM-Omni Metrics UX

Motivation.

Problem

vLLM-Omni currently gives inconsistent timing visibility. From the optimization of GLM-Image #2834 process, I have very poor experience using metrics/profiler to analyze the performance of the whole pipeline:

With --log-stats, users can see full pipeline timing:

Stage 0 AR:        16.23s
Stage 1 diffusion: 13.96s
Request E2E:       30.19s
Overall wall:      32.99s

Without --log-stats, users may only see diffusion timing:

DiffusionEngine.step total=13.94s

For multi-stage models like GLM-Image, this is misleading because diffusion is only one stage. The full request includes:

input preprocessing -> AR -> ar2diffusion -> diffusion -> output

Also, input preprocessing time is not clearly attributed. In the example above:

Overall wall: 32.99s
Request E2E:  30.19s
Gap:           2.80s

That gap is mostly stage-0 input preprocessing, but users must infer it from logs.

Goals

1. Show consistent per-stage timing for all vLLM-Omni pipelines.
2. Avoid making diffusion-only timing look like full request latency.
3. Explicitly report input preprocessing time.
4. Make online serving and offline benchmarks easier to compare.
5. Keep default logs concise.

Proposed Change.

Proposal

Add a concise per-request timing summary, enabled by default for multi-stage pipelines:

[OmniTiming] req=chatcmpl-a0edd05 total=32.99s preprocess=2.80s engine=30.19s stages=[0:ar=16.23s,1:diffusion=13.96s] transfers=[0->1=0.78ms]

Label diffusion breakdowns as stage-local:

[StageTiming stage=1 diffusion] total=13.94s preprocess=0.01ms exec=13.91s postprocess=31.94ms

Add preprocessing metrics around AsyncOmniEngine._build_add_request_message() / InputProcessor.process_inputs():

input_preprocess_time_ms
build_add_request_message_time_ms

Improve --log-stats tables by showing stage names, not only stage IDs:

[StageRequestStats]
stage_id   stage_name   stage_type    stage_gen_time_ms   num_tokens_out
0          ar           llm           16,227.201          1,281
1          diffusion    diffusion     13,960.581          0

Clarify E2E fields:

request_wall_time_ms      includes preprocessing + engine pipeline
engine_pipeline_time_ms   stage pipeline only
input_preprocess_time_ms  input processor / request setup

Implementation Sketch

In AsyncOmniEngine.add_request():

t0 = time.perf_counter()
msg = self._build_add_request_message(...)
msg["input_preprocess_time_ms"] = (time.perf_counter() - t0) * 1000

Attach this value to OrchestratorRequestState, then include it in the final metrics summary.

Use stage config metadata to label stages:

stage_id
stage_type
engine_args.model_stage
model_arch

For GLM-Image this becomes:

0:ar
1:diffusion

Expected Output

For the current GLM-Image example:

[OmniTiming] req=chatcmpl-a0edd05 total=32.99s preprocess=2.80s engine=30.19s stages=[0:ar=16.23s,1:diffusion=13.96s] transfers=[0->1=0.78ms]

With --log-stats:

[Overall Summary]
request_wall_time_ms        32,985.559
input_preprocess_time_ms     2,796.015
engine_pipeline_time_ms     30,189.544

[StageRequestStats]
0:ar                         16,227.201
1:diffusion                  13,960.581

Benefits

1. Users see the real bottleneck immediately.
2. AR time, diffusion time, and preprocessing time are all visible.
3. Default logs no longer imply diffusion time equals full latency.
4. Offline benchmark results are easier to explain.
5. Existing --log-stats behavior can remain backward-compatible.

Open Questions

1. Should [OmniTiming] be always on for multi-stage pipelines, or behind a flag?
2. Should old names like e2e_total_ms remain, with clearer aliases?
3. Should preprocessing include only InputProcessor.process_inputs() or the full _build_add_request_message() time?

Feedback Period.

No response

CC List.

@gcanlin @yinpeiqi @fake0fan @bjf-frz @princepride @JaredforReal

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.