[Followup] Deploy YAML field ownership: stage-level defaults, user knobs, and model-owned config

Context

The deploy config migration exposed a class of ambiguity around where deploy YAML fields should live and who owns their values.

In the compatibility PR, we preserve old Omni behavior by explicitly setting only the fields whose old Omni defaults differ from vLLM defaults / resolved defaults:

Field	Old Omni default	vLLM default / resolved default	Compatibility action
gpu_memory_utilization	0.9	0.9	no explicit override needed
tensor_parallel_size	1	1	no explicit override needed
enforce_eager	False	False	no explicit override needed
data_parallel_size	1	1	no explicit override needed
pipeline_parallel_size	1	1	no explicit override needed
trust_remote_code	True	False	preserve explicitly where needed
enable_prefix_caching	False	None, then resolved from model support; decoder models often become True	preserve explicitly
max_num_batched_tokens	32768	inferred by vLLM, commonly 2048 / 8192 / 16384	preserve explicitly if old behavior depended on it
max_num_seqs	old deploy schema: 64; legacy path also had setdefault(..., 1)	vLLM commonly resolves to 256 / 1024	needs per-path audit before deciding the intended value

That compatibility pass is intentionally conservative. It avoids silent behavior changes, but it does not answer the larger design question:

• Which fields belong at deploy top level?
• Which fields belong inside each stage?
• Which fields are user-tunable runtime knobs?
• Which fields are model-owned requirements that should move into pipeline.py or model-specific defaults?

This issue tracks that follow-up cleanup.

Recommendation

Keep the current PR focused on compatibility only.

For follow-up, classify deploy fields by ownership and precedence before further YAML churn. The migration should be done in small, model-aware steps rather than by mechanically preserving historical defaults forever.

Field Ownership Proposal

1. Stage-owned engine fields

These should usually live inside each stages: entry because different stages may have different runtime constraints:

• max_num_seqs
• max_num_batched_tokens
• gpu_memory_utilization
• enforce_eager
• tensor_parallel_size
• devices
• max_model_len
• skip_mm_profiling
• disable_hybrid_kv_cache_manager

Even when a value is shared today, stage-level placement avoids ambiguity once pipelines become multi-stage or platform overrides diverge.

2. Pipeline-wide fields

These are valid top-level candidates only if they are truly uniform across all stages:

• async_chunk
• dtype
• distributed_executor_backend
• data_parallel_size
• pipeline_parallel_size
• connector definitions / edge topology

We need to document whether top-level values are copied into every stage, whether stage-level values override them, and how this interacts with base_config.

3. Model-owned defaults

Some values are not really user knobs. They exist because a specific model or stage requires them to avoid incorrect behavior or startup failure.

Examples:

• enable_prefix_caching: false for current Omni multimodal/generation paths that are not prefix-cache-safe
• skip_mm_profiling: true for models whose open-source checkpoint cannot run dummy multimodal profiling
• disable_hybrid_kv_cache_manager: true for model/stage-specific KV behavior
• model-specific enforce_eager requirements when CUDA graph capture is unsupported

These may be better owned by model pipeline.py, model config helpers, or per-model default builders instead of being repeatedly written in user-facing YAML.

4. User-tunable runtime knobs

These should remain visible in deploy YAML because users may reasonably adjust them per environment:

• devices
• gpu_memory_utilization
• max_num_seqs
• max_num_batched_tokens
• connector memory / transport settings
• platform overrides

The follow-up should distinguish these from model correctness requirements.

Top-Level vs Stage-Level Compatibility

A concrete ambiguity raised during review is whether fields such as trust_remote_code and enable_prefix_caching should be supported at the deploy top level, stage level, or both.

We need to define:

1. Whether top-level fields are allowed for these engine args.
2. Whether stage-level values override top-level values.
3. Whether inherited base_config files merge top-level and stage-level fields consistently.
4. Whether compatibility should warn when the same field appears in both places.
5. Whether new deploy YAMLs should prefer stage-level placement for all engine args.

Until this is documented, compatibility fixes should prefer explicit stage-level values for stage engine args.

Concrete Follow-Up Plan

1. Document field ownership in vllm_omni/config/stage_config.py or deploy config docs.
2. Add tests for top-level vs stage-level precedence.
3. Add tests for base_config inheritance with stage-level overrides.
4. Audit current deploy YAMLs model by model:
   • Which explicit values are preserving old defaults?
   • Which values are model-required?
   • Which values are user-tunable?
   • Which values can be removed because they match vLLM defaults?
5. Move model-required defaults into model pipeline/default config where appropriate.
6. Keep deploy YAMLs focused on user-facing deployment choices.

What This Issue Is Not

This should not block the current compatibility PR.

The current PR should only prevent behavior regressions from the config migration. This issue tracks the follow-up design cleanup so we do not mix behavior preservation with policy changes.

Related

• #3162 — original deploy config migration / whitelist optimization PR
• #3290 — compatibility PR that preserves old deploy runtime behavior after the revert
• #2887 — broader follow-up tracker for the config refactor