bench: connect fleet-sim capacity modeling to routing policy validation

Summary

Make FleetSim the common offline capacity, workload, latency, cost, and energy evaluation layer for proposed routing and downstream LB/control-plane policies.

Parent roadmap: #2287
Related: #1510, #2332, #2333, #2341, #2513, #2550, #2551, #2552, #2554, #2555, #2556, #2557, #2559

Binding architecture boundary

Per #2513/#2514:

• semantic routing remains per-query;
• capacity/queue/spillover/admission/endpoint choice belongs to the serving/LB layer;
• FleetSim sizes and evaluates that layer offline;
• no simulator or capacity solver is added to ext_proc/model selection.

Audited upstream baseline (2026-07-15)

FleetSim already includes:

• fleet, pool, request, workload CDF/trace, and synthetic workload models;
• random, length, semantic, least-loaded, spillover, and compression routing primitives;
• FleetOptimizer and analytical/threshold/Pareto optimization;
• energy analysis in optimizer/tpw.py, power-aware GPU profiles, and optimizer/grid_flex.py.

The gap is a maintained, reproducible bridge from runtime/replay/backend observations to simulation scenarios and from results to reviewable policy recommendations.

Scope

1. Define versioned trace/scenario projection contracts with provenance, time windows, units, censoring, and privacy limits.
2. Reproduce current/static baselines before adding a new policy.
3. Model queue, concurrency, latency, spillover, cache-affinity approximation, capacity transitions, heterogeneous pools, and energy evidence where supported.
4. Keep measured observations, calibrated estimates, and simulator assumptions visibly distinct.
5. Evaluate router outputs and downstream LB policies as separate stages.
6. Produce machine-readable and human-readable reports with objective values, constraints, uncertainty, and regression thresholds.
7. Support WRP scenarios: diurnal shifts, mixed archetypes, bursts/cold starts, stale data, capacity transition, compression, and energy trade-offs.
8. Connect reports to roadmap acceptance gates without making FleetSim mandatory for unrelated unit tests.

Boundaries

• FleetSim does not replace production telemetry or E2E verification.
• Simulator-only APIs do not leak into hot-path router code.
• Do not over-interpret projected/LOW-quality power models as measurements.
• Capacity/energy cannot override authorization, safety, residency, model compatibility, or SLO constraints.
• A result with unmatched cost/SLO or invalid workload leakage is not accepted evidence.

Acceptance criteria

• At least one runtime/replay snapshot can be projected reproducibly into a FleetSim scenario.
• Static/simple baselines and matched cost/SLO comparisons are mandatory.
• Reports include quality/task success when applicable, cost, latency/SLO, load/capacity, and energy with provenance.
• Scenario and policy versions are recorded.
• Infeasible, stale, unsupported, and high-uncertainty inputs fail explicitly.
• Router preference and downstream LB action are evaluated separately.
• Docs/CLI/dashboard show how to reproduce the relevant validation flow.
• Regression fixtures cover diurnal mix, heterogeneous pools, stale observations, capacity changes, and rollback.