research: forecast serving demand for downstream admission and scaling

Summary

Evaluate workload forecasting for downstream serving admission, scaling, and fleet planning. Forecasts may be exported as bounded hints after semantic model selection; they do not move live queue, capacity, or endpoint selection into the semantic router.

Audited current-main baseline (2026-07-15)

• The semantic router selects a logical model per request. Architecture decision #2513/#2514 keeps capacity, queue, spillover, pool, and endpoint decisions in the serving/LB/control-plane layer.
• FleetSim already provides offline workload, queueing, fleet, and optimizer models.
• Router Replay, latency/in-flight telemetry, and session aggregates provide possible observation inputs, but current main has no production forecasting/actuation contract.
• #2551 tracks pre-dispatch token-demand estimates, #2554 workload-archetype forecasting, #2332/#2349 backend observation contracts, and #2359 FleetSim policy validation.

Research scope

1. Define forecast targets separately: arrival rate, prompt/output demand distribution, SLO class, and pool/model eligibility.
2. Establish simple baselines such as seasonal moving averages and recent-window quantiles before more complex models.
3. Use privacy-safe, versioned aggregates with event time, freshness, missing-data, and late-arrival semantics.
4. Evaluate forecasts in FleetSim against reactive scaling/admission baselines at matched SLO and cost.
5. Export forecast, horizon, interval, provenance, and expiry to the downstream owner through a capability-negotiated contract.
6. Begin with offline evaluation and advise-only/shadow use; downstream actuation owns cooldown, rate limits, acknowledgement, rollback, and safety constraints.

Boundaries

• Do not put a forecasting model or capacity solver synchronously in pkg/extproc model selection.
• A forecast cannot authorize a model, region, or endpoint that violates safety, identity, residency, context, or health constraints.
• Missing, stale, or uncertain forecasts fall back to the downstream layer's ordinary safe policy.
• Report cases where forecasting provides no material benefit.

Acceptance criteria

• Targets, horizons, event-time semantics, and privacy/retention rules are explicit.
• Simple baselines and uncertainty/calibration are reported.
• FleetSim evaluation covers bursts, drift, delayed telemetry, and forecast error.
• Router recommendation, downstream acknowledgement/action, and measured outcome remain separate events.
• No semantic-routing architecture boundary from #2513 is bypassed.
• Any production actuation is a separately owned, reversible milestone.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Use ENV=amd when the change includes AMD runtime/platform behavior.