bench: validate safe exploration and protection policies for agent routing

Summary

Build a benchmark suite for agent routing that validates when Router Learning is allowed to explore, when it must not switch models, and what measurable benefit exploration provides.

Parent roadmap: #2287 Parent learning roadmap: #2238 Related: #2244, #2241, #2243, #2308, #2333

Motivation

Agent sessions introduce routing risks that are not visible in single-turn route correctness. Model switching can break tool loops, lose provider-specific state, damage KV/cache locality, cross privacy or safety boundaries, or increase cost without enough benefit. Router Learning needs explicit protection policies and benchmarks before exploration can be treated as production-safe.

Scope

The benchmark should cover scenarios such as:

• active tool loops where model switching should be pinned or blocked;
• session and conversation continuity;
• provider-specific non-portable state;
• privacy, safety, and domain hard boundaries;
• cache affinity and switch-cost penalties;
• exploration under uncertainty;
• recovery from poor model choice;
• cost/latency/quality tradeoffs for exploration.

Metrics should include:

• route correctness;
• unnecessary switch rate;
• blocked unsafe exploration rate;
• successful exploration benefit;
• cost and latency delta;
• cache or prefix-retention impact;
• replay explainability coverage.

Non-goals

• Do not implement a new learning algorithm in this issue.
• Do not make all model switching illegal; the benchmark should identify when switching is beneficial and safe.
• Do not require synchronous external storage on the hot path.

Acceptance criteria

• A maintained benchmark profile models multi-turn agent sessions.
• The benchmark distinguishes safe exploration, unsafe switching, and missed exploration opportunities.
• Protection policies are evaluated as hard constraints, not soft preferences.
• Reports quantify quality, cost, latency, cache impact, and switching behavior.
• Results can gate Router Learning changes in PR or release validation.

Validation

make agent-report ENV=cpu CHANGED_FILES="bench src/semantic-router/pkg/extproc src/semantic-router/pkg/selection src/semantic-router/pkg/config website/docs"
make agent-lint CHANGED_FILES="bench src/semantic-router/pkg/extproc src/semantic-router/pkg/selection src/semantic-router/pkg/config website/docs"
make agent-ci-gate CHANGED_FILES="bench src/semantic-router/pkg/extproc src/semantic-router/pkg/selection src/semantic-router/pkg/config website/docs"