vllm-project/semantic-router
View on GitHubBuild Router Learning experience materialization
Open
#2,240 opened on Jun 18, 2026
area/corearea/model-selectionarea/observabilityenhancementhelp wantedpriority/P2roadmap
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Parent: #2238
Depends on: #2239 runtime contract
Related: #2340, #2366, #2367, #2393
Goal
Define and implement versioned Router Learning experience materialization and optional offline seed-pack artifacts without adding synchronous durable reads or a separate public “experience” control plane.
Audited current-main baseline (2026-07-15)
routerLearningRuntimeowns in-process model experience updated from typed outcomes and bounded runtime telemetry.- Router Replay stores route diagnostics and typed outcomes and can be an offline evidence source.
- Current
mainhas no general durable/distributed experience snapshot loader and no completed public seed-pack import contract. - Experience is strategy-owned internal state. There is no supported public
experience.enabled,experience.source,router.learning.memory, or method-keyed experience table. - Router Memory in
pkg/memoryis prompt-visible user/conversation memory and is unrelated.
Scope
- Define a versioned materialized experience/seed-pack artifact with:
- strategy/schema version;
- model, decision/tier, recipe/config, and data-window identity;
- aggregate evidence, uncertainty, sample count, and provenance;
- redaction, retention, and compatibility metadata.
- Build asynchronous materializers from typed Replay/outcome/eval projections.
- Validate model references, candidate scope, strategy version, age/freshness, and config compatibility before use.
- Define export first. If runtime import is added, make it explicit, bounded, local-snapshot based, and independently disableable.
- Specify startup/reload/rollback behavior and diagnostics for used, missing, stale, incompatible, invalid, and ignored artifacts.
- Keep runtime telemetry and imported seed evidence distinguishable so operators can reconstruct the final score.
- Design multi-replica/shared-state semantics only with #2243; do not silently treat a local file as distributed truth.
Boundaries
- No request-time durable Replay/database lookup.
- No raw prompt, response, tool argument, or tool result is required in the derived artifact by default; existing Replay retention is configured separately.
- No public legacy adaptation names or per-method config tables.
- Imported evidence cannot expand the configured candidate set or override protection/hard constraints.
- Missing or invalid artifacts fall back to documented local/day-zero behavior.
Acceptance criteria
- Artifact schema, compatibility, provenance, privacy, and retention are documented and tested.
- Materialization is deterministic from a versioned input snapshot.
- Runtime import, if implemented, is bounded, atomic, reversible, and never partially visible.
- Stale/incompatible/invalid evidence cannot silently affect routing.
- Replay diagnostics distinguish runtime evidence from imported evidence.
- Restart, reload, rollback, corrupt artifact, and missing artifact tests exist.
- No synchronous durable storage read appears on the request path.
Validation entrypoint
make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"