vllm-project/semantic-router

Build Router Learning experience materialization

Open

#2,240 opened on Jun 18, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/corearea/model-selectionarea/observabilityenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Parent: #2238
Depends on: #2239 runtime contract
Related: #2340, #2366, #2367, #2393

Goal

Define and implement versioned Router Learning experience materialization and optional offline seed-pack artifacts without adding synchronous durable reads or a separate public “experience” control plane.

Audited current-main baseline (2026-07-15)

  • routerLearningRuntime owns in-process model experience updated from typed outcomes and bounded runtime telemetry.
  • Router Replay stores route diagnostics and typed outcomes and can be an offline evidence source.
  • Current main has no general durable/distributed experience snapshot loader and no completed public seed-pack import contract.
  • Experience is strategy-owned internal state. There is no supported public experience.enabled, experience.source, router.learning.memory, or method-keyed experience table.
  • Router Memory in pkg/memory is prompt-visible user/conversation memory and is unrelated.

Scope

  1. Define a versioned materialized experience/seed-pack artifact with:
    • strategy/schema version;
    • model, decision/tier, recipe/config, and data-window identity;
    • aggregate evidence, uncertainty, sample count, and provenance;
    • redaction, retention, and compatibility metadata.
  2. Build asynchronous materializers from typed Replay/outcome/eval projections.
  3. Validate model references, candidate scope, strategy version, age/freshness, and config compatibility before use.
  4. Define export first. If runtime import is added, make it explicit, bounded, local-snapshot based, and independently disableable.
  5. Specify startup/reload/rollback behavior and diagnostics for used, missing, stale, incompatible, invalid, and ignored artifacts.
  6. Keep runtime telemetry and imported seed evidence distinguishable so operators can reconstruct the final score.
  7. Design multi-replica/shared-state semantics only with #2243; do not silently treat a local file as distributed truth.

Boundaries

  • No request-time durable Replay/database lookup.
  • No raw prompt, response, tool argument, or tool result is required in the derived artifact by default; existing Replay retention is configured separately.
  • No public legacy adaptation names or per-method config tables.
  • Imported evidence cannot expand the configured candidate set or override protection/hard constraints.
  • Missing or invalid artifacts fall back to documented local/day-zero behavior.

Acceptance criteria

  • Artifact schema, compatibility, provenance, privacy, and retention are documented and tested.
  • Materialization is deterministic from a versioned input snapshot.
  • Runtime import, if implemented, is bounded, atomic, reversible, and never partially visible.
  • Stale/incompatible/invalid evidence cannot silently affect routing.
  • Replay diagnostics distinguish runtime evidence from imported evidence.
  • Restart, reload, rollback, corrupt artifact, and missing artifact tests exist.
  • No synchronous durable storage read appears on the request path.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Contributor guide