vllm-project/semantic-router

feat: add Router Memory diagnostics and fail-open runtime receipts

Open

#2,365 opened on Jul 5, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/agentarea/corearea/observabilityenhancementhelp wantedpriority/P1roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Add production diagnostics and fail-open receipts for Router Memory usage in agent routing.

Parent roadmap: #2287 Parent issue: #2339 Related: #2239, #2240, #2243, #2338 Related code: src/semantic-router/pkg/memory, src/semantic-router/pkg/extproc, src/semantic-router/pkg/routerreplay, src/semantic-router/pkg/observability, src/semantic-router/pkg/sessiontelemetry

Motivation

Router Memory can affect multi-turn and agent routing decisions, but production users need to know whether memory was used, stale, unavailable, ignored, or blocked by policy. The current roadmap has runtime, materialization, and distributed-state tracks; this issue focuses on operator-visible receipts.

Scope

Implement diagnostics for:

  • memory lookup/extraction/rewrite status;
  • memory backend availability and freshness;
  • fail-open/fail-closed reason;
  • interaction with safe exploration/protection policies;
  • replay fields and metrics for memory-influenced routing.

Non-goals

  • Do not add a new memory algorithm here.
  • Do not store raw sensitive memory content in aggregate diagnostics.
  • Do not require synchronous memory lookup as a hard dependency for every request.

Acceptance criteria

  • Replay can explain memory status for each routed request.
  • Metrics distinguish memory used, stale, missing, disabled, and policy-blocked states.
  • Missing memory degrades according to the documented runtime contract.

Contributor guide