research: coordinate agent memory tiers, tool surfaces, and scheduler hints at the gateway

Motivation

Agent workloads expose several gateway-level mechanisms—Router Memory, tool filtering, session telemetry, retention directives, token-budget policy, and downstream scheduling hints—but current main does not define one cross-component contract that coordinates them.

This is an integration and evaluation issue, not a claim that every mechanism is already production-complete.

Audited upstream baseline (2026-07-15)

• Router Memory provides prompt-visible user/conversation memory. RouteDiagnostics already expose MemoryStatus, MemoryReason, MemoryFallbackReason, MemoryFailOpen, and MemoryResultCount (#2520).
• Request-side tool retrieval/filtering and response-side tool traces exist.
• Session telemetry provides bounded local aggregates.
• Retention directives exist partially; backend KV eviction/retention support remains an integration gap.
• Inference-aware endpoint routing and trusted caller identity are tracked separately.
• The semantic router remains a per-query logical-model router; pool capacity actuation belongs to the LB/serving/control plane (#2513).

Scope

1. Define a versioned gateway context envelope containing only reviewed fields:
   • trusted caller/session identity;
   • the existing content-free memory RouteDiagnostics/receipt fields (or a versioned reference derived from them), not unrestricted memory contents;
   • authorized/retrieved tool identifiers;
   • token-budget stage/reservation;
   • retention directive;
   • logical model decision and downstream pool/SLO hint.
2. Define which component owns each field and the precedence order: identity/authz and safety, then tool/memory policy, then quality/cost shaping, then downstream scheduling.
3. Propagate the envelope through ext_proc and Router Replay with content-minimized diagnostics.
4. Provide capability negotiation and safe behavior when a backend/LB/gateway does not support a hint.
5. Build an E2E trace showing a request, decision, downstream acknowledgement, response usage, tool outcome, and learning outcome.
6. Evaluate latency, privacy, and failure isolation; keep optional mechanisms independently disableable.

Boundaries

• The router does not execute tools or orchestrate multi-step workflows.
• Pool/capacity solvers do not run in semantic model selection.
• Router Memory is not used for security reputation or Router Learning experience.
• No untrusted request header may assert privileged identity or authorization.
• Missing downstream support must degrade explicitly, not silently claim enforcement.

Acceptance criteria

• Ownership, schema versioning, precedence, and trust boundaries are documented.
• Unsupported capabilities produce explicit no-op/rejection diagnostics.
• The envelope reuses the existing memory receipt fields and does not require raw memory/tool payloads in its telemetry projection.
• Hot-path work and cardinality are bounded.
• End-to-end tests cover partial deployment, retries, streaming, and failure isolation.
• Replay distinguishes recommendation, downstream acknowledgement, execution, and outcome.
• Each dependent capability retains its own rollout/rollback gate.

Likely change surfaces

src/semantic-router/pkg/extproc/, src/semantic-router/pkg/memory/, src/semantic-router/pkg/tools/, src/semantic-router/pkg/sessiontelemetry/, src/semantic-router/pkg/routerreplay/, src/semantic-router/pkg/config/, deployment/gateway contracts, docs, and E2E tests.

Dependencies/related: #2018, #2332, #2349, #2361, #2362, #2547, #2553.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Follow the reported gates and affected E2E profiles.