vllm-project/semantic-router

research: enforce typed cross-request behavioral commitments

Open

#2,549 opened on Jul 15, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/agentarea/corearea/multi-turnarea/researchenhancementevaluationhelp wantedpriority/P2roadmapsafety

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Motivation

Research a bounded mechanism for enforcing an operator-approved behavioral commitment across requests—for example, “do not invoke external tools for this workflow” or “remain within this data-handling mode.”

Current main has classification/decision policy, response checks, session telemetry, and replay, but it does not have a persistent behavioral-commitment state machine.

Scope

  1. Define a small, typed, operator-approved commitment schema. Free-form model text is never directly executable policy.
  2. Define trusted sources:
    • operator/route policy;
    • authenticated workflow context;
    • a model-proposed commitment only after deterministic parsing and policy approval.
  3. Implement bounded session-local state first with TTL, conflict resolution, monotonicity rules, and explicit reset/termination.
  4. Compile active commitments into existing classification/decision/ext_proc enforcement points.
  5. Record proposal, acceptance/rejection, active version, enforcement decision, and expiry in Router Replay.
  6. Evaluate durable/cross-replica state only after identity, consistency, privacy, and recovery semantics are specified.
  7. Test response streaming and tool execution boundaries where enforcement may require a gateway.

Boundaries

  • Router Memory is prompt-visible user/conversation memory, not the authoritative policy store.
  • Model output cannot grant itself permissions or weaken authorization/safety policy.
  • Hard authorization and safety constraints always dominate commitments.
  • Missing or ambiguous commitment state cannot be silently treated as permission.
  • The semantic router does not become a workflow engine or tool executor.

Acceptance criteria

  • Commitment types and allowed transitions are closed, versioned, and validated.
  • Trusted origin and policy approval are verifiable.
  • TTL, reset, conflict, retry, concurrency, restart, and multi-replica semantics are documented.
  • Enforcement points cover request shaping, model/tool visibility, and the external execution boundary as applicable.
  • Replay can reconstruct the active commitment and exact decision reason.
  • Raw model text is not persisted as policy state by default.
  • E2E tests cover attempted self-escalation, stale state, partial deployment, and safety precedence.

Likely change surfaces

src/semantic-router/pkg/sessiontelemetry/ or a dedicated bounded policy-state package, src/semantic-router/pkg/classification/, src/semantic-router/pkg/config/, src/semantic-router/pkg/extproc/, src/semantic-router/pkg/routerreplay/, gateway integration, docs, and tests.

Depends on trusted identity semantics in #2362. Related: #2357, #2547.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Follow the reported gates and affected E2E profiles.

Contributor guide