vllm-project/semantic-router

feat: explore tool- and MCP-aware routing contracts

Open

#2,361 opened on Jul 5, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/agentarea/researcharea/tool-managementenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Define router and gateway contracts for tool- and MCP-aware routing while keeping authorization, capability filtering, learned ranking, request mutation, external execution, and workflow orchestration as separate seams.

Parent roadmap: #2287
Related: #1138, #1220, #2330, #2331, #2338, #2339, #2545, #2546, #2547

Audited upstream baseline (2026-07-15)

  • pkg/tools provides request-side tool retrieval/filtering.
  • MCP configuration/dashboard/provider surfaces exist.
  • ext_proc can see the offered tool catalog and response-side emitted tool calls/traces.
  • Router Replay has bounded tool trace/outcome fields.
  • The semantic router does not execute tools and cannot alone guarantee pre-execution RBAC at an external gateway.

Scope

  1. Define typed tool/MCP capability and availability signals with source, version, freshness, and trust.
  2. Filter the request tool catalog by hard authorization/capability policy before relevance or learned ranking.
  3. Express tool-required/tool-forbidden/model-capability constraints in validated decision policy.
  4. Keep prompt compression and Router Memory effects explicit and independently disableable.
  5. Define a gateway feedback contract for emitted, authorized, executed, and completed/failed tool calls.
  6. Record content-minimized replay events and deterministic reasons.
  7. Specify fail-open/fail-closed behavior by dependency/risk class.
  8. Provide at least one maintained recipe/E2E path for tool-aware routing and partial deployment.

Focused children

  • #2545 — learn tool/model recommendations from normalized trusted outcomes
  • #2546 — coordinate memory/tool/budget/retention/scheduler context at the gateway
  • #2547 — filter denied tools in the router and re-authorize at the execution gateway

Boundaries

  • Authorization precedes relevance, learning, cost, and compression.
  • The router does not become a tool executor or multi-step workflow engine.
  • Response-side observation is not a substitute for pre-execution gateway enforcement.
  • New tool-policy/learning/audit projections must omit raw prompts, tool arguments, and tool results by default. Existing Router Replay can retain bounded raw tool/request/response fields when configured; its retention/redaction contract remains separate.
  • Missing MCP/tool availability cannot silently widen privileges.
  • Router Memory is not the tool-policy or learning-experience store.

Acceptance criteria

  • Tool capability, authorization, relevance, selection, emission, execution, and outcome are distinct typed states.
  • Trusted identity and execution-gateway ownership are explicit.
  • At least one maintained recipe demonstrates capability-aware/tool-aware routing.
  • Replay-derived policy diagnostics explain constraints/fallbacks without copying sensitive arguments; tests also cover configured raw Replay retention/redaction.
  • Streaming, stale availability, policy reload, partial deployment, and gateway failure are tested.
  • Non-agent/non-MCP routing remains unaffected when disabled.

Likely change surfaces

src/semantic-router/pkg/tools/, src/semantic-router/pkg/mcp/, src/semantic-router/pkg/config/, src/semantic-router/pkg/classification/, src/semantic-router/pkg/extproc/, src/semantic-router/pkg/routerreplay/, dashboard/gateway integrations, docs, and E2E tests.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Follow the reported gates and affected E2E profiles.

Contributor guide