vllm-project/semantic-router

api: bound and type external RAG request templates and responses

Open

#2,478 opened on Jul 12, 2026

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Go (699 forks)github user discovery
area/corearea/networkingbughelp wantedpriority/P2roadmapsafety

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Make external RAG request construction typed and enforce an exact maximum successful-response size. Owner surface: src/semantic-router/pkg/extproc/req_filter_rag_external.go.

Current behavior

Configured request bodies use direct string replacement for user values, so ordinary escaping characters can invalidate or alter the outbound JSON structure. Successful responses are decoded without a byte limit even though error text and transport time are bounded.

Expected behavior

User values cannot change the configured request structure, final requests are valid typed JSON, and successful response decoding stops at a configured hard byte limit with an explicit error.

Acceptance

  • Prefer a typed request builder; if templates remain, replace typed JSON nodes and validate the final document.
  • Marshal every user substitution rather than performing raw string replacement.
  • Enforce a configurable exact maximum before successful-response decoding.
  • Return a clear limit error without accepting a truncated prefix.

Validation

  • Add focused tests for quotes, backslashes, line breaks, Unicode, marker substrings, invalid templates, and field-boundary invariance.
  • Add exact-limit success, one-byte-over rejection, chunked response, and cancellation tests.
  • Run cd src/semantic-router && go test -race ./pkg/extproc.
  • Run the affected external RAG E2E profile.

Related

  • Parent audit: #2375
  • This is a P2 API-boundary follow-up and is separate from dashboard outbound-fetch policy.

Contributor guide