research: explore streaming semantic routing on partial input

Summary

Explore streaming semantic routing that can evaluate partial input or token streams before a full user message is available, with explicit latency, correctness, and fallback tradeoffs.

Parent roadmap: #2287 Related: #2322, #1082, #2168, #2285

Motivation

Streaming or partial-input routing could reduce end-to-end latency by starting classification earlier, but it changes the router's assumptions. The router currently reasons over complete request payloads. Partial input can be ambiguous, and early route decisions may be wrong.

Scope

Investigate:

• which request/stream surfaces can expose partial input to the router;
• whether early signal extraction is safe and useful;
• confidence thresholds for early routing;
• fallback when later tokens contradict the early route;
• diagnostics for early, revised, or abandoned routing decisions;
• latency savings versus misroute risk.

Non-goals

• Do not require all routing to become streaming.
• Do not break existing non-streaming request behavior.
• Do not make irreversible upstream calls before the route is sufficiently confident.

Acceptance criteria

• A design describes viable implementation options and constraints.
• A benchmark or prototype estimates latency savings and misroute risk.
• The design defines when partial-input routing is disabled or falls back to full-input routing.