vllm-project/semantic-router
View on GitHubfeat: design hierarchical routing and recipe chaining without becoming a workflow engine
Open
#2,343 opened on Jul 5, 2026
area/corearea/model-selectionarea/researchenhancementhelp wantedpriority/P2roadmap
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Summary
Design hierarchical routing and recipe chaining so one routing decision can delegate to another bounded routing policy, without turning vLLM Semantic Router into a general workflow engine.
Parent roadmap: #2287 Related: #1815, #2322, #2331, #2333
Motivation
Users have asked for hierarchical and multi-step routing: a top-level router picks a domain or agent, then a secondary router makes a more granular decision. This is related to, but different from, fixed multi-step model pipelines.
Scope
Define a bounded router-native model for:
- nested route decisions;
- recipe-to-recipe delegation;
- sub-router candidate constraints;
- recursion and depth limits;
- diagnostics for parent and child decisions;
- interaction with entrypoints and virtual model names.
Non-goals
- Do not implement arbitrary DAG orchestration.
- Do not replace external workflow engines.
- Do not mix this with fixed multi-step pipelines; #1815 tracks that use case.
Acceptance criteria
- A design clearly separates hierarchical routing from workflow orchestration.
- Delegation depth, cost, and recursion behavior are bounded.
- Replay can explain parent and child routing decisions.
- The design states whether implementation belongs in recipes, decisions, algorithms, or a separate layer.