vllm-project/semantic-router

feat: design hierarchical routing and recipe chaining without becoming a workflow engine

Open

#2,343 opened on Jul 5, 2026

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Go (699 forks)github user discovery
area/corearea/model-selectionarea/researchenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Design hierarchical routing and recipe chaining so one routing decision can delegate to another bounded routing policy, without turning vLLM Semantic Router into a general workflow engine.

Parent roadmap: #2287 Related: #1815, #2322, #2331, #2333

Motivation

Users have asked for hierarchical and multi-step routing: a top-level router picks a domain or agent, then a secondary router makes a more granular decision. This is related to, but different from, fixed multi-step model pipelines.

Scope

Define a bounded router-native model for:

  • nested route decisions;
  • recipe-to-recipe delegation;
  • sub-router candidate constraints;
  • recursion and depth limits;
  • diagnostics for parent and child decisions;
  • interaction with entrypoints and virtual model names.

Non-goals

  • Do not implement arbitrary DAG orchestration.
  • Do not replace external workflow engines.
  • Do not mix this with fixed multi-step pipelines; #1815 tracks that use case.

Acceptance criteria

  • A design clearly separates hierarchical routing from workflow orchestration.
  • Delegation depth, cost, and recursion behavior are bounded.
  • Replay can explain parent and child routing decisions.
  • The design states whether implementation belongs in recipes, decisions, algorithms, or a separate layer.

Contributor guide