vllm-project/semantic-router

feat: optimize and harden existing Looper, Flow, Fusion, and ReMoM algorithms

Open

#2,336 opened on Jul 5, 2026

View on GitHub
 (3 comments) (0 reactions) (1 assignee)Go (699 forks)github user discovery
area/corearea/model-selectionarea/momenhancementhelp wantedpriority/P1roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Optimize and harden the existing Looper-family algorithms, including Looper, Flow, Fusion, ReMoM, and related multi-model deliberation paths, before expanding the algorithm set.

Parent roadmap: #2287 Related: #1443, #1456, #1815, #2200, #2331, #2333, #2354, #2357 Child issue: #2371

Motivation

Looper-family algorithms are a major differentiator, but they must be safe, bounded, explainable, and measurable before they become a larger public API surface. Existing security and cost-risk issues show that algorithm expansion should be gated by hardening and evaluation.

The codebase already has a broad pkg/looper surface and selection algorithms. This issue should improve those existing paths first, then let #2337 explore new algorithms under shared benchmark/eval constraints.

Scope

Improve existing Looper-family behavior across:

  • benchmark and regression baselines (#2371);
  • cost and breadth controls;
  • modelRefs and candidate-count limits;
  • replay and diagnostic explainability;
  • header and security boundary hardening through #2357 / #1443 / #1456;
  • deterministic eval fixtures;
  • latency and token accounting;
  • compatibility with entrypoints, recipes, and virtual slugs through #2331 / #2354.

Non-goals

  • Do not add a new Looper algorithm in this issue.
  • Do not implement arbitrary workflow DAGs here.
  • Do not bypass safety plugins or hard policy boundaries for algorithm convenience.

Acceptance criteria

  • Existing Looper-family algorithms have clear cost and breadth caps.
  • Security issues such as header bypass and unbounded modelRefs are accounted for or explicitly blocked by dependencies.
  • Replay explains the algorithm path, candidate models, intermediate choices, and final decision.
  • Decision-level eval can measure Looper-family route correctness and overhead.
  • Algorithm virtual slugs work through the entrypoint/recipe model once #2331/#2354 land.

Validation

make agent-report ENV=cpu CHANGED_FILES="src/semantic-router/pkg/looper src/semantic-router/pkg/selection src/semantic-router/pkg/extproc src/semantic-router/pkg/config src/vllm-sr/cli bench website/docs"
make agent-lint CHANGED_FILES="src/semantic-router/pkg/looper src/semantic-router/pkg/selection src/semantic-router/pkg/extproc src/semantic-router/pkg/config src/vllm-sr/cli bench website/docs"
make agent-ci-gate CHANGED_FILES="src/semantic-router/pkg/looper src/semantic-router/pkg/selection src/semantic-router/pkg/extproc src/semantic-router/pkg/config src/vllm-sr/cli bench website/docs"

Contributor guide