vllm-project/semantic-router

feature: Model onboarding procedure

Open

#1,153 opened on Jan 21, 2026

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/model-selectionarea/momenhancementgood first issuehelp wantedpriority/P1roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Define a model onboarding procedure that turns a new candidate model into a validated router configuration update with benchmark evidence, model capability metadata, routing-policy impact, and monitoring expectations.

Parent roadmap: #2287 Related: #2333, #2334, #2358, #2359, #2360 Related code: src/semantic-router/pkg/modelselection, src/semantic-router/pkg/modelinventory, src/semantic-router/pkg/config/model_config_types.go, src/training, src/fleet-sim, deploy/recipes, website/docs

Motivation

The router config uses model endpoints, signal definitions, and routing decisions, but onboarding a new model still lacks a transparent procedure. A new model may change quality, latency, reasoning behavior, context limits, tool support, multimodal support, pricing, and fallback eligibility. Those properties should be evaluated and captured before the model is added to production recipes.

Scope

Define the onboarding flow for:

  • benchmark and eval selection for latency, accuracy, reasoning, token usage, tool/multimodal capability, and cost;
  • model capability inventory updates;
  • recipe/model_config updates;
  • routing-policy and threshold impact analysis;
  • fleet-sim or load/capacity validation when relevant;
  • monitoring and rollback expectations after activation.

Non-goals

  • Do not require every model to pass the same benchmark set regardless of intended role.
  • Do not silently update routing thresholds or recipes without eval evidence.
  • Do not rely only on online learning to discover model regressions.

Acceptance criteria

  • A contributor can follow a documented model onboarding checklist.
  • Onboarding produces model metadata, eval evidence, config changes, and validation receipts.
  • The process identifies whether the model is usable for routing, fallback, tools, multimodal, or special recipes.
  • Existing recipes can opt into the new model through explicit config changes.

Validation

make agent-report ENV=cpu CHANGED_FILES="src/semantic-router/pkg/modelselection src/semantic-router/pkg/modelinventory src/semantic-router/pkg/config src/training src/fleet-sim deploy/recipes website/docs"
make agent-lint CHANGED_FILES="src/semantic-router/pkg/modelselection src/semantic-router/pkg/modelinventory src/semantic-router/pkg/config src/training src/fleet-sim deploy/recipes website/docs"
make agent-ci-gate CHANGED_FILES="src/semantic-router/pkg/modelselection src/semantic-router/pkg/modelinventory src/semantic-router/pkg/config src/training src/fleet-sim deploy/recipes website/docs"

Contributor guide