vllm-project/semantic-router

feat: expand signal extractors and improve signal model coverage

Open

#2,335 opened on Jul 5, 2026

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/corearea/researchenhancementenvironment/rocmhelp wantedpriority/P1roadmapsignal-decision-engine

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Expand signal extractors and improve signal model coverage while keeping signals registered, observable, evaluated, and versioned through shared contracts.

Parent roadmap: #2287 Related: #2330, #2333, #2347, #2341 Child and related issues: #2360, #2382, #1485, #2154, #2247, #2250, #2251, #2252, #2301

Motivation

The router already has many signal families in config and runtime code: keyword, embedding, domain/category, fact_check, user_feedback, reask, preference, language, context, structure, complexity, modality, authz, jailbreak, pii, kb, conversation, event, and projection. H2 adds two requirements:

  1. make the signal surface extensible and observable through #2330;
  2. improve signal quality through better models, extractors, calibration, and lifecycle discipline.

Scope

This issue coordinates signal/model quality work across:

  • domain classifier accuracy (#1485);
  • complexity signal backend improvements (#2154);
  • cross-encoder reranking as signal/model support (#2247, #2250, #2251, #2252);
  • mmBERT model discovery (#2301);
  • multimodal/image routing robustness (#2347);
  • signal model lifecycle, datasets, evals, and release provenance (#2360);
  • next-generation router base-model and model-family research for rerankers, embeddings, and classifiers under SOTA quality, long-context, high-concurrency, and low-latency constraints (#2382);
  • confidence/threshold calibration (#2341).

Non-goals

  • Do not accept new signal extractors without registration, observation output, and evaluation hooks.
  • Do not silently update default signal model behavior without calibration/provenance.
  • Do not mix recipe-local signal references with global custom-signal runtime configuration.

Acceptance criteria

  • New signal extractors are registered through the unified signal registry/observation model.
  • Signal model accuracy changes include reproducible eval evidence and calibration notes.
  • Runtime diagnostics expose matched signal names, confidence/value where available, and extractor errors/fallbacks.
  • Signal improvements can feed decision-level eval and dashboard analytics.

Validation

make agent-report ENV=cpu CHANGED_FILES="src/semantic-router/pkg/classification src/semantic-router/pkg/config src/semantic-router/pkg/decision src/semantic-router/pkg/extproc src/semantic-router/pkg/modelruntime src/training candle-binding nlp-binding ml-binding website/docs"
make agent-lint CHANGED_FILES="src/semantic-router/pkg/classification src/semantic-router/pkg/config src/semantic-router/pkg/decision src/semantic-router/pkg/extproc src/semantic-router/pkg/modelruntime src/training candle-binding nlp-binding ml-binding website/docs"
make agent-ci-gate CHANGED_FILES="src/semantic-router/pkg/classification src/semantic-router/pkg/config src/semantic-router/pkg/decision src/semantic-router/pkg/extproc src/semantic-router/pkg/modelruntime src/training candle-binding nlp-binding ml-binding website/docs"

Contributor guide