vllm-project/semantic-router

feature: support multiple cross-encoder architectures behind a typed runtime

Open

#2,252 opened on Jun 18, 2026

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/researchhelp wantedroadmapsignal-decision-engine

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Audited upstream baseline (2026-07-15)

Current main has no general cross-encoder runtime. PR #2236 is open and proposes an initial BERT-family implementation, but that branch is not the repository baseline. This issue tracks architecture dispatch only after the base reranker contract is defined.

Scope

  • Define architecture/runtime capability metadata for reranker declarations.
  • Support loading or explicitly rejecting BERT-family, XLM-RoBERTa-family, and ModernBERT-family sequence-classification checkpoints.
  • Keep tokenization, tensor layout, pooling/head behavior, label interpretation, and score normalization architecture-specific behind one typed interface.
  • Add parity receipts against each model's authoritative reference implementation.
  • Report unsupported architectures before serving traffic.

Boundaries

  • Do not infer compatibility only from a repository/model name.
  • Do not promise that every bge-reranker-* artifact shares one architecture or score contract.
  • Do not expose architecture-specific details in routing policy.
  • Do not add a public endpoint contract in this issue unless the base runtime/API issue owns it.

Acceptance criteria

  • Model declarations carry enough metadata for deterministic dispatch.
  • At least one supported checkpoint per implemented architecture has numerical parity evidence.
  • Unsupported/mismatched weights fail with actionable diagnostics.
  • Model-free tests cover dispatch and capability validation.
  • Routing consumers remain architecture-neutral.

Relevant surfaces

Native bindings, src/semantic-router/pkg/modelruntime, model catalog/config (#2250), capability inventory (#2358), tests, and docs.

Related: PR #2236, #2247, #2249.

Contributor guide