vllm-project/semantic-router
View on GitHubfeature: support multiple cross-encoder architectures behind a typed runtime
Open
#2,252 opened on Jun 18, 2026
area/researchhelp wantedroadmapsignal-decision-engine
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Audited upstream baseline (2026-07-15)
Current main has no general cross-encoder runtime. PR #2236 is open and proposes an initial BERT-family implementation, but that branch is not the repository baseline. This issue tracks architecture dispatch only after the base reranker contract is defined.
Scope
- Define architecture/runtime capability metadata for reranker declarations.
- Support loading or explicitly rejecting BERT-family, XLM-RoBERTa-family, and ModernBERT-family sequence-classification checkpoints.
- Keep tokenization, tensor layout, pooling/head behavior, label interpretation, and score normalization architecture-specific behind one typed interface.
- Add parity receipts against each model's authoritative reference implementation.
- Report unsupported architectures before serving traffic.
Boundaries
- Do not infer compatibility only from a repository/model name.
- Do not promise that every
bge-reranker-*artifact shares one architecture or score contract. - Do not expose architecture-specific details in routing policy.
- Do not add a public endpoint contract in this issue unless the base runtime/API issue owns it.
Acceptance criteria
- Model declarations carry enough metadata for deterministic dispatch.
- At least one supported checkpoint per implemented architecture has numerical parity evidence.
- Unsupported/mismatched weights fail with actionable diagnostics.
- Model-free tests cover dispatch and capability validation.
- Routing consumers remain architecture-neutral.
Relevant surfaces
Native bindings, src/semantic-router/pkg/modelruntime, model catalog/config (#2250), capability inventory (#2358), tests, and docs.
Related: PR #2236, #2247, #2249.