feature: Add model-managed cross-encoder reranking for two-stage retrieval/routing

Summary

Add a model-managed cross-encoder reranking primitive for optional two-stage retrieval/routing, plus a compatible HTTP consumption surface if maintainers accept that API contract.

This is narrower than the RAG orchestration epic #155: it provides query/document relevance scoring that routing signals, retrieval, memory/vector-store consumers, and external RAG clients may reuse.

Audited current-main baseline (2026-07-15)

Current main provides:

• /api/v1/embeddings and similarity endpoints backed by bi-encoder embeddings;
• vector-store hybrid reranking in pkg/vectorstore/hybrid.go, which fuses dense and lexical scores;
• memory-store hybrid rerank implementations.

Current main does not provide a general query/document cross-encoder reranker or /v1/rerank route.

PR #2236 is open and prototypes a BERT-family cross-encoder plus /v1/rerank//api/v1/rerank using interim environment-based loading. It is not merged and must not be described as current behavior. Follow-ups #2250, #2251, and #2252 remain open for model-catalog declaration, optional routing-signal integration, and non-BERT architectures.

Performance and quality numbers reported in PR #2236 are prototype-specific results, not guarantees for main or arbitrary models/workloads.

Scope

1. Define the reranker capability in the model catalog/runtime lifecycle (#2250); avoid an environment-only production contract.
2. Define a typed pair-scoring engine with model identity, input limits, batching, timeout, cancellation, and deterministic result ordering.
3. Evaluate an optional two-stage signal path: bi-encoder shortlist, then cross-encoder rerank of bounded top-K (#2251).
4. Decide and document the external API shape. If /v1/rerank is accepted, state exact compatibility and differences from Cohere/vLLM rather than claiming drop-in parity.
5. Support honest fallback/unavailable behavior. Bi-encoder scoring must be labeled as bi-encoder, not silently presented as cross-encoder output.
6. Evaluate additional architectures only through capability dispatch and parity tests (#2252).
7. Keep existing vector-store hybrid score fusion distinct from model-based cross-encoder relevance.

Boundaries

• Optional cross-encoder routing adds bounded latency; do not claim “no added per-request latency.”
• Reranking improves relevance ordering, not factual correctness or safety.
• No model loads in an API handler or creates a second lifecycle/config source.
• Candidate limits, body limits, truncation, model availability, and partial deployment behavior must be explicit.
• The reranker cannot override authorization, safety, or candidate eligibility.

Acceptance criteria

• Model declaration, lifecycle, capability discovery, and unavailable behavior are explicit.
• Pair tokenization/scoring has numeric parity coverage against the declared reference implementation.
• Top-K, batching, truncation, timeout, cancellation, body limits, and concurrency are bounded.
• Routing-signal evaluation reports relevance/route quality and latency/cost against bi-encoder-only baselines.
• API compatibility is documented field by field if the endpoint is accepted.
• Existing hybrid rerank remains behaviorally distinct and covered.
• CPU tests and affected E2E paths are maintained; accelerator behavior is tested when supported.

Likely change surfaces

candle-binding/, src/semantic-router/pkg/modelruntime/, src/semantic-router/pkg/config/, src/semantic-router/pkg/classification/, src/semantic-router/pkg/apiserver/ if the endpoint is accepted, model capability inventory, docs, and tests.

Validation entrypoint

make agent-report ENV=cpu CHANGED_FILES="<space-separated changed files>"

Use the native-binding and affected E2E gates reported by the harness.

Related

#155, #2236, #2250, #2251, #2252, #2333, #2358, #2360.