vllm-project/semantic-router

feat: define provider capability inventory and protocol conformance contract

Open

#2,358 opened on Jul 5, 2026

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Go (699 forks)github user discovery
area/corearea/networkingenhancementhelp wantedoperationspriority/P1roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Define a provider capability inventory and protocol conformance contract across OpenAI-compatible, Anthropic-compatible, self-hosted, and external provider paths.

Parent roadmap: #2287 Related: #1138, #2200, #2294, #2318, #2331, #2332 Related code: src/semantic-router/pkg/openai, src/semantic-router/pkg/anthropic, src/semantic-router/pkg/modelinventory, src/semantic-router/pkg/modelruntime, src/semantic-router/pkg/config/model_config_types.go, src/semantic-router/pkg/extproc

Motivation

The config already has provider profiles, model capabilities, API formats, OpenAI/Anthropic code paths, model inventory, runtime warmup, and multimodal capability discovery needs. H2 features such as fallback, recipes, backend-aware routing, and dashboard analytics need a single capability view instead of scattered assumptions about which model or provider supports streaming, tools, multimodal input, reasoning controls, embeddings, reranking, or fallback.

Scope

Define and implement a contract for:

  • provider profile capabilities and protocol variants;
  • per-model capability discovery and static declarations;
  • OpenAI/Anthropic compatibility expectations;
  • multimodal, tools, reasoning, embedding, rerank, image, and streaming capability flags;
  • fallback/provider failover eligibility;
  • config dump and /models style API exposure where appropriate.

Non-goals

  • Do not make all providers expose the same features.
  • Do not replace model selection or backend-aware routing.
  • Do not silently route unsupported request shapes to incapable models.

Acceptance criteria

  • Capability information has one normalized runtime representation.
  • Routing, fallback, dashboard, and API surfaces can consume capabilities consistently.
  • Unsupported capability mismatches fail closed or fall back with diagnostics.
  • Existing model_config/provider_profiles continue to work through compatibility normalization.

Contributor guide