vllm-project/semantic-router

[Router] /api/v1/embeddings/models cannot report the multimodal model, so image capability is undiscoverable

Open

#2,318 opened on Jul 3, 2026

View on GitHub
 (2 comments) (0 reactions) (1 assignee)Go (699 forks)github user discovery
area/corehelp wantedroadmapsignal-decision-engine

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

What

GetEmbeddingModelsInfo (candle-binding/src/ffi/embedding.rs) reports models from the embedding factory only (qwen3/gemma); the multimodal model is registered in separate standalone storage and never appears in the response. With #2104 (under review) adding image input to /api/v1/embeddings, a client has no way to probe whether image embeddings are available on a deployment.

Why it matters

multimodal_model_path is optional config; when set but failing to load, the runtime logs a structured warning (multimodal_routes_ready: false), and when unset it skips multimodal init with no log at all - either way the API surface gives no signal, and an image request against such a deployment surfaces only as a 500 EMBEDDING_GENERATION_FAILED (making that a 4xx is a separate #2104 review item). Discovery belongs on the models endpoint: loaded state, native dimension (384), supported modalities.

Proposal

Either extend the FFI models-info call to include the standalone multimodal registration, or synthesize the entry Go-side from the runtime's own loaded state. Found while reviewing #2104; the gap is pre-existing (that PR does not touch the FFI info path).

H2 execution scope

Expose multimodal model capability discovery so clients and dashboards can tell whether image/audio embedding support is loaded and usable.

Relevant code surfaces: src/semantic-router/pkg/modelruntime, src/semantic-router/pkg/apiserver, src/semantic-router/pkg/modelinventory, #2358, and dashboard/API docs.

Acceptance criteria

  • Capability discovery reports multimodal support, model identity, and readiness state.
  • Clients can distinguish configured-but-unavailable from unsupported.
  • Dashboard/API docs describe the capability fields.
  • Tests cover no-model, text-only model, and multimodal-ready states where feasible.

Contributor guide