[Router] /api/v1/embeddings/models cannot report the multimodal model, so image capability is undiscoverable
#2,318 opened on Jul 3, 2026
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
What
GetEmbeddingModelsInfo (candle-binding/src/ffi/embedding.rs) reports models from the embedding factory only (qwen3/gemma); the multimodal model is registered in separate standalone storage and never appears in the response. With #2104 (under review) adding image input to /api/v1/embeddings, a client has no way to probe whether image embeddings are available on a deployment.
Why it matters
multimodal_model_path is optional config; when set but failing to load, the runtime logs a structured warning (multimodal_routes_ready: false), and when unset it skips multimodal init with no log at all - either way the API surface gives no signal, and an image request against such a deployment surfaces only as a 500 EMBEDDING_GENERATION_FAILED (making that a 4xx is a separate #2104 review item). Discovery belongs on the models endpoint: loaded state, native dimension (384), supported modalities.
Proposal
Either extend the FFI models-info call to include the standalone multimodal registration, or synthesize the entry Go-side from the runtime's own loaded state. Found while reviewing #2104; the gap is pre-existing (that PR does not touch the FFI info path).
H2 execution scope
Expose multimodal model capability discovery so clients and dashboards can tell whether image/audio embedding support is loaded and usable.
Relevant code surfaces: src/semantic-router/pkg/modelruntime, src/semantic-router/pkg/apiserver, src/semantic-router/pkg/modelinventory, #2358, and dashboard/API docs.
Acceptance criteria
- Capability discovery reports multimodal support, model identity, and readiness state.
- Clients can distinguish configured-but-unavailable from unsupported.
- Dashboard/API docs describe the capability fields.
- Tests cover no-model, text-only model, and multimodal-ready states where feasible.