[Router] Extend HTTP API image input to classify/batch and the similarity endpoints (deferred from #2104)
#2,317 opened on Jul 3, 2026
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
What
#2104 (under review) delivers image input for /api/v1/classify/intent, /api/v1/eval, and /api/v1/embeddings - the first slices of #1911's staged delivery. Its body explicitly defers the remaining endpoints; this issue tracks them so the deferral has a home when #2104's "Closes #1911" fires at merge:
POST /api/v1/classify/batch(BatchClassificationRequest- texts only today)POST /api/v1/similarity(SimilarityRequest)POST /api/v1/similarity/batch(BatchSimilarityRequest)
Design notes carried from the #2104 review
- Reuse
pkg/utils/imageurlas the single input gate, including whatever payload-extraction consolidation lands from that review's findings. - The similarity endpoints are where the embedding-space question bites hardest: text routes to qwen3/gemma/mmbert while images route to the multimodal projection, and cosine across spaces is meaningless.
MultiModalEncodeTextexists in candle-binding but is unreachable from these endpoints - image-vs-text similarity likely requires amodel: "multimodal"option routing BOTH sides into the shared space. See the design question on #2104. - Dimension semantics: the multimodal model's native dimension is 384 while the API default is 768 - the over-dimension contract (#2104 review) should be settled before these endpoints wire in.
- Stage-2 consideration: text-bearing images (screenshots, scanned documents) carry their routing signal in the rendered text, not the image embedding. #2057 tracks the stage-2 text-modality check for the ExtProc path; every HTTP endpoint added here widens the same surface, so any stage-2 design should cover both entry paths.
Why now
#1911 was stale-closed while #2104 was in flight (since reopened). When #2104 merges it will close #1911, and without this issue the deferred endpoints lose tracking - staged-delivery issues are exactly what stale bots eat.
H2 execution scope
Extend image input handling consistently across classify, batch classify, and similarity endpoints as part of #2347 multimodal robustness.
Relevant code surfaces: src/semantic-router/pkg/apiserver, src/semantic-router/pkg/classification, multimodal embedding dispatch, config validation, and API docs.
Acceptance criteria
- Classify, batch classify, and similarity endpoints have documented image input shapes.
- Unsupported model/runtime paths return clear capability errors.
- Tests cover successful image input and unsupported/fallback behavior.
- API docs and capability discovery are updated together.