[Bindings] onnx-binding image preprocessing diverges from candle-binding's PIL-equivalent resize
#2,166 opened on Jun 12, 2026
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
#1943 moved candle-binding image preprocessing to a Catmull-Rom support-weighted resize that tracks PIL bicubic+antialias to ~1e-4 cosine. onnx-binding's image path still uses its previous resize, so the two bindings now produce embeddings from different pixel pipelines for the same input image. Surfaced while reviewing the #1943 figure record.
Two candidate resolutions, maintainer call needed:
- Port the PIL-equivalent resize to onnx-binding (parity, but more code in a binding whose multimodal surface is still settling).
- Fail closed: have onnx-binding declare
MultimodalEmbedding: falseuntil its preprocessing reaches parity, so a mixed deployment cannot silently mix pixel pipelines.
Either way, a parity probe (same fixtures, both bindings, cosine matrix) should be the first step; no committed measurement currently exists for the onnx path.
H2 execution scope
Resolve ONNX/Candle image preprocessing parity or make unsupported divergent paths fail closed. This is a correctness dependency for #2347 and image-routing threshold calibration.
Relevant code surfaces: candle-binding, onnx-binding if present, image embedding dispatch in src/semantic-router/pkg/classification, and multimodal tests.
Acceptance criteria
- ONNX and Candle preprocessing differences are measured on shared fixtures.
- The implementation either matches preprocessing behavior or explicitly blocks divergent unsupported paths.
- Parity tests cover resize/crop/normalization behavior that affects embeddings.
- Downstream threshold calibration notes the fixed preprocessing path.
Validation
make agent-report ENV=cpu CHANGED_FILES="candle-binding onnx-binding src/semantic-router/pkg/classification src/semantic-router/pkg/config website/docs"
make agent-lint CHANGED_FILES="candle-binding onnx-binding src/semantic-router/pkg/classification src/semantic-router/pkg/config website/docs"
make agent-ci-gate CHANGED_FILES="candle-binding onnx-binding src/semantic-router/pkg/classification src/semantic-router/pkg/config website/docs"