kornia/kornia-rs

[Feature]: Add execution provider selection and inference benchmarking

Open

#815 opened on Mar 13, 2026

View on GitHub
 (3 comments) (0 reactions) (0 assignees)Rust (188 forks)auto 404
enhancementhelp wantedtriage

Repository metrics

Stars
 (675 stars)
PR merge metrics
 (PR metrics pending)

Description

🚀 Feature Description

Add execution provider selection (CPU/CUDA) and multi-iteration inference benchmarking to the existing examples/onnx example.

📂 Feature Category

Testing Infrastructure

💡 Motivation

The current examples/onnx example only runs inference using the default CPU execution provider with a single pass. This doesn't reflect real-world applications where backend selection and accurate latency measurement are critical. When evaluating models for deployment on different hardware (CPU vs GPU), having a quick way to compare execution providers within the same example is essential.

💭 Proposed Solution

  • Add a --device CLI flag to select the execution provider (cpu or cuda)
  • Add a --num-iterations CLI flag for multi-run benchmarking (default: 10)
  • Add a warm-up run before timed iterations to exclude cold-start overhead (memory allocation, graph optimization, kernel compilation)
  • Print latency statistics (mean, min, max) across iterations
  • Update README with usage examples for both CPU and CUDA execution

📚 Library Reference

🔄 Alternatives Considered

Considered creating a completely new example, but improving the existing one keeps codebase lean and provides immediate value to current users of the ONNX example.

🎯 Use Cases

  • Comparing inference latency across CPU and CUDA backends for model deployment decisions
  • Providing a starting point for users who want to run ONNX models with GPU acceleration
  • Laying groundwork for TensorRT execution provider support (related: #634, GSoC 2026 VLM Inference project)

📝 Additional Context

This is a small, focused improvement to an existing example. The changes are compatible, default behavior remains CPU with a single run if no extra flags are provided.

🤝 Contribution Intent

  • I plan to submit a PR to implement this feature
  • I'm requesting this feature but not planning to implement it

Contributor guide