Repository metrics
- Stars
- (675 stars)
- PR merge metrics
- (PR metrics pending)
Description
🚀 Feature Description
The current kornia-3d camera module only has PinholeCamera. We need a FisheyeCamera implementing the Kannala-Brandt equidistant projection model, with project/unproject so that downstream consumers can build fisheye → pinhole undistortion pipelines.
PRs for this issue should target the feat/slam-utils branch.
Proof of correctness
Use an actual Hilti fisheye image as your test sample. Images can be extracted from the challenge dataset using the https://github.com/hilti-research/hilti-trimble-slam-challenge-2026.
Include in your PR:
- A before/after image showing the raw fisheye frame and the undistorted pinhole result
Acceptance criteria
FisheyeCamerawith project and unproject using the Kannala-Brandt equidistant model- Round-trip test: project(point) → unproject(pixel) recovers the original ray direction
- Before/after visualization on a real Hilti fisheye image
- Unit tests with known 3D points → expected pixel coordinates
📂 Feature Category
Rust Core Library
💡 Motivation
kornia-slam needs to support fisheye cameras for datasets like https://github.com/Hilti-Research/hilti-trimble-slam-challenge-2026, which uses dual Kannala-Brandt equidistant fisheye cameras (Insta360, One-RS, 1472×1440, ~200° FoV per lens).
💭 Proposed Solution
What to implement
Add `FisheyeCamera` to kornia-3d (new src/camera/ module),
using the Kannala-Brandt equidistant projection model:
/// Fisheye camera using the Kannala-Brandt equidistant projection model.
///
/// Projects a 3D point (x, y, z) to pixel (u, v) using:
/// r = sqrt(x² + y²)
/// theta = atan2(r, z)
/// theta_d = theta + k1*θ³ + k2*θ⁵ + k3*θ⁷ + k4*θ⁹
/// u = fx * theta_d * (x/r) + cx
/// v = fy * theta_d * (y/r) + cy
pub struct FisheyeCamera {
pub fx: f64,
pub fy: f64,
pub cx: f64,
pub cy: f64,
pub k1: f64,
pub k2: f64,
pub k3: f64,
pub k4: f64,
}
impl FisheyeCamera {
/// Project a 3D point to pixel coordinates.
/// Returns None if the point is behind the camera or at the optical center.
pub fn project(&self, point: &Vec3F64) -> Option<[f64; 2]>;
/// Unproject a pixel to a unit bearing ray in camera frame.
/// Requires solving theta_d → theta via Newton's method.
pub fn unproject(&self, pixel: [f64; 2]) -> Vec3F64;
}
📚 Library Reference
- Kannala & Brandt (2006) — original paper
- OpenCV fisheye module (BSD license) — reference for projection and Newton iteration
🔄 Alternatives Considered
No response
🎯 Use Cases
No response
📝 Additional Context
No response
🤝 Contribution Intent
- I plan to submit a PR to implement this feature
- I'm requesting this feature but not planning to implement it