kornia/kornia-rs

[Feature]: Fisheye camera model and undistortion

Open

#822 opened on Mar 16, 2026

View on GitHub
 (13 comments) (0 reactions) (1 assignee)Rust (188 forks)auto 404
enhancementhelp wantedtriage

Repository metrics

Stars
 (675 stars)
PR merge metrics
 (PR metrics pending)

Description

🚀 Feature Description

The current kornia-3d camera module only has PinholeCamera. We need a FisheyeCamera implementing the Kannala-Brandt equidistant projection model, with project/unproject so that downstream consumers can build fisheye → pinhole undistortion pipelines.

PRs for this issue should target the feat/slam-utils branch.

Proof of correctness

Use an actual Hilti fisheye image as your test sample. Images can be extracted from the challenge dataset using the https://github.com/hilti-research/hilti-trimble-slam-challenge-2026.

Include in your PR:

  • A before/after image showing the raw fisheye frame and the undistorted pinhole result

Acceptance criteria

  • FisheyeCamera with project and unproject using the Kannala-Brandt equidistant model
  • Round-trip test: project(point) → unproject(pixel) recovers the original ray direction
  • Before/after visualization on a real Hilti fisheye image
  • Unit tests with known 3D points → expected pixel coordinates

📂 Feature Category

Rust Core Library

💡 Motivation

kornia-slam needs to support fisheye cameras for datasets like https://github.com/Hilti-Research/hilti-trimble-slam-challenge-2026, which uses dual Kannala-Brandt equidistant fisheye cameras (Insta360, One-RS, 1472×1440, ~200° FoV per lens).

💭 Proposed Solution

What to implement

  Add `FisheyeCamera` to kornia-3d (new src/camera/ module),
  using the Kannala-Brandt equidistant projection model:

  /// Fisheye camera using the Kannala-Brandt equidistant projection model.
  ///
  /// Projects a 3D point (x, y, z) to pixel (u, v) using:
  ///   r = sqrt(x² + y²)
  ///   theta = atan2(r, z)
  ///   theta_d = theta + k1*θ³ + k2*θ⁵ + k3*θ⁷ + k4*θ⁹
  ///   u = fx * theta_d * (x/r) + cx
  ///   v = fy * theta_d * (y/r) + cy
  pub struct FisheyeCamera {
      pub fx: f64,
      pub fy: f64,
      pub cx: f64,
      pub cy: f64,
      pub k1: f64,
      pub k2: f64,
      pub k3: f64,
      pub k4: f64,
  }

  impl FisheyeCamera {
      /// Project a 3D point to pixel coordinates.
      /// Returns None if the point is behind the camera or at the optical center.
      pub fn project(&self, point: &Vec3F64) -> Option<[f64; 2]>;

      /// Unproject a pixel to a unit bearing ray in camera frame.
      /// Requires solving theta_d → theta via Newton's method.
      pub fn unproject(&self, pixel: [f64; 2]) -> Vec3F64;
  }

📚 Library Reference

🔄 Alternatives Considered

No response

🎯 Use Cases

No response

📝 Additional Context

No response

🤝 Contribution Intent

  • I plan to submit a PR to implement this feature
  • I'm requesting this feature but not planning to implement it

Contributor guide