[Title]
Bridging the Modality Gap with Differentiable Intensity Rendering for Online LiDAR-Camera Calibration

[Journal]
IEEE Robotics and Automation Letters (RA-L) 2026

[Authors]
Daeho Kim, Jeong Woon Lee, Kyoleen Kwak, Chaneui Song and Hyoseok Hwang*

[Summary]
Precise spatial alignment between cameras and LiDAR is a prerequisite for robust multi-modal perception, yet this alignment is frequently disrupted by mechanical vibrations or drifts during operation, requiring recalibration on the fly. While recent 3D Gaussian Splatting-based methods offer promising differentiable solutions, they generally rely only on RGB and depth consistency and leave the reflectance information carried by LiDAR intensity underexploited, limiting their ability to fully bridge the modality gap. In this paper, we propose a novel online calibration framework that incorporates LiDAR intensity as a learnable Gaussian attribute, enabling dense and differentiable intensity maps to be rendered directly from sparse LiDAR point clouds. To align these rendered intensity maps with camera grayscale images despite their non-linear radiometric gap, we adopt a Modality Independent Neighborhood Descriptor (MIND) loss that captures local self-similarity patterns rather than absolute pixel values, together with a multi-stage optimization schedule that activates each objective only once its gradients become reliable. Experiments on KITTI, KITTI-360, and FAST-LIVO2 confirm that our method calibrates reliably and generalizes to both mechanical spinning and solid-state LiDARs, demonstrating that the differentiable intensity cue provides a complementary constraint beyond RGB and depth for downstream multi-modal perception.