Direct 3D Model-Based Object Tracking with Event Camera by Motion Interpolation

Accompanying video of the article "Direct 3D Model-Based Object Tracking with Event Camera by Motion Interpolation" by Y. Kang, G. Caron, R. Ishikawa, A. Escande, K. Chappellet, R. Sagawa, T. Oishi presented and to appear in the proceedings of the IEEE International Conference on Robotics and Automation in 2024.
Event cameras are recent sensors that measure intensity
changes in each pixel asynchronously. It is being used due
to lower latency and higher temporal resolution compared to
traditional frame-based camera. We propose a method of 3D
model-based object tracking directly from events captured
by event camera. To enable reliable and accurate tracking
of objects, we use a new event representation and predict
brightness increment images with motion interpolation.
Results of object tracking show the new methods
significantly improves tracking duration and robustness,
both for perspective and fisheye cameras. Our
implementation succeeds in tracking objects when the camera
speed is reaching 2 m/s.
Article reference: hal.science/ha...