Lumipen 2: Robust Tracking for Dynamic Projection Mapping

Projection Mapping has recently generated much attention, and can be applied to areas such as performance art and human-computer interfaces. Projection Mapping typically targets static or semi-static objects, and few systems can deal with dynamic objects due to a large delay from measurement to projection. The delay results in significant misalignment in the case of dynamic objects, and leads to discomfort.
In our laboratory, the Lumipen system (URL: • Lumipen: Projection Ma... ) has been proposed to solve the time-geometric inconsistency caused by the delay when using dynamic objects. It consists of a projector and a high-speed optical axis controller with high-speed vision and mirrors, called Saccade Mirror (1ms Auto Pan-Tilt technology, URL: • 1ms Auto Pan-Tilt for ... ). Lumipen can provide projected images that are fixed on dynamic objects such as bouncing balls. However, the robustness of the tracking is sensitive to the simultaneous projection on the object, as well as the environmental lighting.
In order to achieve robust Dynamic Projection Mapping, we introduce a retroreflective background to the Lumipen system (Lumipen 2, URL: ishikawa-vision.org/mvf/Lumipe.... As a result, the object will appear darker than the background during projection, which is observed using a high-speed camera. The tracking will therefore be robust to changes of the content of the projection such as movies, and changes of environmental lighting in the object's vicinity associated with its motion. The tracking technique enables Dynamic Projection Mapping with partially well-lit content and time-geometric consistency. This opens up for new applications of Projection Mapping, such as visualizing a pitched ball as a fire ball.
[Link]
Ishikawa Watanabe Laboratory: ishikawa-vision.org/
Oku Laboratory: www.okulab.cs.gunma-u.ac.jp/
Lumipen 2 Project: ishikawa-vision.org/mvf/Lumipe...

[@realsammyt]

That's some really impressive trackin.

[@IshikawaLab]

A new video "Lumipen 2: Robust Tracking for Dynamic Projection Mapping" has been uploaded to RUclips. It is done with Prof. Oku of Gunma University. Although usual projection mapping systems use static objects as screens, this system perfectly tracking a moving object and make projection mapping on it. In particular, the earth can be projected on a white ball and bounce on a table. The system will be used in media art, sports science, manufacturing, vehicles, and so on.

[@figure66]

Génialissime !

[@freekbeta]

they just keep on amazing me. Real stuff

[@Optimistas777]

Amazing

[@Shaban_Interactive]

damn this is the new way of real time theeater VFX.

[@mykolasmith]

any plan to open source this?

[@stuartharris9931]

How long until this is available to purchase?

[@Barnacules]

This technology has come a very long ways, look for more current videos.

[@Kellerkind80]

So we will have the holo deck before the warp engine.