Simulation to Reality: Transfer Learning for Automating Pseudo-Labeling of Real and Infrared Imagery

This is a video summary of the similarly named Journal article submitted to Advanced Intelligent Systems Journal in July 2024.
Abstract from article: Training a CNN for real-world applications is challenging due to the requirement of high-quality labeled imagery, prone to errors if labeled by humans or unrealistic if synthetically generated.
This study employs pseudo-labeling and transfer learning built upon a 6D pose estimation framework.
A CNN trained on synthetic images predicts bounding boxes (bbox) for an object's components in a real image. The framework then solves for the object's pose relative to the camera. The pose is utilized to reproject bboxes for all components onto that image. Thereby, enabling automated labeling of large datasets with minimal human intervention. The novel system is tested on color and long-wave infrared imagery captured during December 2023 flight tests. These tests highlight its ability to increase the amount of bbox predictions, enhance performance across situations, reduce reprojection error, and stabilize pose predictions.
This technique is significant as it enables labeling of real-world imagery without expensive truth systems, requiring only a camera. It supports learning and labeling of previously captured imagery without camera calibrations, facilitating labeled data creation for impractical-to-simulate sensors. Ultimately, this transfer learning approach provides a low-cost and precise method for creating CNNs trained on operationally relevant data, previously unattainable by the everyday user.