Autonomous Parking based on Reinforcement Rearning and Digital Twins

The video shows a car learning to drive into a parking lot using reinforcement learning. The scenario is performed in a virtual testbed that enables the following key aspects:
Execution of digital twins at different levels of detail. For example, the movement of a car can be carried out both by moving it directly to a predefined position and by calculating the driving dynamics realistically using the accelerator pedal position and steering wheel angle
Mechanisms for implementing the reinforcement learning application, such as specifying an agent including neural networks, the possibility of defining a reward function and specifying terminal states.
In combination, this enables the sensors installed on the vehicle's Digital Twin to be used for a collision-free parking training.
Our key to success for fast and successful reinforcement learning was an incremental increase in complexity.
The training first shows a fast "kinematic" simulation where the changes in steering angles and acceleration and braking commands specified by the agent were used to simply approximate the car's current position.
In addition, the training process has been divided into different levels with increasing difficulty depending on the starting position of the car. In stage 1, the vehicle is positioned straight and directly in front of the parking space. As soon as the agent has learned the appropriate driving maneuvers, the next level is entered. In stage 2, the vehicle is positioned further away but still straight and in stage 3 it is positioned completely randomly. Finally, the position of the parking lot is also varied in level 4.
In the subsequent training the kinematic simulation approach was then replaced by a realistic calculation of the car movement.
While the learned obstacle avoidance is based entirely on radar, the target direction was determined in a first step using ground truth. After completion of the training this was replaced by a camera-based approach. For this purpose, a Yolov8 neural network was trained to detect the parking lot. The virtual testbed was used to take images with the panoramic camera from any position and at the same time automatically assign the label for the free parking space. The target direction could then be derived directly from the position of the detected parking lot in the image.
As a result, the agent was able to locate the parking lot in real time based on the incoming camera data and to park there collision-free by means of suitable driving maneuvers.
www.rif-ev.de
www.mmi.rwth-a...