Master Thesis Proposals

MT1 - Safe Online Reinforcement Learning for Autonomous Racing Vehicles

Recent work has shown that reinforcement learning (RL) can be applied in virtual driving games to learn control policies that are able to outperform the best human players [1, 2]. However, it is still unclear how state-of-the-art RL [3] methods perform in the real world against standard control-based methods, where agents need to process with high-dimensional observations (e.g., RGBD, LIDAR) and output actions with high frequency. Moreover, the policy of these agents needs to follow strict constraints to ensure that their actuation is safe in a real-world environment [4].

The goal of this project is to assess how reinforcement learning methods can be employed for autonomous racing vehicles. The student will start by implementing a training pipeline for reinforcement learning agents in a realistic racing simulator. Subsequently, the student will train and evaluate several online safe reinforcement learning agents in the simulator environment. Finally, (if possible) the student will evaluate the trained policy on a real-world autonomous vehicle.

Required qualifications: proficiency in Machine Learning and Data Science. Applicants are expected to have passed KTH courses such as DD2421 Machine Learning, DD2424 Deep Learning in Data Science, EL2805 Reinforcement Learning or equivalent. Confidence in Python, C++, deep learning frameworks like PyTorch, TensorFlow, Jax and membership to the Formula Student team is a merit.

Contact People: Miguel Vasco (KTH) , Alfredo Reichlin (KTH)

References:

[1] - Wurman, Peter R., et al. "Outracing champion Gran Turismo drivers with deep reinforcement learning." Nature 602.7896 (2022): 223-228.

[2] - Betz, Johannes, et al. "Autonomous vehicles on the edge: A survey on autonomous vehicle racing." IEEE Open Journal of Intelligent Transportation Systems 3 (2022): 458-488.

[3] - Schwarzer, Max, et al. "Bigger, Better, Faster: Human-level Atari with human-level efficiency." International Conference on Machine Learning. PMLR, 2023.

[4] - Gu, Shangding, et al. "A review of safe reinforcement learning: Methods, theory and applications." arXiv preprint arXiv:2205.10330 (2022).