On Febuary 13, 2023, David Bertoin defended his thesis at ISAE-SUPAERO in Toulouse. While working on his thesis, David has been part of the DEEL project at IRT Saint Exupéry.
ABOUT his THESIS
Representations for generalization in reinforcement learning
Abstract
This thesis tackles the problem of learning image-based control policies in simulated environments. Despite their ability to learn such policies from interactions alone, deep reinforcement learning agents tend to memorize trajectories rather than discover state representations leading to the capability to generalize to new situations. This generalization problem hinders the adoption of reinforcement learning in the real world. Within this thesis, we study several aspects of the generalization problem through the prism of the representations an agent can learn of its environment.
First, we propose a method to increase the diversity of representations in a neural policy’s latent space, and promote agents’ robustness to spurious correlations between visual elements and rewards. Second, we consider generalization as robustness to distracting visual elements unobserved during training such as changing backgrounds. We present a method based on neural network interpretability to discover representations encoding crucial information while demonstrating invariance to visual distractions. Third, we consider generalization to situations containing similar semantic information but represented differently in distinct domains. We introduce a method to learn disentangled representations, disambiguating between the useful semantic information common between domains, and its complementary context information. These contributions constitute a step towards learning representations which help close the generalization gap in reinforcement learning.
Scientific publications
- Numerical influence of ReLU’(0) on backpropagation, David Bertoin, Jérôme Bolte, Sébastien Gerchinovitz, Edouard Pauwels – Neurips 2021
- Disentanglement by cyclic reconstruction, David Bertoin, Emmanuel Rachelson – IEEE Transactions on Neural Networks and Learning Systems
- Hijacking an autonomous delivery drone equipped with the ACAS-Xu system, David Bertoin, Adrien Gauffriau, Jayant Sen Gupta – ERTS2022
- Local Feature Swapping for Generalization in Reinforcement Learning , David Bertoin, Emmanuel Rachelson – ICLR 2022
- Autonomous drone interception with Deep Reinforcement Learning, David Bertoin, Adrien Gauffriau, Damien Grasset, Jayant Sen Gupta – IJACI-ECAI 2022 Workshop: 12th International Workshop on Agents in Traffic and Transportation
- Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning, David Bertoin, Adil Zouitine, Mehdi Zouitine, Emmanuel Rachelson – Neurips 2022
ABOUT DEEL PROJECT
The DEEL (DEpendable Explainable Learning) project involves academic and industrial partners in the development of dependable, robust, explainable and certifiable artificial intelligence technological bricks applied to critical systems.
JURY
| Emmanuel RACHELSON | Thesis director | ISAE-SUPAERO |
| Sébastien GERCHINOVITZ | Thesis co-director | Université Paul Sabatier |
| Olivier PIETQUIN | Reviewer | Université de Lille |
| Liam PAULL | Reviewer | Université de Montréal |
| Vincent FRANÇOIS-LAVET | Examiner | Vrije Universiteit Amsterdam |
| Matthieu GEIST | Examiner | Université de Lorraine |
| Thomas OBERLIN | Examiner | ISAE-SUPAERO |
| Amy ZHANG | Examiner | University of Texas at Austin |