[2309.17371] Adversarial Imitation Studying from Visible Observations utilizing Latent Data

Obtain a PDF of the paper titled Adversarial Imitation Studying from Visible Observations utilizing Latent Data, by Vittorio Giammarino and a couple of different authors

Obtain PDF

Summary:We give attention to the issue of imitation studying from visible observations, the place the educational agent has entry to movies of specialists as its sole studying supply. The challenges of this framework embrace the absence of skilled actions and the partial observability of the setting, because the ground-truth states can solely be inferred from pixels. To deal with this downside, we first conduct a theoretical evaluation of imitation studying in partially observable environments. We set up higher bounds on the suboptimality of the educational agent with respect to the divergence between the skilled and the agent latent state-transition distributions. Motivated by this evaluation, we introduce an algorithm referred to as Latent Adversarial Imitation from Observations, which mixes off-policy adversarial imitation methods with a realized latent illustration of the agent’s state from sequences of observations. In experiments on high-dimensional steady robotic duties, we present that our algorithm matches state-of-the-art efficiency whereas offering important computational benefits. Moreover, we present how our methodology can be utilized to enhance the effectivity of reinforcement studying from pixels by leveraging skilled movies. To make sure reproducibility, we offer free entry to our code.