Roberta Raileanu

I am a research scientist at Facebook AI Research (FAIR) based in London. In 2021, I obtained my PhD in computer science from NYU, where I was advised by Rob Fergus as part of the CILVR lab. My research focuses on deep reinforcement learning. During my PhD, I was fortunate to spend time as a research intern at DeepMind, Microsoft Research, and Facebook AI Research.

Previously, I got a B.A. in Astrophysics from Princeton University, where I worked with Michael Strauss on theoretical cosmology and Eve Ostriker on supernovae simulations. During high school, I had the chance to compete in international physics and astrophysics Olympiads.

Email  /  CV  /  Google Scholar  /  GitHub  /  Twitter  /  LinkedIn

profile photo

I am broadly interested in designing machine learning algorithms that can make robust sequential decisions in complex environments, while constantly acquiring new skills and knowledge. In the past, I've worked on various problems in reinforcement learning including learning from demonstrations, exploration of procedurally generated environments, fast adaptation to new dynamics, and multi-agent learning.

My current focus is on understanding and improving the generalization and robustness of reinforcement learning agents. In particular, I am interested in understanding how we can best leverage unsupervised and open-ended learning techniques to expand the range of capabilities of RL agents in many diverse settings.

I'm always looking for collaborations, so if you're interested in working with me, please don't hesitate to get in touch! I'm also happy to advise PhD students on projects.

Decoupling Value and Policy for Generalization in Reinforcement Learning
Roberta Raileanu, Rob Fergus
ICML, 2021 (oral)

Using a common representation for the policy and value function can lead to overfitting in deep reinforcement learning. To improve generalization, use the advantage instead of the value as auxiliary loss to train the policy network, while encouraging the representation to be invariant to task-irrelevant properties of the environment.

Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
Roberta Raileanu, Max Goldstein, Denis Yarats, Ilya Kostrikov, Rob Fergus
NeurIPS, 2021
Inductive Biases, Invariances and Generalization in RL (BIG) Workshop, ICML, 2020 (oral)
paper / code / slides / project page

Use UCB to automatically select an augmentation from a given set, which is then used to regularize the policy and value function of an RL agent.

Fast Adaptation via Policy-Dyamics Value Functions
Roberta Raileanu, Max Goldstein, Arthur Szlam, Rob Fergus
ICML, 2020
Beyond "Tabula Rasa" in Reinforcement Learning (BeTR-RL) Workshop, ICLR, 2020 (oral)
paper / code / slides / project page

Learn a value function for a space of policies and environments (with different dynamics) and use it for fast adaptation in new environments with unseen dynamics.

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
Roberta Raileanu, Tim Rocktäschel
ICLR, 2020
paper / code / slides

Reward agents for taking actions that lead to large changes in the environment and for visiting new states within an episode.

The NetHack Learning Environment
Heinrich Küttler, Nantas Nardelli, Alexander H. Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, Tim Rocktäschel
NeurIPS, 2020
Beyond "Tabula Rasa" in Reinforcement Learning (BeTR-RL) Workshop, ICLR, 2020
paper / code / slides

The NetHack Learning Environment (NLE) is a fast, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular game NetHack.

Learning with AMIGo: Adversarially Motivated Intrinsic Goals
Andres Campero, Roberta Raileanu, Heinrich Küttler, Joshua B. Tenenbaum, Tim Rocktäschel, Edward Grefenstette,
ICLR, 2021
paper / code

A teacher learns to generate goals at an appropriate level of difficulty for a student, creating an automatic curriculum that aids exploration.

Backplay: "Man muss immer umkehren"
Cinjon Resnick*, Roberta Raileanu*, Sanyam Kapoor, Alexander Peysakhovich, Kyunghyun Cho, Joan Bruna
Reinforcement Learning in Games Workshop, AAAI, 2019
paper / slides

Create a curriculum by initializing the RL agent along a single demonstration (either optimal or suboptimal) starting near the end of the trajectory.

Modeling Others using Oneself in Multi-Agent Reinforcement Learning
Roberta Raileanu, Emily Denton, Arthur Szlam, Rob Fergus
ICML, 2018
Emergent Communication Workshop, NeurIPS, 2017
paper / slides

Simulate other agents' behavior and infer their intentions by using your own policy.

Superbubbles in the Multiphase ISM and the Loading of Galactic Winds
Chang-Goo Kim, Eve C. Ostriker, Roberta Raileanu,
The Astrophysical Journal, 2016

Use numerical simulations to analyze the evolution and properties of superbubbles, driven by supernovae, that propagate into the two-phase, cloudy interstellar medium.

website credits