Roberta Raileanu

I am a Research Scientist at Meta GenAI based in London, where I lead a team working on building generally capable AI agents. I also led the team working on tool use for Llama-3.

In 2021, I obtained my PhD in Computer Science from NYU, advised by Rob Fergus. My focus was on deep reinforcement learning. During my PhD, I was fortunate to intern at DeepMind, Microsoft Research, and Facebook AI Research.

Previously, I obtained a B.A. in Astrophysics from Princeton University, where I worked with Michael Strauss on theoretical cosmology and Eve Ostriker on supernovae simulations.

In a past life (aka high school), I was fortunate to have the opportunity of competing in IPhO, IOAA, IAO, as well as national math and computer science olympiads.

Email  /  CV  /  Google Scholar  /  GitHub  /  Twitter  /  LinkedIn

I am broadly interested in designing machine learning algorithms that can make robust sequential decisions in complex environments, while constantly acquiring new skills and knowledge. My long-term goal is to develop models that can solve a wide range of new tasks with little or no supervision and training. I also care deeply about ensuring such systems are reliable, trustworthy, and aligned with human intentions.

My work draws from fields such as reinforcement learning, open-ended learning, self-supervised learning, and natural language processing. Currently, I work at the intersection of large language models (LLMs) and decision making / RL. In particular, I focus on augmenting LLMs with decision making abilities (e.g. planning, reasoning, actions, tools, and goals), and teaching them to learn from (human) feedback and interaction in an open-ended learning fashion.

I'm also interested in understanding and improving the generalization and robustness of autonomous agents and large pretrained models. In the past, I've worked on deep reinforcement learning, exploration, fast adaptation, learning from demonstrations, as well as continual and multi-agent learning.

A method that teaches language models to use tools in a self-supervised way.

Survey on augmenting language models with reasoning, actions, and tool use.

Introduce a continual reinforcement learning method that incrementally builds a subspace of policies and adaptively prunes it to preserve a good trade-off between model size and performance.

Present the NetHack Learning Dataset (NLD), a large and highly scalable dataset of human and bot trajectories on the popular game of NetHack.

Extend episodic count-based bonuses to continuous state spaces for better exploration of contextual MDPs with high-dimensional observations.

Using language to highlight relevant state abstractions leads to better exploration in sparse reward procedurally generated environments.

Training agents on dynamically changing task distributions leads to more general agents capable of solving a wide range of tasks in procedurally generated environments.

Using a common representation for the policy and value function can lead to overfitting in deep reinforcement learning. To improve generalization, use the advantage instead of the value as auxiliary loss to train the policy network, while encouraging the representation to be invariant to task-irrelevant properties of the environment.

Use UCB to automatically select an augmentation from a given set, which is then used to regularize the policy and value function of an RL agent.

Learn a value function for a space of policies and environments (with different dynamics) and use it for fast adaptation in new environments with unseen dynamics.

Reward agents for taking actions that lead to large changes in the environment and for visiting new states within an episode.

The NetHack Learning Environment (NLE) is a fast, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular game NetHack.

A teacher learns to generate goals at an appropriate level of difficulty for a student, creating an automatic curriculum that aids exploration.

Create a curriculum by initializing the RL agent along a single demonstration (either optimal or suboptimal) starting near the end of the trajectory.

Simulate other agents' behavior and infer their intentions by using your own policy.

Use numerical simulations to analyze the evolution and properties of superbubbles, driven by supernovae, that propagate into the two-phase, cloudy interstellar medium.

website credits