Sharath Chandra Raparthy

I am currently an AI Resident at FAIR at Meta, collaborating closely with Roberta Raileanu and Mikael Henaff. My research focus has evolved to primarily include Large Language Model (LLM) reasoning, tool fine-tuning, and in-context learning. Additionally, I am engaged in exploring the dimensions of LLM safety to ensure responsible and ethical AI development.

Before joining Meta, I completed a Master's (with thesis) at Mila under the guidance of Prof. Irina Rish. My academic journey also included a valuable stint at Recursion, where my projects were centered around GFlowNets in the realm of generative chemistry. Although my earlier research was anchored in reinforcement learning, particularly in developing efficient algorithms for sample efficiency in continual RL scenarios, my interests have since shifted to the cutting-edge domains of LLMs.

Outside the realm of AI research, my passions include photography, long-distance running, reading and cooking.

Email  /  GitHub  /  Google Scholar

We introduce Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. We achieve SOTA performance for LLM models at these scales.

Introducing Rainbow Teaming, a new method for generating diverse adversarial prompts for LLMs via LLMs. It's a versatile tool 🛠️ for diagnosing model vulnerabilities across domains and creating data to enhance robustness & safety.

How to bootstrap the reasoning refinement capabilities of LLMs using synthetic data? Introducing "GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements". Applied on GSM8K we can improve a strong RL finetuned LLama-2 13B by 12%

In this work, we set out to understand how different algorithms fare at improving LLM reasoning from feedback. We compare expert iteration, PPO, and return-conditioned RL using Llama-2 as the base model.

Training autonomous agents to learn new tasks from few demonstrations is challenging, especially for sequential decision making which is sensitive to errors. In this paper, we show that training transformers on diverse offline datasets of trajectories enables in-context learning of out-of-distribution sequential decision tasks from just a handful of demonstrations.

We examine the standard approach to multi-objective optimization in machine learning applications like drug discovery and material design from a fresh perspective, noting the failure of existing methods to achieve a diverse set of Pareto-optimal candidates. Motivated by the successful use of GFlowNets in single-objective settings, we introduce a new approach, Multi-Objective GFlowNets (MOGFNs), which features a novel Conditional GFlowNet to handle a variety of single-objective sub-problems derived from decomposing the multi-objective problem. Our research, the first to empirically test Conditional GFlowNets, shows that MOGFNs outperform existing methods in Hypervolume, R2-distance, and candidate diversity, even demonstrating their effectiveness in active learning settings.

We view the standard Multi-Head attention mechanism from the "Search-Retrieval" perspective and highlight the rigid associations of keys and values. We propose a new drop-in replacement mechanism, Compositional Attention, where the redundancies highlighted are addressed by disentangling the Searches and Retrievals and composing them dynamically in a context dependent way.

In this work, we concentrate on the major contributor to poor scaling, "Mixing time" of a markov chain induced by a policy. Mixing times, when ignored, can create myopic biases in the optimization and hence is an impediment to the success in the continual RL problems of greatest interest. We categorize the continual RL problems as Scalable MDPs and formally demonstrate that these exhibit polynomial mixing times. We comment on how exisiting RL algorithms face difficulties in this regime and propose three algorithms which clearly demonstrate sample efficiency.

In this work we study the under-studied parameter in meta learning, "Task Distributions". We show that Model Agnostic Meta-Learning (MAML) is sensitive to task distributions, and learning a curriculum of tasks instead of uniformly sampling helps the adaptation performance substantially.

Transfer of policies from simulation to physical robots is an important open problem in deep reinforcement learning. Prior work has introduced the model-based Neural-Augmented Simulator (NAS) method, which uses task-independent data to create a model of the differences between simulated and real robot. In this work, we show that this method is sensitive to the sampling of motor actions and the control frequency. To overcome this problem, we propose a simple extension based on artificial curiosity. We demonstrate on a physical robot, that this leads to a better exploration of the state space and consequently better transfer performance when compared to the NAS baseline.