Research

Demo2Code leverages LLMs to translate demonstrations to robot task code via an extended chain-of-thought that recursively summarizes demos to specification, and recursively expands specification to code.

We propose a novel, lazy approach that addresses two fundamental challenges in Model-based Reinforcement Learning (MBRL): the computational expense of repeatedly finding a good policy in the learned model, and the objective mismatch between model fitting and policy computation.

We explore inverse reinforcement learning and show that leveraging the state distribution of the expert can significantly reduce the complexities of the RL problem, theoretically providing an exponential speedup and practically enhancing performance in continuous control tasks.

We investigate sequential decision making with "Impossibly Good" experts possessing privileged information, propose necessary criteria for an optimal policy recovery within limited information, and introduce a novel approach, ELF Distillation, outperforming baselines in Minigrid and Vizdoom environments.

We study imitation learning when the expert has privileged information and show that on-policy algorithms provably learn to recover from their initially suboptimal actions, while off-policy methods naively repeat the past action.

Imitation learning from noisy experts leads to biased policies! Replay estimation fixes this by smoothing the expert by repeatedly executing cached expert actions in a stochastic simulator and imitating that.

We look at imitation learning where the demonstrators have varying quality and seek to induce behavior that is unambiguously better (i.e., Pareto dominant or minimally subdominant) than all human demonstrations.

All of imitation learning can be reduced to a game between a learner (generator) and a value function (discriminator) where the payoff is the performance difference between learner and expert.

Blend model predictive control (MPC) with learned value estimates to trade-off MPC model errors with learner approximation errors.

Not all imitation learning problems are alike -- some are easy (do behavior cloning), some are hard (call interactive expert), and some are just right (just need a simulator).

In Bayesian RL, while solving the belief MDP is hard, solving individual latent MDP is easy. Combine value functions from each MDP along with a learned residual belief policy.

Instead of sampling from an informed set ellipse (high recall, low precision), samples from sets of sub-ellipses (lower recall, higher precision).

Many old (and new!) imitation learning algorithms are simply minimizing various f-divergences estimates between the expert and the learner trajectory distributions.

How can we learn from human interventions? Every intervention reveals some information about expert's implicit value function. Infer this function and optimize it.

We can model multi-modal feedback from human (demonstrations, interventions, verbal) as a stream of losses that can be minimized using any no-regret online learning algorithm.

A full-stack mobile manipulator that autonomously navigates and manipulates objects in the wild.

Anytime motion planning can be viewed through a Bayesian lens where we are initially uncertain about the shortest path, and must probe the environment to progressively yield shorter and shorter paths.

Unified framework for interleaving search and edge evaluation to provably minimize total planning time.

Casts manipulation under occlusion as a search on a graph where feasibility of an edge is only revealed when an agent attempts to traverse it. Use Bayesian prior to explore exploit.

The laziest search is one that checks the minimal number of edges to eliminate all potential shortest paths. We use imitation learning to imitate such oracles to learn truly lazy planners.

Learn a sampling distribution that creates graphs that are sparse (for speedy search) but with nodes carefully placed at bottleneck regions (to cover optimal paths).

Learn a policy that directly maps state and belief over MDPs to action. Leverages the fact that belief can be compressed significantly.

Computes a near-optimal value function by covering the continuous state-belief-action space with a finite set of representative samples and exploiting the Lipschitz continuity of the value function

A selfie drone that can film a target moving in a cluttered environment with almost no prior information.

Given a prior over possible worlds, learn a decision tree to near-optimally collpase uncertainty to compute a feasible path.

Given a graph with N edges that could independently be 0/1, and a prior belief, check an optimal number of edges till you find a feasible path.

Train planners (that operate on partial information) to imitate clairvoyant planners (that have full information) to choose optimal planning decisions. (applies to heuristic search, exploration planning, etc)

Search algorithms use heuristics to balance exploration, i.e., discovering promising new states, and exploitation, i.e., expanding the current best state. Learn heuristics by imitating optimal planners.

POMDPs are hard. But MDPs are relatively easy. Train POMDP policies by imitating MPD oracles to get good, and sometimes near-optimal, POMDP policies.

How efficiently a robot can map a new area depends on the geometry of the world. We show how learning can be leveraged to design more efficient information gathering policies.

Anytime motion planning by -- select a subgraph, search it, and use the results to select a better subgraph till the shortest path on the original dense graph is found.

Interleave search and optimization by applying CHOMP to only a subset of promising edges in a BIT* search tree.

Train a learner to produce a diverse set of planner options (initializations, heuristics, hyperparameters) that can be run in parallel such that at least one has good performance.

How fast can a drone fly in a forest even if it knew the location of every single tree? We answer this question with the help of percolation theory on random graphs.

The first approach to planning safe, real-time trajectories for a full-scale, autonomous helicopter from takeoff to landing, with more than 700 flight test hours.

Can a single planner solve all planning problems? Hedge our bets and predict an ensemble of diverse planners that can be run in parallel such that at least one solves the problem.

A nonlinear projection operator as a control Lyapunov function that takes an optimized workspace trajectory and projects it to a configuration space trajectory with guarantees on sub-optimality.

A fully autonomous UAV that can map riverines stretching over several hundred-meters of tight winding rivers. </td> </tr>

A fast, 3D sparse visibility graph that is orders of magnitude faster than sampling-based or discrete search. Key idea -- shortest path is a geodesic that can only deviate around surface normals. </td> </tr>

A planning system that lands a helicopter safely when it's engines fail -- chooses a set of safe landing sites, plans a diverse set of routes, and controls vehicle to touchdown.

Find multiple, diverse, near-optimal solutions to a planning problem. Define equivalence classes in path space and not allow the search tree to have 2 paths in the same class.