Two ingredients in our approach for combining task and motion planning:

1) Geometric Backtracking

Geometric backtracking consists in reconsidering the choices made at the geometric level when an action fails. The first video illustrates how geometric configurations are sampled (red spots) for each action. The second video shows geometric backtracking in action (top) versus symbolic backtracking (bottom). The last video show on the real robot an example where geometric backtracking is needed: the first two cups have to be placed carefully, so that it is possible to place the third cup.


2) A planning domain capable of coarse geometric reasoning

Operators contain coarse geometric representations, e.g., side (left/right), grasp type (top, side, bottom), coarse orientation (upright/up-side-down, x/y/z). The exact positions/orientations are decided at the geometric level. Hence the same task can be decomposed in different ways at the task level (video 1) Video 2 and 3 illustrate this on the real robot Justin: the same task is decomposed in a completely different way depending on the initial configuration.


Other experiments

A complex re-grasping sequence

The task is to move the bottle from the torus on the right into the torus on the left. Problem: inside a torus, the bottle can only be grasped by the top. There is no table to place the bottle in-between. This task requires 3 re-grasp operations to be solved:

Moving obstacles

The task is to place both cups into the box. The milk boxes have to be moved before moving the cups. During the experiment on the real robot (video 2) the planner found a motion plan for moving the second cup without moving the obstacle.


Current work: planning with GraphPlan

GraphPlan presents nice features for combining task and motion planning: (i) the concept of mutex can be generalized to geometric mutex, which allows for a tighter coupling between task and motion planning, hence solving more complex tasks. (ii) GraphPlan allows to generate plans with parallel actions.


Failures: working with the real robot is never easy!