Object manipulation planning and replanning for robots with humans

We introduce new heuristics of HTN (Hierarchical Task Network) planning for mobile robots with two arms/hands that pick and place objects among movable obstacles. Based on our new heuristics, the robot moves obstacles if necessary, picks and places the target objects without collisions. The robot chooses the (right or left) hand to use for each manipulation in order to avoid collisions and reduce the number of obstacle movements. Although we do not use detailed 2D/3D maps and specifications of arms, our heuristics roughly check the collisions. Therefore, we can produce task plans that are executable by lower-level modules. This paper also shows that replanning is effective not only for adapting to human behaviors but also for saving total plan execution time. For example, when a person removes obstacles for the robot, the robot can save time by replanning and omitting this obstacle removal. In another example, suppose that the robot is moving an object to place A, following the user's instruction. If the user changes his/her mind and reinstructs the robot to move the object to another place B, the robot does not have to move the object to A before moving it to B. In these cases, we show that the robot can save much time by task-level replanning.