Dagstuhl Seminar 15411
Multimodal Manipulation Under Uncertainty
( Oct 04 – Oct 09, 2015 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/15411
Organizers
- Jan Peters (TU Darmstadt, DE)
- Justus Piater (Universität Innsbruck, AT)
- Robert Platt (Northeastern University - Boston, US)
- Siddhartha Srinivasa (Carnegie Mellon University, US)
Contact
- Andreas Dolzmann (for scientific matters)
- Dagmar Glaser (for administrative matters)
Schedule
While robots have been used for decades to perform highly specialized tasks in engineered environments, robotic manipulation is still crude and clumsy in settings not specifically designed for robots. There is a huge gap between human and robot capabilities, including actuation, perception, and reasoning. However, recent developments such as low-cost manipulators and sensing technologies place the field in a good position to make progress on robot manipulation in unstructured environments. Various techniques are emerging for computing or inferring grasp configurations based on object identity, shape, or appearance, using simple grippers and robot hands.
Beyond grasping, a key ingredient of sophisticated manipulation is the management of state information and its uncertainty. One approach to handling uncertainty is to develop grasping and manipulation skills that are robust to environmental variation. Another approach is to develop methods of interacting with the environment in order to gain task-relevant information, for example, by touching, pushing, changing viewpoint, etc. Managing state information and uncertainty will require a tight combination of perception and planning. When the sensor evidence is unambiguous, the robot needs to be able to recognize that and perform the task accurately and efficiently. When greater uncertainty is present, the robot needs to adjust its actions so that they will succeed in the worst case or it needs to gain additional information in order to improve its situation. Different sensing modalities as well as world models can often be combined to good effect due to their complementary properties.
This Seminar seeks to formulate research questions and agendas in order to accelerate progress towards robust manipulation under uncertainty, including topics such as the following:
- How to synthesize robust grasping/manipulation actions or behaviors in the presence of state uncertainty?
- How to integrate multi-modal information from vision, range, and tactile sensors into a single consistent state estimate?
- How to reduce state uncertainty by synthesizing specific sensing actions or planning information-gathering activities?
- How to synthesize robust, reactive manipulation in the presence of distractors such as other agents or clutter?
- How to use nonprehensile, physics-based strategies to robustly grasp and manipulate in clutter?
Multi-modal manipulation under uncertainty is a broad research agenda that requires strong synergy between theoreticians and practitioners of robotic manipulation, motion planning, control, perception, and machine learning. Our objective is to bring together key contributors in each of these fields and cultivate discussion and collaboration in order to create synergies and accelerate progress. Results of the Seminar will include
- a characterization of concrete capabilities of manipulation under uncertainty that are needed for high-level applications such as flexible manufacturing, service or household robotics;
- a clear understanding of the state of the art in these areas, including solved problems and open questions;
- concrete research questions whose solution will move the field significantly forward, as well as research agendas designed to address them.
While robots have been used for decades to perform highly specialized tasks in engineered environments, robotic manipulation is still crude and clumsy in settings not specifically designed for robots. There is a huge gap between human and robot capabilities, including actuation, perception, and reasoning. However, recent developments such as low-cost manipulators and sensing technologies place the field in a good position to make progress on robot manipulation in unstructured environments. Various techniques are emerging for computing or inferring grasp configurations based on object identity, shape, or appearance, using simple grippers and robot hands.
Beyond grasping, a key ingredient of sophisticated manipulation is the management of state information and its uncertainty. One approach to handling uncertainty is to develop grasping and manipulation skills that are robust to environmental variation. Another approach is to develop methods of interacting with the environment in order to gain task-relevant information, for example, by touching, pushing, changing viewpoint, etc. Managing state information and uncertainty will require a tight combination of perception and planning. When the sensor evidence is unambiguous, the robot needs to be able to recognize that and perform the task accurately and efficiently. When greater uncertainty is present, the robot needs to adjust its actions so that they will succeed in the worst case or it needs to gain additional information in order to improve its situation. Different sensing modalities as well as world models can often be combined to good effect due to their complementary properties.
This seminar discussed research questions and agendas in order to accelerate progress towards robust manipulation under uncertainty, including topics such as the following:
- Is there a master algorithm or are there infinitely many algorithms that solve specialized problems? Can we decompose multimodal manipulation under uncertainty into I/O boxes? If so, what would these be?
- Do we prefer rare-feedback / strong-model or frequent-feedback / weak-model approaches? Is there a sweet spot in between? Is this the way to think about underactuated hands?
- What are useful perceptual representations for manipulation? What should be the relationship between perception and action? What kind of perception is required for reactive systems, planning systems, etc.?
- How do we do deformable-object manipulation? What planning methods, what types of models are appropriate?
- How should we be benchmarking manipulation? What kind of objects; what kind of tasks should be used?
- How should humans and robots collaborate on manipulation tasks? This question includes humans collaborating with autonomous robots as well as partially-autonomous robots acting under human command.
In the area of perception, we concluded that the design of representations remains a central issue. While it would be beneficial to develop representations that encompass multiple levels of abstraction in a coherent fashion, it is also clear that specific visual tasks suggest distinct visual representations.
How useful or limiting is the engineering approach of decomposing functionality into separate modules? Although this question was heavily debated, the majority view among seminar participants was that modules are useful to keep design complexity manageable for humans, and to keep the event horizon manageable for planning systems. It seems that to build more flexible and powerful systems, modules will need to be more strongly interconnected than they typically are these days. Fundamental challenges lie in the specification of each module and of their interconnections. There is a lot of room for creative innovation in this area.
Benchmarking questions were discussed chiefly in the context of the YCB Object Set. Specific benchmarks were suggested and discussed, covering perception and planning in the context of autonomous manipulation.
Creative Commons BY 3.0 Unported license
Jan Peters, Justus Piater, Robert Platt, and Siddhartha Srinivasa
- Ron Alterovitz (University of North Carolina at Chapel Hill, US) [dblp]
- Brenna D. Argall (Northwestern University - Evanston, US) [dblp]
- Yasemin Bekiroglu (KTH Royal Institute of Technology - Stockholm, SE) [dblp]
- Kostas Bekris (Rutgers University - Piscataway, US) [dblp]
- Dmitry Berenson (Worcester Polytechnic Institute, US) [dblp]
- Bastian Bischoff (Robert Bosch GmbH - Stuttgart, DE) [dblp]
- Jeannette Bohg (MPI für Intelligente Systeme - Tübingen, DE) [dblp]
- Oliver Brock (TU Berlin, DE) [dblp]
- Matei Ciocarlie (Columbia University, US) [dblp]
- Fan Dai (ABB Corporate Research, DE) [dblp]
- Renaud Detry (University of Liège, BE) [dblp]
- Mehmet R. Dogar (University of Leeds, GB) [dblp]
- Aaron M. Dollar (Yale University, US) [dblp]
- Roderic A. Grupen (University of Massachusetts - Amherst, US) [dblp]
- Simon Hangl (Universität Innsbruck, AT) [dblp]
- David Hsu (National University of Singapore, SG) [dblp]
- Leslie Pack Kaelbling (MIT - Cambridge, US) [dblp]
- Marek S. Kopicki (University of Birmingham, GB) [dblp]
- Dirk Kraft (University of Southern Denmark - Odense, DK) [dblp]
- Norbert Krüger (University of Southern Denmark - Odense, DK) [dblp]
- Ville Kyrki (Aalto University, FI) [dblp]
- Ales Leonardis (University of Birmingham, GB) [dblp]
- Shuai Li (Rensselaer Polytechnic, US) [dblp]
- Maxim Likhachev (Carnegie Mellon University, US) [dblp]
- Tomás Lozano-Pérez (MIT - Cambridge, US) [dblp]
- Matthew T. Mason (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Mark Moll (Rice University - Houston, US) [dblp]
- Duy Nguyen-Tuong (Robert Bosch GmbH - Schwieberdingen, DE) [dblp]
- Erhan Öztop (Özyegin University - Istanbul, TR) [dblp]
- Jan Peters (TU Darmstadt, DE) [dblp]
- Justus Piater (Universität Innsbruck, AT) [dblp]
- Robert Platt (Northeastern University - Boston, US) [dblp]
- Maximo A. Roa (German Aerospace Center-DLR, DE) [dblp]
- Veronica Santos (UCLA, US) [dblp]
- Siddhartha Srinivasa (Carnegie Mellon University, US) [dblp]
- Ales Ude (Jozef Stefan Institute - Ljubljana, SI) [dblp]
- Francisco Valero Cuevas (USC - Los Angeles, US) [dblp]
- Jeremy L. Wyatt (University of Birmingham, GB) [dblp]
- Michael Zillich (TU Wien, AT) [dblp]
Classification
- artificial intelligence / robotics
Keywords
- robotics
- manipulation
- uncertainty
- perception
- computer vision
- range sensing
- tactile sensing
