학술논문

Benchmark for Skill Learning from Demonstration: Impact of User Experience, Task Complexity, and Start Configuration on Performance
Document Type
Conference
Source
2020 IEEE International Conference on Robotics and Automation (ICRA) Robotics and Automation (ICRA), 2020 IEEE International Conference on. :7561-7567 May, 2020
Subject
Robotics and Control Systems
Task analysis
Robots
Trajectory
Videos
Pressing
Benchmark testing
Complexity theory
Language
ISSN
2577-087X
Abstract
We contribute a study benchmarking the performance of multiple motion-based learning from demonstration approaches. Given the number and diversity of existing methods, it is critical that comprehensive empirical studies be performed comparing the relative strengths of these techniques. In particular, we evaluate four approaches based on properties an end user may desire for real-world tasks. To perform this evaluation, we collected data from nine participants, across four manipulation tasks. The resulting demonstrations were used to train 180 task models and evaluated on 720 task reproductions on a physical robot. Our results detail how i) complexity of the task, ii) the expertise of the human demonstrator, and iii) the starting configuration of the robot affect task performance. The collected dataset of demonstrations, robot executions, and evaluations are publicly available. Research insights and guidelines are also provided to guide future research and deployment choices about these approaches.