학술논문
Hierarchical Reinforcement Learning With Automatic Sub-Goal Identification
Document Type
Periodical
Author
Source
IEEE/CAA Journal of Automatica Sinica IEEE/CAA J. Autom. Sinica Automatica Sinica, IEEE/CAA Journal of. 8(10):1686-1696 Oct, 2021
Subject
Language
ISSN
2329-9266
2329-9274
2329-9274
Abstract
In reinforcement learning an agent may explore ineffectively when dealing with sparse reward tasks where finding a reward point is difficult. To solve the problem, we propose an algorithm called hierarchical deep reinforcement learning with automatic sub-goal identification via computer vision (HADS) which takes advantage of hierarchical reinforcement learning to alleviate the sparse reward problem and improve efficiency of exploration by utilizing a sub-goal mechanism. HADS uses a computer vision method to identify sub-goals automatically for hierarchical deep reinforcement learning. Due to the fact that not all sub-goal points are reachable, a mechanism is proposed to remove unreachable sub-goal points so as to further improve the performance of the algorithm. HADS involves contour recognition to identify sub-goals from the state image where some salient states in the state image may be recognized as sub-goals, while those that are not will be removed based on prior knowledge. Our experiments verified the effect of the algorithm.