Policy gradient reinforcement learning with environmental dynamics and action-values in policies

Seiji Ishihara, Harukazu Igarashi

研究成果: Conference contribution

抜粋

The knowledge concerning an agent's policies consists of two types: the environmental dynamics for defining state transitions around the agent, and the behavior knowledge for solving a given task. However, these two types of information, which are usually combined into state-value or action-value functions, are learned together by conventional reinforcement learning. If they are separated and learned independently, either might be reused in other tasks or environments. In our previous work, we presented learning rules using policy gradients with an objective function, which consists of two types of parameters representing environmental dynamics and behavior knowledge, to separate the learning for each type. In such a learning framework, state-values were used as an example of the set of parameters corresponding to behavior knowledge. By the simulation results on a pursuit problem, our method properly learned hunter-agent policies and reused either bit of knowledge. In this paper, we adopt action-values as a set of parameters in the objective function instead of state-values and present learning rules for the function. Simulation results on the same pursuit problem as in our previous work show that such parameters and learning rules are also useful.

元の言語English
ホスト出版物のタイトルKnowledge-Based and Intelligent Information and Engineering Systems - 15th International Conference, KES 2011, Proceedings
ページ120-130
ページ数11
エディションPART 1
DOI
出版物ステータスPublished - 2011 9 29
イベント15th International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, KES 2011 - Kaiserslautern, Germany
継続期間: 2011 9 122011 9 14

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
番号PART 1
6881 LNAI
ISSN(印刷物)0302-9743
ISSN(電子版)1611-3349

Conference

Conference15th International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, KES 2011
Germany
Kaiserslautern
期間11/9/1211/9/14

    フィンガープリント

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

これを引用

Ishihara, S., & Igarashi, H. (2011). Policy gradient reinforcement learning with environmental dynamics and action-values in policies. : Knowledge-Based and Intelligent Information and Engineering Systems - 15th International Conference, KES 2011, Proceedings (PART 1 版, pp. 120-130). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 巻数 6881 LNAI, 番号 PART 1). https://doi.org/10.1007/978-3-642-23851-2_13