Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments

Tien Dung Le, Takashi Komeda, Motoki Takagi

研究成果: Conference contribution

抜粋

Reinforcement Learning (RL) has been widely used to solve problems with a little feedback from environment. Q learning can solve Markov Decision Processes quite well. For Partially Observable Markov Decision Processes, a Recurrent Neural Network (RNN) can be used to approximate Q values. However, learning time for these problems is typically very long. In this paper, we present a method to speed up learning performance in non-Markovian environments by focusing on necessary state-action pairs in learning episodes. Whenever the agent can attain the goal, the agent checks the episode and relearns necessary actions. We use a table, storing minimum number of appearances of states in all successful episodes, to remove unnecessary state-action pairs in a successful episode and to form a min-episode. To verify this method, we performed two experiments: The E maze problem with Time-delay Neural Network and the lighting grid world problem with Long Short Term Memory RNN. Experimental results show that the proposed method enables an agent to acquire a policy with better learning performance compared to the standard method.

元の言語English
ホスト出版物のタイトルProceedings of the 11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007
ページ179-184
ページ数6
出版物ステータスPublished - 2007 12 1
イベント11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007 - Palma de Mallorca, Spain
継続期間: 2007 8 292007 8 31

出版物シリーズ

名前Proceedings of the 11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007

Conference

Conference11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007
Spain
Palma de Mallorca
期間07/8/2907/8/31

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Software

フィンガープリント Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments' の研究トピックを掘り下げます。これらはともに一意のフィンガープリントを構成します。

  • これを引用

    Le, T. D., Komeda, T., & Takagi, M. (2007). Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments. : Proceedings of the 11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007 (pp. 179-184). (Proceedings of the 11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007).