Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments

Tien Dung Le, Takashi Komeda, Motoki Takagi

研究成果: Conference contribution

抄録

Reinforcement Learning (RL) has been widely used to solve problems with a little feedback from environment. Q learning can solve Markov Decision Processes quite well. For Partially Observable Markov Decision Processes, a Recurrent Neural Network (RNN) can be used to approximate Q values. However, learning time for these problems is typically very long. In this paper, we present a method to speed up learning performance in non-Markovian environments by focusing on necessary state-action pairs in learning episodes. Whenever the agent can attain the goal, the agent checks the episode and relearns necessary actions. We use a table, storing minimum number of appearances of states in all successful episodes, to remove unnecessary state-action pairs in a successful episode and to form a min-episode. To verify this method, we performed two experiments: The E maze problem with Time-delay Neural Network and the lighting grid world problem with Long Short Term Memory RNN. Experimental results show that the proposed method enables an agent to acquire a policy with better learning performance compared to the standard method.

本文言語English
ホスト出版物のタイトルProceedings of the 11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007
ページ179-184
ページ数6
出版ステータスPublished - 2007
イベント11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007 - Palma de Mallorca, Spain
継続期間: 2007 8 292007 8 31

出版物シリーズ

名前Proceedings of the 11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007

Conference

Conference11th IASTED International Conference on Artificial Intelligence and Soft Computing, ASC 2007
国/地域Spain
CityPalma de Mallorca
Period07/8/2907/8/31

ASJC Scopus subject areas

  • 人工知能
  • コンピュータ サイエンスの応用
  • ソフトウェア

フィンガープリント

「Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル