Autonomous grading work using deep reinforcement learning based control

Masayuki Nakatani, Zeyuan Sun, Yutaka Uchimura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The field of artificial intelligence (AI) has advanced significantly over the years. One of its achievements is the deep reinforcement learning algorithm using which AI can play some Atari 2600 games better than humans. In this paper, optimal route of construction machines such as bulldozers is modeled based on deep reinforcement learning. The aim of this study is to apply deep reinforcement learning to a grading machine to enable it to grade various surface types autonomously. A simple grading simulator is created to simulate the grading task. In addition, the overall scenario is made visible to the network by entering the simulation into the network so that human operators can construct suitable ground path from the surrounding sediment environment. The method is evaluated with the grading simulator, and the agent is shown to exhibit desirable control behavior and fulfill the goals of the simple grading simulation. Despite the environment being virtual, the simulation results demonstrate the feasibility of the proposed approach.

Original languageEnglish
Title of host publicationProceedings
Subtitle of host publicationIECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages5068-5073
Number of pages6
ISBN (Electronic)9781509066841
DOIs
Publication statusPublished - 2018 Dec 26
Event44th Annual Conference of the IEEE Industrial Electronics Society, IECON 2018 - Washington, United States
Duration: 2018 Oct 202018 Oct 23

Publication series

NameProceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society

Conference

Conference44th Annual Conference of the IEEE Industrial Electronics Society, IECON 2018
CountryUnited States
CityWashington
Period18/10/2018/10/23

Fingerprint

Reinforcement learning
Artificial intelligence
Simulators
Virtual reality
Learning algorithms
Sediments

ASJC Scopus subject areas

  • Energy Engineering and Power Technology
  • Electrical and Electronic Engineering
  • Industrial and Manufacturing Engineering
  • Control and Optimization

Cite this

Nakatani, M., Sun, Z., & Uchimura, Y. (2018). Autonomous grading work using deep reinforcement learning based control. In Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society (pp. 5068-5073). [8591189] (Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IECON.2018.8591189

Autonomous grading work using deep reinforcement learning based control. / Nakatani, Masayuki; Sun, Zeyuan; Uchimura, Yutaka.

Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society. Institute of Electrical and Electronics Engineers Inc., 2018. p. 5068-5073 8591189 (Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nakatani, M, Sun, Z & Uchimura, Y 2018, Autonomous grading work using deep reinforcement learning based control. in Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society., 8591189, Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society, Institute of Electrical and Electronics Engineers Inc., pp. 5068-5073, 44th Annual Conference of the IEEE Industrial Electronics Society, IECON 2018, Washington, United States, 18/10/20. https://doi.org/10.1109/IECON.2018.8591189
Nakatani M, Sun Z, Uchimura Y. Autonomous grading work using deep reinforcement learning based control. In Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society. Institute of Electrical and Electronics Engineers Inc. 2018. p. 5068-5073. 8591189. (Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society). https://doi.org/10.1109/IECON.2018.8591189
Nakatani, Masayuki ; Sun, Zeyuan ; Uchimura, Yutaka. / Autonomous grading work using deep reinforcement learning based control. Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 5068-5073 (Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society).
@inproceedings{aeddc90af96949a9a54e81b01bd094c8,
title = "Autonomous grading work using deep reinforcement learning based control",
abstract = "The field of artificial intelligence (AI) has advanced significantly over the years. One of its achievements is the deep reinforcement learning algorithm using which AI can play some Atari 2600 games better than humans. In this paper, optimal route of construction machines such as bulldozers is modeled based on deep reinforcement learning. The aim of this study is to apply deep reinforcement learning to a grading machine to enable it to grade various surface types autonomously. A simple grading simulator is created to simulate the grading task. In addition, the overall scenario is made visible to the network by entering the simulation into the network so that human operators can construct suitable ground path from the surrounding sediment environment. The method is evaluated with the grading simulator, and the agent is shown to exhibit desirable control behavior and fulfill the goals of the simple grading simulation. Despite the environment being virtual, the simulation results demonstrate the feasibility of the proposed approach.",
author = "Masayuki Nakatani and Zeyuan Sun and Yutaka Uchimura",
year = "2018",
month = "12",
day = "26",
doi = "10.1109/IECON.2018.8591189",
language = "English",
series = "Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "5068--5073",
booktitle = "Proceedings",

}

TY - GEN

T1 - Autonomous grading work using deep reinforcement learning based control

AU - Nakatani, Masayuki

AU - Sun, Zeyuan

AU - Uchimura, Yutaka

PY - 2018/12/26

Y1 - 2018/12/26

N2 - The field of artificial intelligence (AI) has advanced significantly over the years. One of its achievements is the deep reinforcement learning algorithm using which AI can play some Atari 2600 games better than humans. In this paper, optimal route of construction machines such as bulldozers is modeled based on deep reinforcement learning. The aim of this study is to apply deep reinforcement learning to a grading machine to enable it to grade various surface types autonomously. A simple grading simulator is created to simulate the grading task. In addition, the overall scenario is made visible to the network by entering the simulation into the network so that human operators can construct suitable ground path from the surrounding sediment environment. The method is evaluated with the grading simulator, and the agent is shown to exhibit desirable control behavior and fulfill the goals of the simple grading simulation. Despite the environment being virtual, the simulation results demonstrate the feasibility of the proposed approach.

AB - The field of artificial intelligence (AI) has advanced significantly over the years. One of its achievements is the deep reinforcement learning algorithm using which AI can play some Atari 2600 games better than humans. In this paper, optimal route of construction machines such as bulldozers is modeled based on deep reinforcement learning. The aim of this study is to apply deep reinforcement learning to a grading machine to enable it to grade various surface types autonomously. A simple grading simulator is created to simulate the grading task. In addition, the overall scenario is made visible to the network by entering the simulation into the network so that human operators can construct suitable ground path from the surrounding sediment environment. The method is evaluated with the grading simulator, and the agent is shown to exhibit desirable control behavior and fulfill the goals of the simple grading simulation. Despite the environment being virtual, the simulation results demonstrate the feasibility of the proposed approach.

UR - http://www.scopus.com/inward/record.url?scp=85061556154&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061556154&partnerID=8YFLogxK

U2 - 10.1109/IECON.2018.8591189

DO - 10.1109/IECON.2018.8591189

M3 - Conference contribution

T3 - Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society

SP - 5068

EP - 5073

BT - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -