Reconstruction of a 3D model from single 2D image by GAN

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we propose a method for reconstructing the 3D model from a single 2D image. The current cutting-edge methods for 3D reconstruction use the GAN (Generative Adversarial Network) to generate the model. However, the methods require multiple 2D images to reconstruct the 3D model, because all the information of a real object cannot be obtained from only one side, especially the back of the object is invisible. Since rebuilding a 3D model from a single view is an important issue in practical applications, the system requires the ability to obtain information about the surrounding environment of the object more quickly without the need for the object to move around. Therefore, we propose a method for reconstructing 3D models from an image by learning the relationship between 3D model and 2D image. Mainly this method consists of three parts. The first part is the view layer, observing real-world objects and capturing 2D images. The layer searches the related 2D image of the 3D model that exists in the 3D model library. The second part is the corresponding layer. The 2D image corresponding to the 3D model is taken out, and contrast with real-world 2D images of objects. The 2D cross-section of the 3D model is found as the most similar one to the 2D image of the real-world object. The third part is generative layer that based on the model library to find the corresponding 3D model, reconstructing a 3D model that corresponds to the real object by using the GAN.

Original languageEnglish
Title of host publicationMulti-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings
EditorsRainer Malaka, Manasawee Kaenampornpan, Duc Dung Nguyen, Nicolas Schwind
PublisherSpringer Verlag
Pages226-232
Number of pages7
ISBN (Print)9783030030131
DOIs
Publication statusPublished - 2018 Jan 1
Event12th Multi-disciplinary International Conference on Artificial Intelligence, MIWAI 2018 - Hanoi, Viet Nam
Duration: 2018 Nov 182018 Nov 20

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11248 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other12th Multi-disciplinary International Conference on Artificial Intelligence, MIWAI 2018
CountryViet Nam
CityHanoi
Period18/11/1818/11/20

Keywords

  • 3D reconstruction
  • Corresponding layer
  • GAN
  • Generative layer
  • View layer

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Si, L., & Yasumura, Y. (2018). Reconstruction of a 3D model from single 2D image by GAN. In R. Malaka, M. Kaenampornpan, D. D. Nguyen, & N. Schwind (Eds.), Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings (pp. 226-232). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11248 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-030-03014-8_20

Reconstruction of a 3D model from single 2D image by GAN. / Si, Lei; Yasumura, Yoshiaki.

Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings. ed. / Rainer Malaka; Manasawee Kaenampornpan; Duc Dung Nguyen; Nicolas Schwind. Springer Verlag, 2018. p. 226-232 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11248 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Si, L & Yasumura, Y 2018, Reconstruction of a 3D model from single 2D image by GAN. in R Malaka, M Kaenampornpan, DD Nguyen & N Schwind (eds), Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11248 LNAI, Springer Verlag, pp. 226-232, 12th Multi-disciplinary International Conference on Artificial Intelligence, MIWAI 2018, Hanoi, Viet Nam, 18/11/18. https://doi.org/10.1007/978-3-030-03014-8_20
Si L, Yasumura Y. Reconstruction of a 3D model from single 2D image by GAN. In Malaka R, Kaenampornpan M, Nguyen DD, Schwind N, editors, Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings. Springer Verlag. 2018. p. 226-232. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-03014-8_20
Si, Lei ; Yasumura, Yoshiaki. / Reconstruction of a 3D model from single 2D image by GAN. Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings. editor / Rainer Malaka ; Manasawee Kaenampornpan ; Duc Dung Nguyen ; Nicolas Schwind. Springer Verlag, 2018. pp. 226-232 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e1915cc75e354544930f244c82b4f023,
title = "Reconstruction of a 3D model from single 2D image by GAN",
abstract = "In this paper, we propose a method for reconstructing the 3D model from a single 2D image. The current cutting-edge methods for 3D reconstruction use the GAN (Generative Adversarial Network) to generate the model. However, the methods require multiple 2D images to reconstruct the 3D model, because all the information of a real object cannot be obtained from only one side, especially the back of the object is invisible. Since rebuilding a 3D model from a single view is an important issue in practical applications, the system requires the ability to obtain information about the surrounding environment of the object more quickly without the need for the object to move around. Therefore, we propose a method for reconstructing 3D models from an image by learning the relationship between 3D model and 2D image. Mainly this method consists of three parts. The first part is the view layer, observing real-world objects and capturing 2D images. The layer searches the related 2D image of the 3D model that exists in the 3D model library. The second part is the corresponding layer. The 2D image corresponding to the 3D model is taken out, and contrast with real-world 2D images of objects. The 2D cross-section of the 3D model is found as the most similar one to the 2D image of the real-world object. The third part is generative layer that based on the model library to find the corresponding 3D model, reconstructing a 3D model that corresponds to the real object by using the GAN.",
keywords = "3D reconstruction, Corresponding layer, GAN, Generative layer, View layer",
author = "Lei Si and Yoshiaki Yasumura",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-03014-8_20",
language = "English",
isbn = "9783030030131",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "226--232",
editor = "Rainer Malaka and Manasawee Kaenampornpan and Nguyen, {Duc Dung} and Nicolas Schwind",
booktitle = "Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings",

}

TY - GEN

T1 - Reconstruction of a 3D model from single 2D image by GAN

AU - Si, Lei

AU - Yasumura, Yoshiaki

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In this paper, we propose a method for reconstructing the 3D model from a single 2D image. The current cutting-edge methods for 3D reconstruction use the GAN (Generative Adversarial Network) to generate the model. However, the methods require multiple 2D images to reconstruct the 3D model, because all the information of a real object cannot be obtained from only one side, especially the back of the object is invisible. Since rebuilding a 3D model from a single view is an important issue in practical applications, the system requires the ability to obtain information about the surrounding environment of the object more quickly without the need for the object to move around. Therefore, we propose a method for reconstructing 3D models from an image by learning the relationship between 3D model and 2D image. Mainly this method consists of three parts. The first part is the view layer, observing real-world objects and capturing 2D images. The layer searches the related 2D image of the 3D model that exists in the 3D model library. The second part is the corresponding layer. The 2D image corresponding to the 3D model is taken out, and contrast with real-world 2D images of objects. The 2D cross-section of the 3D model is found as the most similar one to the 2D image of the real-world object. The third part is generative layer that based on the model library to find the corresponding 3D model, reconstructing a 3D model that corresponds to the real object by using the GAN.

AB - In this paper, we propose a method for reconstructing the 3D model from a single 2D image. The current cutting-edge methods for 3D reconstruction use the GAN (Generative Adversarial Network) to generate the model. However, the methods require multiple 2D images to reconstruct the 3D model, because all the information of a real object cannot be obtained from only one side, especially the back of the object is invisible. Since rebuilding a 3D model from a single view is an important issue in practical applications, the system requires the ability to obtain information about the surrounding environment of the object more quickly without the need for the object to move around. Therefore, we propose a method for reconstructing 3D models from an image by learning the relationship between 3D model and 2D image. Mainly this method consists of three parts. The first part is the view layer, observing real-world objects and capturing 2D images. The layer searches the related 2D image of the 3D model that exists in the 3D model library. The second part is the corresponding layer. The 2D image corresponding to the 3D model is taken out, and contrast with real-world 2D images of objects. The 2D cross-section of the 3D model is found as the most similar one to the 2D image of the real-world object. The third part is generative layer that based on the model library to find the corresponding 3D model, reconstructing a 3D model that corresponds to the real object by using the GAN.

KW - 3D reconstruction

KW - Corresponding layer

KW - GAN

KW - Generative layer

KW - View layer

UR - http://www.scopus.com/inward/record.url?scp=85057082141&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057082141&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-03014-8_20

DO - 10.1007/978-3-030-03014-8_20

M3 - Conference contribution

SN - 9783030030131

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 226

EP - 232

BT - Multi-disciplinary Trends in Artificial Intelligence - 12th International Conference, MIWAI 2018, Proceedings

A2 - Malaka, Rainer

A2 - Kaenampornpan, Manasawee

A2 - Nguyen, Duc Dung

A2 - Schwind, Nicolas

PB - Springer Verlag

ER -