Subtitle-based Viewport Prediction for 360-degree Virtual Tourism Video

Chuanzhe Jing, Tho Nguyen Duc, Phan Xuan Tan, Eiji Kamioka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

360-degree streaming videos can provide a rich immersive experiences to the users. However, it requires an extremely high bandwidth network. One of the common solutions for saving bandwidth consumption is to stream only a portion of video covered by the user's viewport. To do that, the user's viewpoint prediction is indispensable. In existing viewport prediction methods, they mainly concentrate on the user's head movement trajectory and video saliency. None of them consider navigation information contained in the video, which can turn the attention of the user to specific regions in the video with high probability. Such information can be included in video subtitles, especially the one in 360-degree virtual tourism videos. This fact reveals the potential contribution of video subtitles to viewport prediction. Therefore, in this paper, a subtitle-based viewport prediction model for 360-degree virtual tourism videos is proposed. This model leverages the navigation information in the video subtitles in addition to head movement trajectory and video saliency, to improve the prediction accuracy. The experimental results demonstrate that the proposed model outperforms baseline methods which only use head movement trajectory and video saliency for viewport prediction.

Original languageEnglish
Title of host publication13th International Conference on Information, Intelligence, Systems and Applications, IISA 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665463904
DOIs
Publication statusPublished - 2022
Event13th International Conference on Information, Intelligence, Systems and Applications, IISA 2022 - Corfu, Greece
Duration: 2022 Jul 182022 Jul 20

Publication series

Name13th International Conference on Information, Intelligence, Systems and Applications, IISA 2022

Conference

Conference13th International Conference on Information, Intelligence, Systems and Applications, IISA 2022
Country/TerritoryGreece
CityCorfu
Period22/7/1822/7/20

Keywords

  • 360-degree video
  • video subtitles
  • viewport prediction
  • virtual reality
  • virtual tourism videos

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems
  • Software
  • Information Systems and Management
  • Control and Optimization
  • Communication

Fingerprint

Dive into the research topics of 'Subtitle-based Viewport Prediction for 360-degree Virtual Tourism Video'. Together they form a unique fingerprint.

Cite this