Fixation prediction for 360° video streaming in head-mounted virtual reality

Ching Ling Fan, Jean Lee, Wen Chih Lo, Chun-Ying Huang, Kuan Ta Chen, Cheng Hsin Hsu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

100 Scopus citations

Abstract

We study the problem of predicting the Field-of-Views (FoVs) of viewers watching 360° videos using commodity Head-Mounted Displays (HMDs). Existing solutions either use the viewer's current orientation to approximate the FoVs in the future, or extrapolate future FoVs using the historical orientations and dead-reckoning algorithms. In this paper, we develop fixation prediction networks that concurrently leverage sensor- and content related features to predict the viewer fixation in the future, which is quite different from the solutions in the literature. The sensor-related features include HMD orientations, while the content-related features include image saliency maps and motion maps. We build a 360° video streaming testbed to HMDs, and recruit twenty-five viewers to watch ten 360° videos. We then train and validate two design alternatives of our proposed networks, which allows us to identify the better-performing design with the optimal parameter settings. Trace-driven simulation results show the merits of our proposed fixation prediction networks compared to the existing solutions, including: (i) lower consumedbandwidth, (ii) shorter initial buffering time, and (iii) short running time.

Original languageEnglish
Title of host publicationProceedings of the 27th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV 2017
PublisherAssociation for Computing Machinery, Inc
Pages67-72
Number of pages6
ISBN (Electronic)9781450350037
DOIs
StatePublished - 20 Jun 2017
Event27th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV 2017 - Taipei, Taiwan
Duration: 23 Jun 2017 → …

Publication series

NameProceedings of the 27th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV 2017

Conference

Conference27th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV 2017
CountryTaiwan
CityTaipei
Period23/06/17 → …

Keywords

  • 360° Video
  • HMD
  • Prediction
  • Virtual reality

Fingerprint Dive into the research topics of 'Fixation prediction for 360° video streaming in head-mounted virtual reality'. Together they form a unique fingerprint.

Cite this