Intelligent Field-of-View Prediction

Unlike traditional 2D video, in the 360-degree video (also referred to as “omnidirectional video”), only a portion of the entire scene is watched at a time and users keep exploring and navigating the new view direction constantly according to the video contents and viewers’ interests. We investigated the viewing behaviors of 360-degree videos and designed intelligent Field-of-View (FoV) prediction algorithm.

We developed effective algorithms for predicting user FoVs, based on the past FoV trajectory and the audio and visual content through novel deep learning architectures. We further propose to study personalized FoV prediction based on other users’ view trajectories under the framework of recommender systems.

• Chenge Li, Weixi Zhang, Yong Liu, Yao Wang, “Very Long Term Field of View Prediction for 360-degree Video Streaming“, invited paper in IEEE Multimedia Information Processing and Retrieval (MIPR), 2019.
• Fanyi Duanmu, Yixiang Mao, Shuai Liu, Sumanth Srinivasan, and Yao Wang, “A Subjective Study of Viewer Navigation Behaviors When Watching 360-degree Videos on Computers,” in Proc. of IEEE International Conference on Multimedia Expo (ICME), San Diego, California, USA, 2018.