학술논문
A Content-based Viewport Prediction Framework for 360° Video Using Personalized Federated Learning and Fusion Techniques
Document Type
Conference
Author
Source
2023 IEEE International Conference on Multimedia and Expo (ICME) ICME Multimedia and Expo (ICME), 2023 IEEE International Conference on. :654-659 Jul, 2023
Subject
Language
ISSN
1945-788X
Abstract
Viewport prediction is a key enabler for 360° video streaming over wireless networks. To improve the prediction accuracy, a common approach is to use a content-based viewport prediction model. Saliency detection based on traditional convolutional neural networks (CNNs) suffers from distortion due to equirectangular projection. Also, the viewers may have their own viewing behavior and are not willing to share their historical head movement with others. To address the aforementioned issues, in this paper, we first develop a saliency detection model using a spherical CNN (SPCNN). Then, we train the viewers’ head movement prediction model using personalized federated learning (PFL). Finally, we propose a content-based viewport prediction framework by integrating the video saliency map and the head orientation map of each viewer using fusion techniques. The experimental results show that our proposed framework provides higher average accuracy and precision when compared with three state-of-the-art algorithms from the literature.