학술논문
Multiple Human Tracking Using Deep Learning with Shadow Clues
Document Type
Conference
Author
Source
2023 9th International Conference on Virtual Reality (ICVR) Virtual Reality (ICVR), 2023 9th International Conference on. :71-77 May, 2023
Subject
Language
ISSN
2331-9569
Abstract
Occlusion is an inevitable problem in virtual reality systems where a single camera is used to track multiple players during the interaction. In this work, we observe that a user’s shadow is an important hint for tracking when the user is occluded. Therefore, we propose a novel tracking approach that effectively leverages the shadow information of target users, which leads to more robust tracking in complex environments. Our key idea is to train a shadow detection model based on Mask R-CNN to extract shadows from image frames. To handle different levels of occlusion, we propose to define a series of tracking statuses for occlusion representation. To better evaluate the proposed method, we also contribute a dataset that contains numerous image frames with various forms of human shadows. Experiments demonstrate that our method is not only effective for handling user tracking even in full and long-term occlusions, but also exhibits superior real-time efficiency.