학술논문

VirtualIMU: Generating Virtual Wearable Inertial Data from Video for Deep Learning Applications
Document Type
Conference
Source
2023 IEEE 19th International Conference on Body Sensor Networks (BSN) Body Sensor Networks (BSN), 2023 IEEE 19th International Conference on. :1-4 Oct, 2023
Subject
Bioengineering
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Signal Processing and Analysis
Deep learning
Accelerometers
Costs
Systematics
Biological system modeling
Data models
Skeleton
Gyroscopes
Web sites
Videos
Human activity recognition
wearables accelerometer
wearable gyroscope
data augmentation
Language
ISSN
2376-8894
Abstract
In the era of deep learning, accessibility to a large amount of wearable Inertial Measurement Unit (IMU) data plays a crucial role in various biomedical and health applications. However, collection of ‘big’ IMU data is extremely challenging due to its cost and time requirements. To address this, researchers have explored using publicly available videos, such as those on YouTube, to extract human skeletal models and synthesize IMU data. However, existing methods for converting skeletal models to virtual IMU data are oversimplified and lack systematic data augmentation capabilities. In this study, we propose a systematic approach to synthesize realistic and diverse IMU data, including three-axis accelerometer and gyroscope measurements, from video-based skeleton representations. Through experiments involving seven healthy individuals, we demonstrate that our method can accurately synthesize accelerometer and gyroscope data with a normalized root mean square error of 14.4 % and 16.0 %, respectively. Furthermore, we qualitatively evaluate the algorithm’s ability to generate a large volume of diverse IMU data. Our findings affirm the potential of obtaining diverse synthetic IMU data from videos, offering a promising solution to reduce the costs associated with collecting IMU data in deep learning-based applications.