부산대학교 도서관

Nowadays, intensive interests are targeting the deep learning on edge precision health towards instantaneous disease measurements. However, edge inference usually has constrained computing resource, which poses a great challenge on running the heavy deep learning for real-time measurements. In this study, we propose to leverage a knowledge distillation methodology to enable ultra-efficient deep learning on edge. We take a special interest in Electrocardiogram (ECG)-based cardiac abnormality measurement. More specifically, we propose to train two deep learning models, including a heavy teacher model and a light-weight student model, and leverage the ‘soft target distribution’ learned by the teacher model to supervise the learning of the student model. So, the powerful teacher model can transfer learned knowledge to the student model and boost the latter’s accuracy. Further, to mitigate the vulnerability of the deep learning model under adversarial attacks, we further introduce preserving-robustness learning to the student model, without needing extra computing resources, through enhancing its loss function under adversarial perturbations. Our experiments on real-time heart disease measurement have demonstrated that, the learned lightweight student model, with a model reduction of 45x and under adversarial attacks, can still achieve comparable disease detection performance. The proposed robust knowledge distillation methodology has effectively enabled light-weight and secure cardiac measurement. Significance: This study is expected to contribute to on-edge deep learning-powered disease detection, for real-time, long term, and secured cardiac precision health.