학술논문

DA2F: Research on Robustness of Deep Learning Models Using Approximate Activation Function
Document Type
Conference
Source
2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys) HPCC-DSS-SMARTCITY-DEPENDSYS High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys), 2022 IEEE 24th. :412-419 Dec, 2022
Subject
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Deep learning
Costs
Computational modeling
Piecewise linear approximation
Computer architecture
Robustness
Safety
Deep learning models
approximate computing
adversarial example
model robustness
Language
Abstract
Deep learning structures have been adopted in many application domains. However, these architectures are vulnerable to adversarial attacks which are often instantiated by adversarial examples: carefully crafted inputs by adding disturbances that are imperceptible to humans can easily mislead a learned classifier to make incorrect predictions. Since deep learning is fast achieving the maturity to enter into safety- critical and security-sensitive applications, such attacks may have catastrophic security and safety consequences. In this paper, we propose a new software-based approach to enhance the robustness of deep learning models to adversarial attacks which we call the Defensive Approximate Activation Function (DA 2 F). Specifically, we mainly consider deep learning architectures using the sigmoid function or tanh function, two complex non-linear involving exponential operation functions, as activation functions. We propose the piecewise linear approximation method where a new non-uniform segmentation scheme is presented. By replacing the exact activation function used in the deep learning architectures with the approximate activation function, which reduces computation cost theoretically and improves deep learning models' robustness to adversarial samples. The experiments validated that our approach was effective in defending against adversarial attacks. For LeNet-5 CNN architecture along with MNIST datasets, the approximate classifiers were more robust against adversarial attacks than the exact classifiers, with negligible loss in accuracy.