학술논문

Hardware-Based Noisy Deep Q-Networks Using Low-Frequency Noise of Synaptic Devices for Efficient Exploration
Document Type
Periodical
Source
IEEE Electron Device Letters IEEE Electron Device Lett. Electron Device Letters, IEEE. 44(9):1571-1574 Sep, 2023
Subject
Engineered Materials, Dielectrics and Plasmas
Components, Circuits, Devices and Systems
Training
1/f noise
Neurons
Noise level
Noise measurement
Games
Low-frequency noise
Reinforcement learning (RL)
deep Q-networks (DQNs)
exploration
neuromorphic
synaptic device
Language
ISSN
0741-3106
1558-0563
Abstract
We propose an efficient exploration method using low-frequency noise of synaptic devices applicable to hardware-based deep Q-networks. The proposed method efficiently implements the exploration with a relatively low hardware burden compared with other published studies. A rounded dual channel flash memory cell is used as a synaptic device. The performance evaluation based on a simple Snake game shows that the proposed system achieves performance similar to that using the $\varepsilon $ -greedy exploration method. Sufficient exploration can be conducted for network training even with a small noise level of the synaptic devices without an additional circuit.