학술논문

ATT-TA: A Cooperative Multiagent Deep Reinforcement Learning Approach for TSV Assignment in 3-D ICs
Document Type
Periodical
Source
IEEE Transactions on Very Large Scale Integration (VLSI) Systems IEEE Trans. VLSI Syst. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on. 31(12):1905-1917 Dec, 2023
Subject
Components, Circuits, Devices and Systems
Computing and Processing
Through-silicon vias
Integrated circuits
Routing
Optimization
Training
Reinforcement learning
Planning
Multiagent deep reinforcement learning (MADRL)
three-dimensional integrated circuits (3-D ICs)
through-silicon via (TSV) assignment
Language
ISSN
1063-8210
1557-9999
Abstract
Three-dimensional integrated circuit (3-D IC) technology has emerged as a solution to address the limitations of 2-D ICs. One critical component of 3-D ICs is the through-silicon via (TSV), which plays a crucial role in connecting signal nets and dissipating heat. The assignment of TSVs significantly impacts the wirelength and temperature of 3-D ICs. Although several studies have focused on the TSV assignment problem, most existing heuristic algorithms suffer from long iteration cycles and lack performance guarantees. Meanwhile, these methods struggle to optimize large-scale cases due to the curse of dimensionality inherent in complex IC designs. Recently, learning-based algorithms, particularly reinforcement learning (RL), have demonstrated success in solving various combinatorial optimization problems by leveraging past experiences. In this article, we propose attention-TSV assignment (ATT-TA), a novel TSV assignment method based on a multiagent deep RL (MADRL) algorithm. The TSV assignment problem is formulated as a Markov decision process (MDP), where each TSV layer serves as an independent agent. To enhance collaboration, we incorporate an attention mechanism that models and utilizes teammate strategies within a cooperative multiagent system. By applying multiagent systems, we mitigate the curse of dimensionality, while the attention mechanism enables adaptive modeling of joint strategies among agents. Notably, this is the first attempt to solve the TSV assignment problem using a MADRL algorithm. The experimental results demonstrate that compared with the state-of-the-art heuristic-based TSV assignment methods, the proposed ATT-TA approach consistently outperforms these methods in terms of wirelength and temperature optimization.