학술논문

Structures of Neural Network Effective Theories
Document Type
Working Paper
Source
Subject
High Energy Physics - Theory
Condensed Matter - Disordered Systems and Neural Networks
Computer Science - Machine Learning
High Energy Physics - Phenomenology
Statistics - Machine Learning
Language
Abstract
We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations.
Comment: 7+13 pages, 5 figures