학술논문

Random Linear Estimation With Rotationally-Invariant Designs: Asymptotics at High Temperature
Document Type
Periodical
Author
Source
IEEE Transactions on Information Theory IEEE Trans. Inform. Theory Information Theory, IEEE Transactions on. 70(3):2118-2153 Mar, 2024
Subject
Communication, Networking and Broadcast Technologies
Signal Processing and Analysis
Mutual information
Mathematical models
Signal to noise ratio
Estimation
Bayes methods
Analytical models
Predictive models
mutual information
multiaccess communication
Language
ISSN
0018-9448
1557-9654
Abstract
We study estimation in the linear model $y=A \beta ^{\star} +\epsilon $ , in a Bayesian setting where $ \beta ^{\star} $ has an entrywise i.i.d. prior and the design $A$ is rotationally-invariant in law. In the large system limit as dimension and sample size increase proportionally, a set of related conjectures have been postulated for the asymptotic mutual information, Bayes-optimal mean squared error, and TAP mean-field equations that characterize the Bayes posterior mean of $ \beta ^{\star} $ . In this work, we prove these conjectures for a general class of signal priors and for arbitrary rotationally-invariant designs $A$ , under a “high-temperature” condition that restricts the range of eigenvalues of $A^{\top} A$ and encompasses regimes of sufficiently low signal-to-noise ratio. Our proof uses a conditional second-moment method argument, where we condition on the iterates of a version of the Vector AMP algorithm for solving the TAP mean-field equations.