학술논문
Random Linear Estimation With Rotationally-Invariant Designs: Asymptotics at High Temperature
Document Type
Periodical
Source
IEEE Transactions on Information Theory IEEE Trans. Inform. Theory Information Theory, IEEE Transactions on. 70(3):2118-2153 Mar, 2024
Subject
Language
ISSN
0018-9448
1557-9654
1557-9654
Abstract
We study estimation in the linear model $y=A \beta ^{\star} +\epsilon $ , in a Bayesian setting where $ \beta ^{\star} $ has an entrywise i.i.d. prior and the design $A$ is rotationally-invariant in law. In the large system limit as dimension and sample size increase proportionally, a set of related conjectures have been postulated for the asymptotic mutual information, Bayes-optimal mean squared error, and TAP mean-field equations that characterize the Bayes posterior mean of $ \beta ^{\star} $ . In this work, we prove these conjectures for a general class of signal priors and for arbitrary rotationally-invariant designs $A$ , under a “high-temperature” condition that restricts the range of eigenvalues of $A^{\top} A$ and encompasses regimes of sufficiently low signal-to-noise ratio. Our proof uses a conditional second-moment method argument, where we condition on the iterates of a version of the Vector AMP algorithm for solving the TAP mean-field equations.