Stein's Paradox for Eigenvectors of Large Covariance Matrices

Music City Center 

We describe a version of Stein's paradox for eigenvectors of a sample covariance matrix. It shows, much like Charles Stein did for the sample mean in the 1950s, that in high dimensions, provably better estimators exist. We develop a Stein type estimator for a spiked covariance model that shrinks the spiked eigenvectors to an arbitrary low dimensional subspace. We prove that this estimator has a strictly better mean-squared error in the high dimensional limit, leading to a more accurate low dimensional representation of the data. That this result holds even for a randomly chosen subspace highlights a new paradox in probability and statistics, one that is a consequence of the geometry of high dimensional spaces.