Abstract Number:
3767
Submission Type:
Contributed Abstract
Contributed Abstract Type:
Paper
Participants:
Yuchen Wu (1), Pratik Patil (2), Ryan Tibshirani (2)
Institutions:
(1) University of Pennsylvania, N/A, (2) University of California, Berkeley, N/A
Co-Author(s):
First Author:
Presenting Author:
Abstract Text:
We analyze the statistical properties of generalized cross-validation (GCV) and leave-one-out cross-validation (LOOCV) applied to early-stopped gradient descent (GD) iterates in high-dimensional least squares regression. Surprisingly, our results show that GCV can be inconsistent for estimating the squared prediction risk, even under a well-specified linear model with isotropic design. In contrast, we prove that LOOCV converges uniformly along the GD trajectory to the prediction risk. Our theory holds under mild assumptions on the data distribution and does not require the underlying regression function to be linear. Furthermore, by suitably extending LOOCV, we construct consistent estimators for the entire prediction error distribution along the GD trajectory and for a wide class of its functionals. This in particular enables the construction of pathwise prediction intervals for the unknown response with asymptotically correct nominal coverage conditional on the training data.
Keywords:
Cross validation|Gradient descent|Linear regression| | |
Sponsors:
IMS
Tracks:
Statistical Theory
Can this be considered for alternate subtype?
No
Are you interested in volunteering to serve as a session chair?
Yes
I have read and understand that JSM participants must abide by the Participant Guidelines.
Yes
I understand that JSM participants must register and pay the appropriate registration fee by June 1, 2024. The registration fee is non-refundable.
I understand