From Penalization to Over-Parameterization: Achieving Implicit Regularization for High-Dimensional Linear Errors-in-Variables Models

Regularization methods are crucial for the analysis of high-dimensional data, with most methods adding a penalty term to the loss function explicitly. In this paper, we introduce an implicit regularization technique through over-parameterization and propose a calibrated penalty-free (CPF) estimation method for high-dimensional linear errors-in-variables models. This method calibrates the bias caused by measurement errors while avoiding the bias introduced by penalty terms. We use the gradient descent algorithm to minimize the over-parameterized calibrated loss function without penalties, resulting in a sparse estimate of regression coefficients and thus achieving implicit regularization. Furthermore, a novel bootstrap methodology is introduced to address the challenge posed by the unknown covariance matrix of measurement errors. The oracle inequality for estimation error is established under certain regularity conditions. Extensive simulation studies and a real data analysis illustrate the competitive finite sample performance of the proposed method.