High Dimensional Variable Selection with Reciprocal L 1 -Regularization

We propose a new penalized likelihood method, reciprocal Lasso (or in short, rLasso), based on a new class of penalty functions which are decreasing in (0, ∞), discontinuous at 0, and converge to infinity when the coefficients approach zero.

The new penalty functions give nearly zero coefficients infinity penalties; in contrast, the conventional penalty functions give nearly zero coefficients nearly zero penalties (e.g., Lasso and SCAD) or constant penalties (e.g., L 0 penalty). This distinguishing feature makes rLasso very attractive for variable selection: It can effectively avoid to select overly dense models. We establish the consistency of the rLasso for variable selection and coefficient estimation under both the low and high dimensional settings.