Top-$k$k Feature Selection in Sparse Learning via Accelerated Coordinate Descent Method

Top-$k$k feature selection in sparse learning is a fundamental problem in machine learning. It is difficult to conquer due to the rigid $\ell _{2,0}$ℓ2,0-norm constraint. Existing literature mostly relaxes the constraint and seeks the approximation of the selection matrix, degenerating primitive models and missing the genuine solutions. This research tackles the primitive top-$k$k feature selection model in sparse learning. From the perspective of universality, we investigate both supervised and semi-supervised models of top-$k$k feature selection in sparse learning. By disassembling the feature selection matrix, it is revealed that two different objectives could be unified into one general ratio-trace problem, which is a non-convex optimization problem. The accelerated coordinate descent method is raised to efficiently solve the non-convex objective, through which the local optimal solution of top-$k$k feature indices is obtained with a competitive time cost. To verify the proposed algorithm, we design toy experiments that could visualize the advantages of the selected features. Meanwhile, experimental results on nine normal datasets and the large-scale ImageNet dataset comprehensively show the superiority of our methods compared to representative and state-of-the-art supervised and semi-supervised algorithms.