Sparse Subspace Clustering via Two-Step Reweighted L1-Minimization: Algorithm and Provable Neighbor Recovery Rates

Jwo-Yuh Wu*, Liang Chi Huang, Ming Hsun Yang, Chun Hung Liu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Sparse subspace clustering (SSC) relies on sparse regression for accurate neighbor identification. Inspired by recent progress in compressive sensing, this paper proposes a new sparse regression scheme for SSC via two-step reweighted ℓ1 -minimization, which also generalizes a two-step ℓ1 -minimization algorithm introduced by E. J. Candès et al. in [The Annals of Statistics, vol. 42, no. 2, pp. 669-699, 2014] without incurring extra algorithmic complexity. To fully exploit the prior information offered by the computed sparse representation vector in the first step, our approach places a weight on each component of the regression vector, and solves a weighted LASSO in the second step. We propose a data weighting rule suitable for enhancing neighbor identification accuracy. Then, under the formulation of the dual problem of weighted LASSO, we study in depth the theoretical neighbor recovery rates of the proposed scheme. Specifically, an interesting connection between the locations of nonzeros of the optimal sparse solution to the weighted LASSO and the indexes of the active constraints of the dual problem is established. Afterwards, under the semi-random model, analytic probability lower/upper bounds for various neighbor recovery events are derived. Our analytic results confirm that, with the aid of data weighting and if the prior neighbor information is accurate enough, the proposed scheme with a higher probability can produce many correct neighbors and few incorrect neighbors as compared to the solution without data weighting. Computer simulations are provided to validate our analytic study and evidence the effectiveness of the proposed approach.

Original languageEnglish
Article number9262938
Pages (from-to)1216-1263
Number of pages48
JournalIEEE Transactions on Information Theory
Volume67
Issue number2
DOIs
StatePublished - Feb 2021

Keywords

  • compressive sensing
  • discovery rate
  • neighbor recovery
  • performance guarantees
  • sparse representation
  • sparse subspace clustering
  • Subspace clustering
  • weighted LASSO

Cite this