Approximate Orthogonal Sparse Embedding for Dimensionality Reduction

Zhihui Lai, Wai Keung Wong, Yong Xu, Jian Yang, Dapeng Zhang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

136 Citations (Scopus)


Locally linear embedding (LLE) is one of the most well-known manifold learning methods. As the representative linear extension of LLE, orthogonal neighborhood preserving projection (ONPP) has attracted widespread attention in the field of dimensionality reduction. In this paper, a unified sparse learning framework is proposed by introducing the sparsity or L-1-norm learning, which further extends the LLE-based methods to sparse cases. Theoretical connections between the ONPP and the proposed sparse linear embedding are discovered. The optimal sparse embeddings derived from the proposed framework can be computed by iterating the modified elastic net and singular value decomposition. We also show that the proposed model can be viewed as a general model for sparse linear and nonlinear (kernel) subspace learning. Based on this general model, sparse kernel embedding is also proposed for nonlinear sparse feature extraction. Extensive experiments on five databases demonstrate that the proposed sparse learning framework performs better than the existing subspace learning algorithm, particularly in the cases of small sample sizes.
Original languageEnglish
Article number7102762
Pages (from-to)723-735
Number of pages13
JournalIEEE Transactions on Neural Networks and Learning Systems
Issue number4
Publication statusPublished - 1 Apr 2016


  • Dimensionality reduction
  • elastic net
  • image recognition
  • manifold learning
  • sparse projections

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this