Sparse Coding

ufldl.stanford.edu · 1,323 words · saved by 1 readers

Sparse coding is a class of unsupervised methods for learning sets of over-complete bases to represent data efficiently. The aim of sparse coding is to find a set of basis vectors ϕ i 𝜙 𝑖 such that we can represent an input vector x 𝑥 as a linear combination of these basis vectors: While techniques such as Principal Component Analysis (PCA) allow us to learn a complete set of basis vectors efficiently, we wish to learn an over-complete set of basis vectors to represent input vectors x∈ ℝ n 𝑥 ∈ 𝑅 𝑛 (i.e. such that k>n 𝑘 > 𝑛 ). The advantage of having an over-complete basis is that our basis vectors are better able to capture structures and patterns inherent in the input data. However, with an over-complete basis, the coefficients a i 𝑎 𝑖 are no longer uniquely determined by the input vector x 𝑥 . Therefore, in sparse coding, we introduce the additional criterion of sparsity to resolve the degeneracy introduced by over-completeness. Here, we define sparsity as having

Unsupervised Feature Learning and Deep Learning Tutorial UFLDL Tutorial Home !--> Sparse Coding 28 Aug 2013 ---> Sparse coding is a class of unsupervised methods for learning sets of over-complete bases to represent data efficiently. The aim of sparse coding is to find a set of basis vectors \mathbf{\phi}_i such that we can represent an input vector \mathbf{x} as a linear combination of these basis vectors: \begin{align} \mathbf{x} = \sum_{i=1}^k a_i \mathbf{\phi}_{i} \end{align} While techniques such as Principal Component Analysis (PCA) allow us to learn a complete set of basis vectors effic

Sparse Coding

related reading