13.pdf
people.eecs.berkeley.edu · 1,765 words · saved by 1 readers
N/A
Shrinkage: Ridge Regression, Subset Selection, and Lasso 71 13 Shrinkage: Ridge Regression, Subset Selection, and Lasso RIDGE REGRESSION aka Tikhonov Regularization Least-squares linear regression + `2 penalized mean loss. (1) + (A) + (a) + (d). Find w that minimizes kXw yk2 + kw0 k2 = J(w) where w0 is w with component ↵ replaced by 0. X has fictitious dimension but we DON’T penalize ↵. Adds a regularization term, aka a penalty term, for shrinkage: to encourage small kw0 k. Why? 1. Guarantees positive definite normal eq’ns;…
related reading
- Regularization (mathematics) - Wikipediaen.wikipedia.org
- What Is Ridge Regression? | IBMibm.com
- 5.1 - Ridge Regression | STAT 897Donline.stat.psu.edu
- Lasso (statistics) - Wikipediaen.wikipedia.org
- Understanding regularization for logistic regression | KNIMEknime.com
- L1 regularization: sparsity through singularitiesejenner.com
- notes.pdfjingbol.web.illinois.edu
- Linear Regressionmlu-explain.github.io
- Gregory Gundersengregorygundersen.com
- generalized surearxiv.org
- deeplearningbook.org/contents/ml.htmldeeplearningbook.org
- Minimum description length - Wikipediaen.wikipedia.org