Difference between revisions of "10-601 Bias-Variance"

Latest revision as of 11:44, 20 October 2014

This isn't covered well in Mitchell. Valentini and Dietterich is a good source for bias-variance for classification. Wikipedia has a reasonable description of the regression case, which goes back at least to Geman et al 1992.
See also Littman/Isbell on overfitting

How overfitting/underfitting can be understood as a tradeoff between high-bias and high-variance learners.
Mathematically, how to decompose error for linear regression into bias and variance.
Intuitively, how classification can be decomposed into bias and variance.
Which sorts of classifier variants lead to more bias and/or more variance: e.g., large vs small k in k-NN, etc.

@@ Line 1: / Line 1: @@
 === Slides ===
-[http://curtis.ml.cmu.edu/w/courses/images/2/2e/Lecture11-bv.pdf Slides in PDF]
+* William's [http://www.cs.cmu.edu/~wcohen/10-601/bias-variance.ppt Slides in Powerpoint], and [http://www.cs.cmu.edu/~wcohen/10-601/bias-variance.pdf in PDF]
+=== Readings ===
+* This isn't covered well in Mitchell. [http://dl.acm.org/citation.cfm?id=1016783 Valentini and Dietterich] is a good source for bias-variance for classification.  Wikipedia has a reasonable description of the [http://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff regression case], which goes back at least to [http://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff Geman et al 1992].
+* See also Littman/Isbell [https://www.youtube.com/watch?v=DQWI1kvmwRg on overfitting]
+=== What you should know ===
+* How overfitting/underfitting can be understood as a tradeoff between high-bias and high-variance learners.
+* Mathematically, how to decompose error for linear regression into bias and variance.
+* Intuitively, how classification can be decomposed into bias and variance.
+* Which sorts of classifier variants lead to more bias and/or more variance: e.g., large vs small k in k-NN, etc.