Difference between revisions of "10-601 Ensembles 1"

From Cohen Courses
Jump to navigationJump to search
(Created page with "=== Slides === * [http://www.cs.cmu.edu/~wcohen/10-601/ensembles1.ppt Slides in PowerPoint]. === Readings === *Bishop: Chap 1, 2 *Mitchell: Chap 5, 6 *[http://www.cs.cmu.ed...")
 
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
This a lecture used in the [[Syllabus for Machine Learning 10-601 in Fall 2014]]
 +
 
=== Slides ===
 
=== Slides ===
 +
* [http://www.cs.cmu.edu/~zivbj/classF14/boosting.pdf Slides in pdf].
  
 
* [http://www.cs.cmu.edu/~wcohen/10-601/ensembles1.ppt Slides in PowerPoint].
 
* [http://www.cs.cmu.edu/~wcohen/10-601/ensembles1.ppt Slides in PowerPoint].
Line 5: Line 8:
 
=== Readings ===
 
=== Readings ===
  
−
*Bishop: Chap 1, 2
+
* [http://dl.acm.org/citation.cfm?id=743935 Ensemble Methods in Machine Learning], Tom Dietterich
−
*Mitchell: Chap 5, 6
+
* [http://cseweb.ucsd.edu/~yfreund/papers/IntroToBoosting.pdf A Short Introduction to Boosting], Yoav Freund and Robert Schapire.
−
*[http://www.cs.cmu.edu/~epxing/papers/Old_papers/feature.pdf Feature Selection in Microarray Analysis], E.P. Xing, in D.P. Berrar, W. Dubitzky and M. Granzow (Eds.), A Practical Approach to Microarray Data Analysis, Kluwer Academic Publishers, 2003.
 
−
*[http://ai.stanford.edu/~ang/papers/icml98-fs.pdf On Feature Selection: Learning with Exponentially many Irrelevant Features as Training Examples], Andrew Y. Ng. In Proceedings of the Fifteenth International Conference on Machine Learning, 1998.
 
−
 
 
−
=== Take home message ===
 
−
 
 
−
* Overfitting
 
−
** kNN
 
−
** Regression
 
−
 
 
−
* Bias-variance decomposition
 
  
−
* Structural risk minimization
+
===  Summary  ===
  
−
* The battle against overfitting
+
You should know how to implement  these ensemble methods, and what their relative advantages and disadvantages are:
−
** Cross validation
+
* (Ziv - not sure if can do in one lecture if we do boosting) Bagging
−
** Regularization
+
* Boosting
−
** Feature selection
+
* Stacking
 +
* Multilevel Stacking
 +
* (Ziv - not sure if I will do this) The "bucket of models" classifier
 +
* Random forest

Latest revision as of 08:53, 22 October 2014

This a lecture used in the Syllabus for Machine Learning 10-601 in Fall 2014

Slides

Readings

Summary

You should know how to implement these ensemble methods, and what their relative advantages and disadvantages are:

  • (Ziv - not sure if can do in one lecture if we do boosting) Bagging
  • Boosting
  • Stacking
  • Multilevel Stacking
  • (Ziv - not sure if I will do this) The "bucket of models" classifier
  • Random forest
Retrieved from "http://curtis.ml.cmu.edu/w/courses/index.php?title=10-601_Ensembles_1&oldid=17254"