10-601 Ensembles 1

Slides

• Slides in PowerPoint.

Readings

• Bishop: Chap 1, 2
• Mitchell: Chap 5, 6
• Feature Selection in Microarray Analysis, E.P. Xing, in D.P. Berrar, W. Dubitzky and M. Granzow (Eds.), A Practical Approach to Microarray Data Analysis, Kluwer Academic Publishers, 2003.
• On Feature Selection: Learning with Exponentially many Irrelevant Features as Training Examples, Andrew Y. Ng. In Proceedings of the Fifteenth International Conference on Machine Learning, 1998.

Take home message

• Overfitting
  • kNN
  • Regression

• Bias-variance decomposition

• Structural risk minimization

• The battle against overfitting
  • Cross validation
  • Regularization
  • Feature selection