10-601 Logistic Regression

This a lecture used in the Syllabus for Machine Learning 10-601 in Fall 2014

Slides

• Slides in Powerpoint.

Readings

• Optional:
  • Bishop 4.2-4.3
  • William's notes on SGD (for 10605)
  • Charles Elkan's notes on SGD
  • Lazy Sparse Stochastic Gradient Descent for Regularized Multinomial Logistic Regression, Carpenter, Bob. 2008. See also his blog post on logistic regression.

What You Should Know Afterward

• How to implement logistic regression.
• How to determine the best parameters for logistic regression models
• Why regularization matters for logistic regression.
• How logistic regression and naive Bayes are similar and different.
• The difference between a discriminative and a generative classifier.
• What "overfitting" is, and why optimizing performance on a training set does not necessarily lead to good performance on a test set.