Class meeting for 10-605 SGD and Hash Kernels

This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-605 in Spring_2014.

Slides

Stochastic gradient descent:

• Slides in Powerpoint

Readings for the Class

• William's notes on SGD

Optional readings

• For logistic regression, and the sparse updates for it: Lazy Sparse Stochastic Gradient Descent for Regularized Multinomial Logistic Regression, Carpenter, Bob. 2008. See also his blog post on logistic regression. I also recommend Charles Elkan's notes on logistic regression (local saved copy).
• For hash kernels: Feature Hashing for Large Scale Multitask Learning, Weinberger et al, ICML 2009.