Difference between revisions of "Syllabus for Machine Learning with Large Datasets 10-405 in Spring 2018"

From Cohen Courses
Jump to navigationJump to search
Line 21: Line 21:
 
* Homeworks, unless otherwise posted, will be due when the next HW comes out.
 
* Homeworks, unless otherwise posted, will be due when the next HW comes out.
 
* Lecture notes and/or slides will be (re)posted around the time of the lectures.
 
* Lecture notes and/or slides will be (re)posted around the time of the lectures.
 +
 +
=== Schedule ===
  
 
* Wed Jan 17, 2018 [[Class meeting for 10-405 Overview|Overview]].  Grading policies and etc, History of Big Data, Complexity theory and cost of important operations
 
* Wed Jan 17, 2018 [[Class meeting for 10-405 Overview|Overview]].  Grading policies and etc, History of Big Data, Complexity theory and cost of important operations
Line 36: Line 38:
 
* Wed Feb 14, 2018 [[Class meeting for 10-405 Parallel Perceptrons|Parallel Perceptrons 1]].  The "delta trick", Averaged perceptrons, Debugging ML algorithms
 
* Wed Feb 14, 2018 [[Class meeting for 10-405 Parallel Perceptrons|Parallel Perceptrons 1]].  The "delta trick", Averaged perceptrons, Debugging ML algorithms
 
* Mon Feb 19, 2018 [[Class meeting for 10-405 Parallel Perceptrons|Parallel Perceptrons 2]].  Hash kernels, Ranking perceptrons, Structured perceptrons
 
* Mon Feb 19, 2018 [[Class meeting for 10-405 Parallel Perceptrons|Parallel Perceptrons 2]].  Hash kernels, Ranking perceptrons, Structured perceptrons
** '''Start work on''' Assignment 3: scalable SGD; http://www.cs.cmu.edu/~wcohen/10-405/hw-3.pdf
+
** '''Start work on''' Assignment 3: scalable SGD; http://www.cs.cmu.edu/~wcohen/10-405/assignments/hw3.pdf
 
* Wed Feb 21, 2018 [[Class meeting for 10-405 Parallel Perceptrons|Parallel Perceptrons 3]].  Iterative parameter mixing paper, Parallel SGD via Param Mixing
 
* Wed Feb 21, 2018 [[Class meeting for 10-405 Parallel Perceptrons|Parallel Perceptrons 3]].  Iterative parameter mixing paper, Parallel SGD via Param Mixing
 
* Mon Feb 26, 2018 [[Class meeting for 10-405 SGD for MF|SGD for MF]].  Matrix factorization, Matrix factorization with SGD, distributed matrix factorization with SGD
 
* Mon Feb 26, 2018 [[Class meeting for 10-405 SGD for MF|SGD for MF]].  Matrix factorization, Matrix factorization with SGD, distributed matrix factorization with SGD

Revision as of 10:47, 21 February 2018

This is the syllabus for Machine Learning with Large Datasets 10-405 in Spring 2018.

Ideas for extensions to the HW assignments

This is not a complete list! you can use any of these as a starting point, but feel free to think up your own extensions.

HW2 (NB in GuineaPig):

  • The assignment proposes one particular scheme for parallelizing the training/testing algorithm. Consider another parallelization algorithm.
  • Implement a similarly scalable Rocchio algorithm and compare it with NB.
  • Reimplement the same algorithm in Spark (or some other dataflow language) and compare.

HW3 (Logistic regression and SGD)

  • Evaluate the hash trick for Naive Bayes systematically on a series of datasets.
  • Implement a parameter-mixing version of logistic regression and evaluate it.
  • A recent paper proposes (roughly) using SVM with NB-transformed features. Implement this and compare.
  • The personalization method described in class is based on a transfer learning method which works similarly. Many wikipedia pages are available in multiple languages, and works in related languages tend to be lexically similar (eg, "astrónomo" is Spanish for "astronomer"). Suppose features were character n-grams (eg "astr", "stro", "tron", ...) - does domain transfer work for the task of classifying wikipedia pages? Construct a dataset and experiment to test this hypothesis.

Notes

  • Homeworks, unless otherwise posted, will be due when the next HW comes out.
  • Lecture notes and/or slides will be (re)posted around the time of the lectures.

Schedule