Syllabus for Machine Learning with Large Datasets 10-605 in Fall 2017

This is the syllabus for Machine Learning with Large Datasets 10-605 in Fall 2016.

---

Notes:

• Homeworks, unless otherwise posted, will be due when the next HW comes out.
• Lecture notes and/or slides will be (re)posted around the time of the lectures.
• Classes are cancelled for Sept 21 (Rosh Hashana)
• No classes will be held on Nov 23 (Thanksgiving)

---

Schedule for 805 projects:

• 11:59pm Sun 10/1: Initial 805 project proposal due.
• 11:59pm Sun 10/15: Final 805 project proposal due.
  • This is a revised writeup that will address any comments William raises from the initial proposal.
• 11:59pm Sun 11/12: Midterm 805 project report due.
• 1:30-2:50pm Tues 12/5: Project presentations (in class).
• 11:59pm Sun 12/10: Final 805 project writeup due.

---

Tentative schedule for lectures and 605 assignments:

• Tues Aug 29, 2017 Overview. Grading policies and etc, History of Big Data, Complexity theory and cost of important operations
• Thurs Aug 31, 2017 Probability Review. Counting for big data and density estimation, streaming Naive Bayes, Rocchio and TFIDF
  • Start work on Assignment 1a: Streaming NB. Draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-1a-naivebayes-hashtab.pdf
• Tues Sep 5, 2017 Streaming Naive Bayes. Notes on scalable naive bayes, Local counting in stream and sort
• Thurs Sep 7, 2017 Hadoop Overview. Intro to Hadoop, Hadoop Streaming
  • Start work on Assignment 1b: Streaming NB on Hadoop. Draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-1b-naivebayes-hadoop.pdf
• Tues Sep 12, 2017 Workflows For Hadoop 1. Scalable classification, Scalable Rocchio and TFIDF, Abstracts for map-reduce algorithms, Joins in Hadoop
• Thurs Sep 14, 2017 Workflows For Hadoop 2. TFIDF in Pig, Guinea Pig intro, TFIDF in Guinea Pig, Similarity joins, Similarity joins with TFIDF, Parallel simjoins
  • Start work on Assignment 2: Naive bayes testing in Guinea Pig, draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-2-naivebayes-gpig/main.pdf
• Tues Sep 19, 2017 Workflows For Hadoop 3. PageRank in Pig, K-means in Pig, Spark
• Tues Sep 26, 2017 Phrase Finding. Systems built on top of Hadoop, Phrase-finding in Pig, Other work with phrases
• Thurs Sep 28, 2017 SGD and Hash Kernels. Learning as optimization, Logistic regression with SGD, Regularized SGD, Hash kernels for logistic regression
• Tues Oct 3, 2017 Parallel Perceptrons 1. Debugging ML algorithms
  • Start work on Assignment 3: scalable SGD Draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-3-sga-logreg/main.pdf
• Thurs Oct 5, 2017 Parallel Perceptrons 2.
• Tues Oct 10, 2017 Parallel Perceptrons 3. Structured perceptrons, Interative parameter mixing paper
• Thurs Oct 12, 2017 SGD for MF. Matrix factorization, Matrix factorization with SGD, distributed matrix factorization with SGD
• Tues Oct 17, 2017 Midterm review.
  • Last assignment due
• Thurs Oct 19, 2017 Midterm.
• Tues Oct 24, 2017 Subsampling a Graph. Sampling a graph, Local partitioning
  • Start work on Assignment 4: Subsampling a Graph with Approximate PageRank, draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-4-apr/main.pdf
• Thurs Oct 26, 2017 Deep Learning 1. Deep learning intro, BackProp following Nielson, Expressiveness of MLPs, Deep learning and GPUs, Exploding and vanishing gradients, Modern deep learning models
• Tues Oct 31, 2017 Deep Learning 2. Reverse-mode differentiation, Some systems using autodiff, Details on Wengert lists, Breakdown of xman.py, Recursive ANNs, Convolutional ANNs
• Thurs Nov 2, 2017 Randomized Algorithms 1. Bloom filters, The countmin sketch
  • Start work on Assignment 5: Autodiff with IPM. Draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-5-autodiff/main.pdf
• Tues Nov 7, 2017 Randomized Algorithms 2 someday, redo the count-min stuff. Review of Bloom filters, Locality sensitive hashing
• Thurs Nov 9, 2017 Graph Architectures for ML. Graph-based ML architectures, Pregel, Signal-collect, GraphLab, PowerGraph, GraphChi, GraphX
• Tues Nov 14, 2017 SSL on Graphs. Semi-supervised learning intro, Multirank-walk SSL method, Harmonic fields, Modified Adsorption SSL method, MAD with countmin sketches
• Thurs Nov 16, 2017 Unsupervised Learning On Graphs. Spectral clustering, Power iteration clustering, Label propagation for clustering non-graph data, Label propagation for SSL on non-graph data
  • Start work on Assignment 6: Phrase-finding in Spark. Draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-6-spark-phrases/main.pdf
• Tues Nov 21, 2017 Parameter Servers. Parameter servers, PS vs Hadoop, State Synchronous Parallel (SSP) model, Managed Communication in PS, LDA Sampler with PS
• Tues Nov 28, 2017 LDA 1. DGMs for naive Bayes, Gibbs sampling for LDA
  • Start work on Assignment 7: LDA with a Parameter Server, draft at http://www.cs.cmu.edu/~wcohen/10-605/assignments/2016-fall/hw-7-lda-ps/main.pdf
• Thurs Nov 30, 2017 LDA 2. Parallelizing LDA, Fast sampling for LDA, DGMs for graphs
• Tues Dec 5, 2017 Review session for final.
  • Last assignment due
• Thurs Dec 7, 2017 Final Exam.