Difference between revisions of "Class meeting for 10-405 Parallel Perceptrons"
From Cohen Courses
Jump to navigationJump to search (Created page with "This is one of the class meetings on the schedule for the course Machine Learning with Large Data...") |
|||
| (11 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
* Lecture 1: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pdf in PDF]. | * Lecture 1: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pdf in PDF]. | ||
* Lecture 2: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pdf in PDF]. | * Lecture 2: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pdf in PDF]. | ||
| − | * Lecture | + | * Lecture 3: : [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-3.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-3.pdf in PDF]. |
=== Quiz === | === Quiz === | ||
* [https://qna.cs.cmu.edu/#/pages/view/55 Lecture 1 quiz] | * [https://qna.cs.cmu.edu/#/pages/view/55 Lecture 1 quiz] | ||
| − | * | + | * [https://qna.cs.cmu.edu/#/pages/view/62 Lecture 2 quiz] |
| − | * [https://qna.cs.cmu.edu/#/pages/view/ | + | * [https://qna.cs.cmu.edu/#/pages/view/245 Lecture 3 quiz] |
=== Readings === | === Readings === | ||
| − | * [http://www.cs.cmu.edu/~wcohen/10- | + | * [http://www.cs.cmu.edu/~wcohen/10-601/vp-notes/vp.pdf Notes on voted perceptron.] Note: these were updated --[[User:Wcohen|Wcohen]] ([[User talk:Wcohen|talk]]) 10:28, 6 March 2018 (EST) |
| − | === Readings === | + | === Optional Readings === |
* [https://cseweb.ucsd.edu/~yfreund/papers/LargeMarginsUsingPerceptron.pdf Large Margin Classification Using the Perceptron Algorithm], Freund and Schapire, MLJ 1999 | * [https://cseweb.ucsd.edu/~yfreund/papers/LargeMarginsUsingPerceptron.pdf Large Margin Classification Using the Perceptron Algorithm], Freund and Schapire, MLJ 1999 | ||
* [http://www.cs.columbia.edu/~mcollins/papers/tagperc.pdf Discriminative Training Methods for Hidden Markov Models], Collins EMNLP 2002. | * [http://www.cs.columbia.edu/~mcollins/papers/tagperc.pdf Discriminative Training Methods for Hidden Markov Models], Collins EMNLP 2002. | ||
*[http://www.ryanmcd.com/papers/parallel_perceptronNAACL2010.pdf Distributed Training Strategies for the Structured Perceptron], R. McDonald, K. Hall and G. Mann, North American Association for Computational Linguistics (NAACL), 2010. | *[http://www.ryanmcd.com/papers/parallel_perceptronNAACL2010.pdf Distributed Training Strategies for the Structured Perceptron], R. McDonald, K. Hall and G. Mann, North American Association for Computational Linguistics (NAACL), 2010. | ||
| + | |||
| + | === Things to Remember === | ||
| + | |||
| + | * Definition of mistake bound | ||
| + | * Definition of perceptron algorithm | ||
| + | ** Mistake bound analysis for perceptrons, in terms of margin and example radius | ||
| + | * Converting perceptrons to batch: voted perceptron, averaged perceptron | ||
| + | * Definition of the ranking perceptron and kernel perceptron | ||
| + | * Relationship of hash trick to kernels | ||
| + | |||
| + | * Parallellizing streaming ML algorithms | ||
| + | ** Parameter mixing, and the effect it has on the mistake bounds for perceptrons | ||
| + | ** Iterative parameter mixing, and the effect it has on the mistake bounds for perceptrons | ||
| + | * The ALLREDUCE algorithm and its complexity | ||
Latest revision as of 11:28, 6 March 2018
This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-405 in Spring 2018.
Slides
- Lecture 1: in Powerpoint, in PDF.
- Lecture 2: in Powerpoint, in PDF.
- Lecture 3: : in Powerpoint, in PDF.
Quiz
Readings
- Notes on voted perceptron. Note: these were updated --Wcohen (talk) 10:28, 6 March 2018 (EST)
Optional Readings
- Large Margin Classification Using the Perceptron Algorithm, Freund and Schapire, MLJ 1999
- Discriminative Training Methods for Hidden Markov Models, Collins EMNLP 2002.
- Distributed Training Strategies for the Structured Perceptron, R. McDonald, K. Hall and G. Mann, North American Association for Computational Linguistics (NAACL), 2010.
Things to Remember
- Definition of mistake bound
- Definition of perceptron algorithm
- Mistake bound analysis for perceptrons, in terms of margin and example radius
- Converting perceptrons to batch: voted perceptron, averaged perceptron
- Definition of the ranking perceptron and kernel perceptron
- Relationship of hash trick to kernels
- Parallellizing streaming ML algorithms
- Parameter mixing, and the effect it has on the mistake bounds for perceptrons
- Iterative parameter mixing, and the effect it has on the mistake bounds for perceptrons
- The ALLREDUCE algorithm and its complexity
