Difference between revisions of "Class meeting for 10-405 Parallel Perceptrons"
From Cohen Courses
Jump to navigationJump to search (→Quiz) |
|||
(8 intermediate revisions by the same user not shown) | |||
Line 5: | Line 5: | ||
* Lecture 1: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pdf in PDF]. | * Lecture 1: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-1.pdf in PDF]. | ||
* Lecture 2: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pdf in PDF]. | * Lecture 2: [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-2.pdf in PDF]. | ||
− | * Lecture 3: .... | + | * Lecture 3: : [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-3.pptx in Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-405/perceptrons-3.pdf in PDF]. |
=== Quiz === | === Quiz === | ||
Line 14: | Line 14: | ||
=== Readings === | === Readings === | ||
− | * [http://www.cs.cmu.edu/~wcohen/10- | + | * [http://www.cs.cmu.edu/~wcohen/10-601/vp-notes/vp.pdf Notes on voted perceptron.] Note: these were updated --[[User:Wcohen|Wcohen]] ([[User talk:Wcohen|talk]]) 10:28, 6 March 2018 (EST) |
− | === Readings === | + | === Optional Readings === |
* [https://cseweb.ucsd.edu/~yfreund/papers/LargeMarginsUsingPerceptron.pdf Large Margin Classification Using the Perceptron Algorithm], Freund and Schapire, MLJ 1999 | * [https://cseweb.ucsd.edu/~yfreund/papers/LargeMarginsUsingPerceptron.pdf Large Margin Classification Using the Perceptron Algorithm], Freund and Schapire, MLJ 1999 | ||
* [http://www.cs.columbia.edu/~mcollins/papers/tagperc.pdf Discriminative Training Methods for Hidden Markov Models], Collins EMNLP 2002. | * [http://www.cs.columbia.edu/~mcollins/papers/tagperc.pdf Discriminative Training Methods for Hidden Markov Models], Collins EMNLP 2002. | ||
*[http://www.ryanmcd.com/papers/parallel_perceptronNAACL2010.pdf Distributed Training Strategies for the Structured Perceptron], R. McDonald, K. Hall and G. Mann, North American Association for Computational Linguistics (NAACL), 2010. | *[http://www.ryanmcd.com/papers/parallel_perceptronNAACL2010.pdf Distributed Training Strategies for the Structured Perceptron], R. McDonald, K. Hall and G. Mann, North American Association for Computational Linguistics (NAACL), 2010. | ||
+ | |||
+ | === Things to Remember === | ||
+ | |||
+ | * Definition of mistake bound | ||
+ | * Definition of perceptron algorithm | ||
+ | ** Mistake bound analysis for perceptrons, in terms of margin and example radius | ||
+ | * Converting perceptrons to batch: voted perceptron, averaged perceptron | ||
+ | * Definition of the ranking perceptron and kernel perceptron | ||
+ | * Relationship of hash trick to kernels | ||
+ | |||
+ | * Parallellizing streaming ML algorithms | ||
+ | ** Parameter mixing, and the effect it has on the mistake bounds for perceptrons | ||
+ | ** Iterative parameter mixing, and the effect it has on the mistake bounds for perceptrons | ||
+ | * The ALLREDUCE algorithm and its complexity |
Latest revision as of 11:28, 6 March 2018
This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-405 in Spring 2018.
Slides
- Lecture 1: in Powerpoint, in PDF.
- Lecture 2: in Powerpoint, in PDF.
- Lecture 3: : in Powerpoint, in PDF.
Quiz
Readings
- Notes on voted perceptron. Note: these were updated --Wcohen (talk) 10:28, 6 March 2018 (EST)
Optional Readings
- Large Margin Classification Using the Perceptron Algorithm, Freund and Schapire, MLJ 1999
- Discriminative Training Methods for Hidden Markov Models, Collins EMNLP 2002.
- Distributed Training Strategies for the Structured Perceptron, R. McDonald, K. Hall and G. Mann, North American Association for Computational Linguistics (NAACL), 2010.
Things to Remember
- Definition of mistake bound
- Definition of perceptron algorithm
- Mistake bound analysis for perceptrons, in terms of margin and example radius
- Converting perceptrons to batch: voted perceptron, averaged perceptron
- Definition of the ranking perceptron and kernel perceptron
- Relationship of hash trick to kernels
- Parallellizing streaming ML algorithms
- Parameter mixing, and the effect it has on the mistake bounds for perceptrons
- Iterative parameter mixing, and the effect it has on the mistake bounds for perceptrons
- The ALLREDUCE algorithm and its complexity