Difference between revisions of "Class meeting for 10-605 Parameter Servers"
From Cohen Courses
Jump to navigationJump to search (Created page with "This is one of the class meetings on the schedule for the course Machine Learning with Large Datase...") |
|||
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | This is one of the class meetings on the [[Syllabus for Machine Learning with Large Datasets 10-605 in Fall | + | This is one of the class meetings on the [[Syllabus for Machine Learning with Large Datasets 10-605 in Fall 2017|schedule]] for the course [[Machine Learning with Large Datasets 10-605 in Fall_2017]]. |
=== Slides === | === Slides === | ||
* Lecture: [http://www.cs.cmu.edu/~wcohen/10-605/param-server.pptx Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-605/param-server.pdf PDF]. | * Lecture: [http://www.cs.cmu.edu/~wcohen/10-605/param-server.pptx Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-605/param-server.pdf PDF]. | ||
| − | |||
=== Quiz === | === Quiz === | ||
| Line 10: | Line 9: | ||
* [https://qna.cs.cmu.edu/#/pages/view/102 Today's quiz] | * [https://qna.cs.cmu.edu/#/pages/view/102 Today's quiz] | ||
| − | === Readings === | + | === Optional Readings === |
| + | |||
| + | * [http://www.cs.cmu.edu/~feixia/files/ps.pdf Parameter Server for Distributed Machine Learning] | ||
| + | |||
| + | * [https://arxiv.org/pdf/1512.09295v1.pdf Strategies and Principles of Distributed Machine Learning on Big Data] | ||
* [https://arxiv.org/pdf/1312.7651.pdf Petuum: A new platform for distributed machine learning on big data] | * [https://arxiv.org/pdf/1312.7651.pdf Petuum: A new platform for distributed machine learning on big data] | ||
| Line 16: | Line 19: | ||
=== Things to remember === | === Things to remember === | ||
| − | * | + | * Architecture of a generic parameter server (PS), with get/put access to parameters |
| + | * Pros/cons of asynchronous ''vs'' bounded asynchronous ''vs'' fully synchronous PS | ||
| + | * Pros/cons of PS model versus Hadoop plus IPM | ||
| + | * Stale synchronous parallel (SSP) computation model | ||
| + | * Data-parallel versus model-parallel algorithms | ||
| + | ** Data-parallel example: SGD on sharded data | ||
| + | ** Model-parallel example: Lasso accounting for parameter dependencies and parameter importance | ||
Latest revision as of 11:37, 28 November 2017
This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-605 in Fall_2017.
Slides
- Lecture: Powerpoint, PDF.
Quiz
Optional Readings
Things to remember
- Architecture of a generic parameter server (PS), with get/put access to parameters
- Pros/cons of asynchronous vs bounded asynchronous vs fully synchronous PS
- Pros/cons of PS model versus Hadoop plus IPM
- Stale synchronous parallel (SSP) computation model
- Data-parallel versus model-parallel algorithms
- Data-parallel example: SGD on sharded data
- Model-parallel example: Lasso accounting for parameter dependencies and parameter importance
