Difference between revisions of "Class meeting for 10-605 Parameter Servers"
From Cohen Courses
Jump to navigationJump to search| Line 18: | Line 18: | ||
=== Things to remember === | === Things to remember === | ||
| − | * | + | * Architecture of a generic parameter server (PS), with get/put access to parameters |
| + | * Pros/cons of asynchronous ''vs'' bounded asynchronous ''vs'' fully synchronous PS | ||
| + | * Pros/cons of PS model versus Hadoop plus IPM | ||
| + | * Stale synchronous parallel (SSP) computation model | ||
| + | * Data-parallel versus model-parallel algorithms | ||
| + | ** Data-parallel example: SGD on sharded data | ||
| + | ** Model-parallel example: Lasso accounting for parameter dependencies and parameter importance | ||
Revision as of 18:17, 28 November 2016
This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-605 in Fall_2016.
Slides
- Lecture: Powerpoint, PDF.
Quiz
Optional Readings
Things to remember
- Architecture of a generic parameter server (PS), with get/put access to parameters
- Pros/cons of asynchronous vs bounded asynchronous vs fully synchronous PS
- Pros/cons of PS model versus Hadoop plus IPM
- Stale synchronous parallel (SSP) computation model
- Data-parallel versus model-parallel algorithms
- Data-parallel example: SGD on sharded data
- Model-parallel example: Lasso accounting for parameter dependencies and parameter importance
