Class meeting for 10-405 Parameter Servers

From Cohen Courses
Jump to navigationJump to search

This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-405 in Spring 2018.

Slides

Quiz

Optional Readings

Things to remember

  • Architecture of a generic parameter server (PS), with get/put access to parameters
  • Pros/cons of asynchronous vs bounded asynchronous vs fully synchronous PS
  • Pros/cons of PS model versus Hadoop plus IPM
  • Stale synchronous parallel (SSP) computation model
  • Data-parallel versus model-parallel algorithms
    • Data-parallel example: SGD on sharded data
    • Model-parallel example: Lasso accounting for parameter dependencies and parameter importance