Difference between revisions of "Class meeting for 10-605 Deep Learning"

From Cohen Courses
Jump to navigationJump to search
Line 3: Line 3:
 
=== Slides ===
 
=== Slides ===
  
* Lecture 1: [http://www.cs.cmu.edu/~wcohen/10-605/2016/deep-1.pptx Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-605/2016/deep-1.pdf PDF].
+
* Lecture 1: [http://www.cs.cmu.edu/~wcohen/10-605/deep-1.pptx Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-605/deep-1.pdf PDF].
  
 
* Lecture 2: [http://www.cs.cmu.edu/~wcohen/10-605/2016/deep-2.pptx Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-605/2016/deep-2.pdf PDF].
 
* Lecture 2: [http://www.cs.cmu.edu/~wcohen/10-605/2016/deep-2.pptx Powerpoint], [http://www.cs.cmu.edu/~wcohen/10-605/2016/deep-2.pdf PDF].

Revision as of 11:52, 24 October 2017

This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-605 in Fall_2017.

Slides

Quizzes

Sample code

Readings

Things to remember

  • The underlying reasons deep networks are hard to train
  • Exploding/vanishing gradients
  • Saturation
  • The importance of key recent advances in neural networks:
  • Matrix operations and GPU training
  • ReLU, cross-entropy, softmax
  • How backprop can be generalized to a sequence of assignment operations (autodiff)
    • Wengert lists
    • How to evaluate and differentiate a Wengert list
  • Common architectures
    • Multi-layer perceptron
    • Recursive NNs (RNNS) and Long/short term memory networks (LSTMs)
    • Convolutional Networks (CNNs)