Difference between revisions of "Class meeting for 10-405 Deep Learning"

From Cohen Courses
Jump to navigationJump to search
 
(One intermediate revision by the same user not shown)
Line 11: Line 11:
 
=== Quizzes ===
 
=== Quizzes ===
  
These are not updated yet --[[User:Wcohen|Wcohen]] ([[User talk:Wcohen|talk]]) 14:58, 19 March 2018 (EDT)
 
  
 
* [https://qna.cs.cmu.edu/#/pages/view/79 Quiz for lecture 1]
 
* [https://qna.cs.cmu.edu/#/pages/view/79 Quiz for lecture 1]
Line 17: Line 16:
 
* [https://qna.cs.cmu.edu/#/pages/view/246 Quiz for lecture 2]
 
* [https://qna.cs.cmu.edu/#/pages/view/246 Quiz for lecture 2]
  
* [https://qna.cs.cmu.edu/#/pages/view/212 Quiz for lecture 3] - draft
+
* [https://qna.cs.cmu.edu/#/pages/view/212 Quiz for lecture 3]
  
 
=== Sample code ===
 
=== Sample code ===

Latest revision as of 12:04, 26 March 2018

This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-405 in Spring 2018.

Slides

Quizzes

Sample code

Readings

Things to remember

  • The underlying reasons deep networks are hard to train
  • Exploding/vanishing gradients
  • Saturation
  • The importance of key recent advances in neural networks:
  • Matrix operations and GPU training
  • ReLU, cross-entropy, softmax
  • How backprop can be generalized to a sequence of assignment operations (autodiff)
    • Wengert lists
    • How to evaluate and differentiate a Wengert list
  • Common architectures
    • Multi-layer perceptron
    • Recursive NNs (RNNS) and Long/short term memory networks (LSTMs)
    • Convolutional Networks (CNNs)