Difference between revisions of "10-601 Deep Learning 2"
From Cohen Courses
Jump to navigationJump to search| Line 24: | Line 24: | ||
=== Things to remember === | === Things to remember === | ||
| + | * How backprop can be generalized to a sequence of assignment operations | ||
* Convolutional networks | * Convolutional networks | ||
** 2-d convolution | ** 2-d convolution | ||
Revision as of 14:23, 11 April 2016
This a lecture used in the Syllabus for Machine Learning 10-601B in Spring 2016
Slides
Readings
This area is moving very fast and the textbooks are not up-to-date. Some recommended readings:
- Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition has nice on-line notes.
I also used some on-line visualizations in the materials for the lecture, especially the part on ConvNets.
- the Wikipedia page for convolutions has nice animations of 1-D convolutions.
- On-line demo of 2-D convolutions for image processing.
- There's an on-line demo of CNNs which are trained in your browser (!)
- 3D visualization of a trained net.
The LSTM figures and examples I used are mostly from
- Christopher Olah's blog
- The unreasonable effectiveness of RNNs
- For a great counterpoint: see Yoav Goldberg's response
Things to remember
- How backprop can be generalized to a sequence of assignment operations
- Convolutional networks
- 2-d convolution
- How to construct a convolution layer
- Architecture of CNN: convolution/downsampling pairs
- Recurrent neural networks
- When they are useful
- Why they are hard to train (if trained naively)
- The basic ideas used in an LSTM: forget, insert, and output gates
