Difference between revisions of "10-601 Deep Learning 2"
From Cohen Courses
Jump to navigationJump to search| Line 28: | Line 28: | ||
** How to construct a convolution layer | ** How to construct a convolution layer | ||
** Architecture of CNN: convolution/downsampling pairs | ** Architecture of CNN: convolution/downsampling pairs | ||
| + | * Recurrent neural networks | ||
| + | ** When they are useful | ||
| + | ** Why they are hard to train (if trained naively) | ||
| + | ** The basic ideas used in an LSTM: forget, insert, and output gates | ||
Revision as of 14:23, 11 April 2016
This a lecture used in the Syllabus for Machine Learning 10-601B in Spring 2016
Slides
Readings
This area is moving very fast and the textbooks are not up-to-date. Some recommended readings:
- Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition has nice on-line notes.
I also used some on-line visualizations in the materials for the lecture, especially the part on ConvNets.
- the Wikipedia page for convolutions has nice animations of 1-D convolutions.
- On-line demo of 2-D convolutions for image processing.
- There's an on-line demo of CNNs which are trained in your browser (!)
- 3D visualization of a trained net.
The LSTM figures and examples I used are mostly from
- Christopher Olah's blog
- The unreasonable effectiveness of RNNs
- For a great counterpoint: see Yoav Goldberg's response
Things to remember
- Convolutional networks
- 2-d convolution
- How to construct a convolution layer
- Architecture of CNN: convolution/downsampling pairs
- Recurrent neural networks
- When they are useful
- Why they are hard to train (if trained naively)
- The basic ideas used in an LSTM: forget, insert, and output gates
