Difference between revisions of "Class Meeting for 10-710 09-08-2011"
From Cohen Courses
Jump to navigationJump to search| (8 intermediate revisions by 3 users not shown) | |||
| Line 3: | Line 3: | ||
=== Linear-chain CRFs === | === Linear-chain CRFs === | ||
| − | * [http://www.cs.cmu.edu/~wcohen/10- | + | * [http://www.cs.cmu.edu/~wcohen/10-710/09-08-crfs.ppt Slides in PowerPoint] |
| + | * [http://www.cs.cmu.edu/~wcohen/10-710/09-08-crfs.pdf Slides in PDF] | ||
* [http://www.cs.cmu.edu/~wcohen/10-707/crf-notes/crf-update.pdf Supplement to Sha & Pereira's paper] - a more detailed derivation of the CRF gradient. | * [http://www.cs.cmu.edu/~wcohen/10-707/crf-notes/crf-update.pdf Supplement to Sha & Pereira's paper] - a more detailed derivation of the CRF gradient. | ||
| Line 9: | Line 10: | ||
* [[required::sha_2003_shallow_parsing_with_conditional_random_fields | {{MyCiteconference| booktitle = Proceedings of HLT-NAACL| coauthors = F. Pereira| date = 2003| first = F.| last = Sha| pages = 213–220| title = Shallow parsing with conditional random fields}}]] | * [[required::sha_2003_shallow_parsing_with_conditional_random_fields | {{MyCiteconference| booktitle = Proceedings of HLT-NAACL| coauthors = F. Pereira| date = 2003| first = F.| last = Sha| pages = 213–220| title = Shallow parsing with conditional random fields}}]] | ||
| + | |||
| + | I will also cover most of the material in the paper below in lecture. This paper defines CRFs somewhat more generally than Sha & Pereira do. | ||
| + | |||
| + | * [http://www.cis.upenn.edu/~pereira/papers/crf.pdf Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Lafferty et al, 2001]. The original CRF paper. | ||
=== Optional Readings === | === Optional Readings === | ||
| − | * [http:// | + | * [http://people.binf.ku.dk/~krogh/publications/pdf/Krogh94.pdf Hidden Markov Models for Labeled Sequences, Krogh 1994]. The method of this paper appears to be equivalent to linear-chain CRFs - so why didn't it catch on? |
* [http://jmlr.csail.mit.edu/papers/volume9/dietterich08a/dietterich08a.pdf Gradient tree boosting for training CRFs, Dietterich et al, ICML 2004]. A very different training method for CRFs, based on regression trees. | * [http://jmlr.csail.mit.edu/papers/volume9/dietterich08a/dietterich08a.pdf Gradient tree boosting for training CRFs, Dietterich et al, ICML 2004]. A very different training method for CRFs, based on regression trees. | ||
* [http://acl.ldc.upenn.edu/P/P06/P06-1027.pdf Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling, Jiao et al, ACL 2006]. A very nice paper from the UofA group on semi-supervised CRF learning. | * [http://acl.ldc.upenn.edu/P/P06/P06-1027.pdf Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling, Jiao et al, ACL 2006]. A very nice paper from the UofA group on semi-supervised CRF learning. | ||
| − | * [http:// | + | * [http://www.cs.ubc.ca/~murphyk/Papers/icml06_camera.pdf Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods, Vishwanathan et al, ICML 2006]. CRF learning methods seem complicated - this paper shows that stochastic gradient methods, a class of very simple on-line methods, can be competitive. |
| − | * Choi, Y., and C. Cardie. Hierarchical Sequential Learning for Extracting Opinions and their Attributes. ACL-2010 (short paper) | + | * [http://www.aclweb.org/anthology-new/P/P10/P10-2050.pdf Choi, Y., and C. Cardie. Hierarchical Sequential Learning for Extracting Opinions and their Attributes. ACL-2010] (short paper) |
| − | * Lavergne, T., O. Cappé, T. ParisTech, and F. Yvon. | + | * Lavergne, T., O. Cappé, T. ParisTech, and F. Yvon. Practical very large scale CRFs. ACL-2010. Full of detailed implementation-oriented tricks. |
| + | * [http://www.morganclaypool.com/doi/abs/10.2200/S00361ED1V01Y201105HLT013 <i>LSP</i>]: section 3.5. | ||
=== Background === | === Background === | ||
| − | + | ||
* [http://www.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf An Introduction to Conditional Random Fields for Relational Learning]. A longish tutorial overview of CRFs. | * [http://www.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf An Introduction to Conditional Random Fields for Relational Learning]. A longish tutorial overview of CRFs. | ||
Latest revision as of 02:10, 30 September 2011
This is one of the class meetings on the schedule for the course Syllabus for Structured Prediction 10-210 in Fall 2011.
Linear-chain CRFs
- Slides in PowerPoint
- Slides in PDF
- Supplement to Sha & Pereira's paper - a more detailed derivation of the CRF gradient.
Required Readings
I will also cover most of the material in the paper below in lecture. This paper defines CRFs somewhat more generally than Sha & Pereira do.
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Lafferty et al, 2001. The original CRF paper.
Optional Readings
- Hidden Markov Models for Labeled Sequences, Krogh 1994. The method of this paper appears to be equivalent to linear-chain CRFs - so why didn't it catch on?
- Gradient tree boosting for training CRFs, Dietterich et al, ICML 2004. A very different training method for CRFs, based on regression trees.
- Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling, Jiao et al, ACL 2006. A very nice paper from the UofA group on semi-supervised CRF learning.
- Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods, Vishwanathan et al, ICML 2006. CRF learning methods seem complicated - this paper shows that stochastic gradient methods, a class of very simple on-line methods, can be competitive.
- Choi, Y., and C. Cardie. Hierarchical Sequential Learning for Extracting Opinions and their Attributes. ACL-2010 (short paper)
- Lavergne, T., O. Cappé, T. ParisTech, and F. Yvon. Practical very large scale CRFs. ACL-2010. Full of detailed implementation-oriented tricks.
- LSP: section 3.5.
Background
- An Introduction to Conditional Random Fields for Relational Learning. A longish tutorial overview of CRFs.
