View source for Class Meeting for 10-707 9/30/2009

This is one of the class meetings on the [[Syllabus for Information Extraction 10-707 in Fall 2009|schedule]] for the course [[Information Extraction 10-707 in Fall 2009]].

=== Meta-Learning: Stacking and Sequential Models ===

The notes also have a short review of last week's session on CRFs.

* [http://www.cs.cmu.edu/~wcohen/10-707/09-30-crfs+stacking.ppt Slides]
* [http://www.cs.cmu.edu/~wcohen/10-707/crf-notes/crf-update.pdf Additional notes on Sha & Pereira], including derivation of the gradient of the loglikelihood for CRFs].

=== Required Readings ===

* [[required::cohen_2005_stacked_sequential_learning | {{MyCiteconference| booktitle = International Joint Conference on Artificial Intelligence| coauthors = Vitor Carvalho| date = 2005| first = William W.| last = Cohen| pages = 671-676| title = Stacked sequential learning}}]].  A few people find it awkward to review a paper the instructor wrote, so if you prefer, just answer this question:
** If you used stacked sequential learning with naive Bayes as the base learning and K=10, how many times slower would it be than just running the base learner?

=== Optional Readings ===

* [http://i.stanford.edu/~kvijay/krishnan-manning-colacl06sub.pdf An Effective Two-Stage Model for Exploiting Non-Local Dependencies in Named Entity Recognition, Krishnan and Manning, ACL 2006.]  Another take on stacked sequential learning.
* [http://www-2.cs.cmu.edu/~wcohen/postscript/sdm-2007-stack.pdf Zhenzhen Kou and William W. Cohen (2007): Stacked Graphical Models for Efficient Inference in Markov Random Fields in SDM-2007].  Extended version of the stacked sequential learning method that applies to arbitrary graphs.
* [http://www.cs.jhu.edu/~brill/CompLing95.ps Transformation-Based Error-Driven Learning and Natural Language Processing, Brill, COLING 1995].  The learning algorithm in the Brill parser, which has also been used for NER (e.g., in Abgene).
* [http://hal3.name/docs/daume09searn.pdf Search-based Structured Prediction, Daume, Langford, and Marcu, Machine Learning Journal (2009)].  Another clever meta-learning algorithm that works well for sequences.
* [http://learning.eng.cam.ac.uk/zoubin/papers/CGM.pdf Conditional graphical models, Perez-Cruz & Ghahramani, 2007, in Predicting Structured Data. MIT Press, Cambridge, MA, USA, pp. 265-282.].  A very simple and effective meta-learning method.