Class Meeting for 10-710 09-27-2011

This is one of the class meetings on the schedule for the course Syllabus for Structured Prediction 10-210 in Fall 2011.

Meta-Learning: Stacking and Sequential Models

• Slides on stacking
• Slides on Searn

Required Readings

• Stacked sequential learning, by William W. Cohen, Vitor Carvalho. In International Joint Conference on Artificial Intelligence, 2005..
• Search-Based Structured Prediction, Dame, Langford and Marcu. This is a journal-length paper, but it's not very dense. We will not be going through the theorems in any detail.
• Searn in Practice Unpublished manuscript, Daume et al.

Optional Readings

• An Effective Two-Stage Model for Exploiting Non-Local Dependencies in Named Entity Recognition, Krishnan and Manning, ACL 2006. Another take on stacked sequential learning.
• Zhenzhen Kou and William W. Cohen (2007): Stacked Graphical Models for Efficient Inference in Markov Random Fields in SDM-2007. Extended version of the stacked sequential learning method that applies to arbitrary graphs.
• Transformation-Based Error-Driven Learning and Natural Language Processing, Brill, COLING 1995. The learning algorithm in the Brill parser, which has also been used for NER (e.g., in Abgene).
• Search-based Structured Prediction, Daume, Langford, and Marcu, Machine Learning Journal (2009). Another clever meta-learning algorithm that works well for sequences.
• Conditional graphical models, Perez-Cruz & Ghahramani, 2007, in Predicting Structured Data. MIT Press, Cambridge, MA, USA, pp. 265-282.. A very simple and effective meta-learning method.
• Learning as Search Optimization: Approximate Large Margin Methods for Structured Prediction An alternative formal analysis of Searn.
• Unsupervised Search-based Structured Prediction. Applying Searn to unsupervised and/or semi-supervised problems.