Class Meeting for 10-710 11-01-2011

This is one of the class meetings on the schedule for the course Structured Prediction 10-710 in Fall 2011.

Unsupervised Grammar Induction

• Slides

Required Readings

• Intro to chapter 4 and section 4.1 of Linguistic Structure Prediction, Smith 2011

Optional Readings

• Inside-outside reestimation from partially-bracketed corpora, Pereira and Schabes, 1992
• Tagging English text with a probabilistic model, Merialdo, CL, 1994
• Does Baum-Welch re-estimation help taggers?, Elworthy, 1994
• Bayesian learning of probabilistic language models, Stolcke's 1994 thesis at Berkeley (not Bayesian in today's sense!)
• Bayesian grammar induction for language modeling, Chen 1995 (not Bayesian in today's sense!)
• Linguistic structure as composition and perturbation, de Marcken, 1996
• A view of the EM algorithm that justifies incremental, sparse, and other variants, Neal and Hinton, 1998
• A generative constituent-context model for improved grammar induction, Klein and Manning, 2002
• Corpus-based induction of syntactic structure: models of dependency and constituency, Klein and Manning 2004
• Annealing techniques for unsupervised statistical language learning, Smith and Eisner, 2004
• Annealing structural bias in multilingual weighted grammar induction, Smith and Eisner, 2006
• Improving unsupervised dependency parsing with richer contexts and smoothing, Headden, Johnson, and McClosky, 2009

Background Readings

• The estimation of stochastic context-free grammars using the inside-outside algorithm, Lari and Young, 1990

Classic papers on learning grammars:

• Language identification in the limit, Gold, Information and Control, 1967
• A study of grammatical inference, Horning's 1969 thesis at Stanford