Difference between revisions of "Smith and Eisner 2005:Contrastive Estimation: Training Log-Linear Models on Unlabeled Data"

From Cohen Courses
Jump to navigationJump to search
Line 16: Line 16:
  
 
== Related papers ==
 
== Related papers ==
 +
 +
From a structured prediction perspective, this paper presents an interesting contrastive estimation approach that can be compared with many existing estimation techniques, for example, joint likelihood maximization in HMMs, conditional likelihood estimation and sum of conditional likelihoods. Secondly, this paper is also in line with some other numerical optimization approach that optimizes a convex objective function. 
 +
Moreover, from the empirical evaluation standpoint, the proposed unsupervised approach might not be able to outperform the standard supervised POS tagging, but it can be applied to some sequential modeling tasks where labeled data are not abundantly available, for example, Named Entity Tagging, Parts-of-speech Tagging, and Constituent Parsing for resource-poor languages. Below shows some of the related papers to this work.
 +
 +
* [[RelatedPaper::Globerson et al. ICML 2007. Exponentiated Gradient Algorithms for Log Linear Structured Prediction]]
 +
 +
* [[RelatedPaper::Berg-Kirkpatrick et al, ACL 2010: Painless Unsupervised Learning with Features]]
 +
 +
* [[RelatedPaper::Klein 2002 conditional structure versus conditional estimation in nlp models]]

Revision as of 22:04, 29 September 2011

Citation

Smith, Noah A. and Jason Eisner (2005). Contrastive estimation: Training log-linear models on unlabeled data. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 354-362, Ann Arbor, Michigan, June.

Online version

Smith and Eisner 2005

Summary

This is an interesting paper that presents an unsupervised Contrastive Estimation method for Conditional Random Fields and other Log-Linear Models, which can be easily applied to estimation problems in Part of Speech Tagging, Named Entity Recognition and Semantic Role Labeling. When applying this technique to POS tagging, the observed results outperforms Expectation Maximization, and is robust when the dictionary quality is poor.

Brief description of the method

Experimental Result

Related papers

From a structured prediction perspective, this paper presents an interesting contrastive estimation approach that can be compared with many existing estimation techniques, for example, joint likelihood maximization in HMMs, conditional likelihood estimation and sum of conditional likelihoods. Secondly, this paper is also in line with some other numerical optimization approach that optimizes a convex objective function. Moreover, from the empirical evaluation standpoint, the proposed unsupervised approach might not be able to outperform the standard supervised POS tagging, but it can be applied to some sequential modeling tasks where labeled data are not abundantly available, for example, Named Entity Tagging, Parts-of-speech Tagging, and Constituent Parsing for resource-poor languages. Below shows some of the related papers to this work.