Difference between revisions of "Lau et al HLT 1993"
| Line 20: | Line 20: | ||
Use trigger pairs: if a word sequence A is significantly correlated with another word sequence B (A->B) this is considered a trigger pair. | Use trigger pairs: if a word sequence A is significantly correlated with another word sequence B (A->B) this is considered a trigger pair. | ||
| + | |||
| + | [[File:Triggerpairs.png]] | ||
Given the document that was processed so far (h) and a word considered for the next position (w), there are many different estimates P(w|h), derived from the various triggers. How to combine them? | Given the document that was processed so far (h) and a word considered for the next position (w), there are many different estimates P(w|h), derived from the various triggers. How to combine them? | ||
Revision as of 17:03, 26 September 2011
Contents
Citation
Raymond Lau, Ronald Rosenfeld and Salim Roukos. Adaptive Language Modeling Using the Maximum Entropy Principle. In Proceedings of the ARPA Human Language Technology Workshop, published as Human Language Technology, pages 108–113. Morgan Kaufmann, March 1993.
Online version
Summary
In this paper the authors focus on the development of Language Models using Maximum Entropy principles, in order to combine evidence from multiple sources (for example: trigrams and long distance triggers).
The problem
State of the art language model was a trigram model (prob of a word, based on the two word preceeding it). ==> Static model, not able to adapt to style and topic of the document
Adaptive model ==> changes estimates as a result of "seeing" some of the text
- process a large heterogeneous data source
- trained on data from one domain, can be used in another domain
Use trigger pairs: if a word sequence A is significantly correlated with another word sequence B (A->B) this is considered a trigger pair.
Given the document that was processed so far (h) and a word considered for the next position (w), there are many different estimates P(w|h), derived from the various triggers. How to combine them?
