Brill, CL 1995

Citation

Brill, E. 1995. Transformation-based error-driven learning and natural language processing: a case study in part-of-speech tagging. Computational Linguistics. 21. 4. p543-565

Online version

Transformation-based error-driven learning and natural language processing: a case study in part-of-speech tagging

Summary

This paper introduces a learning technique called "Transformation-based error-driven learning", a.k.a. Transformation Based Learning (TBL).

The key points from the paper are:

• Transformation Based Learning is an algorithm that learns a sequence of transformations to improve tagging on some baseline tagger
• Transformations are broken down into two components: a triggering event (such as if the previous word is a determiner) and a re-write rule (such as change tag from modal to noun)
• Authors described the use of Transformation Based Learning on POS Tagging.

Transformation-Based Learning

The learning algorithm is summarized as follows (see Figure 1 from paper as well):

• Pass un-annotated corpus (training data) through initial-state annotator
• Compare against truth to get current score (based on number of classification errors)
• Loop until no transformation can be found to improve score (Greedy search)
  • Consider all transformations rules applied to training data, select best
  • Apply to transformation to data & get current score
  • Add transformation to ordered transformation list

Transformations are applied as follows:

• Run initial-state annotator on unseen data
• Loop through ordered list of transformations, and apply each transformation.

Transformation-based Learning for POS Tagging

...