Sgardine writesup Collins EMNLP 2002

This is a review of Collins_2002_Discriminative_Training_Methods_for_HMMs by user:sgardine.

Summary

Similarly to CRFs, perceptrons can be adapted to tagging problems to address the shortcomings of ME models by incorporating diverse features. A perceptron training scheme for tagging is introduced, and error-bounding theorems from perceptrons generally are shown to apply. The model is applied to NP Chunking and POS Tagging. The best results are achieved by averaging (as expected, the final perceptron weights seem to overfit the training data) and by making use of all features which occur at all in the training data (also expected, since the model will simply down-weight irrelevant features). The training scheme is also much faster than ME using GIS.

Commentary

The authors mention CRFs in the introduction but perform no direct comparison between their perceptron model and CRFs. This may be because of the timing of the paper and CRFs but it seems an odd omission: we learn that both outperform ME so a direct comparison seems natural.