Mnduong writeup of Sutton & McCallum 2004

This is a writeup of Sutton_2004_collective_segmentation_and_labeling_of_distant_entities_in_information_extraction by user:mnduong

• This paper introduces a new approach to model long range dependencies in named-entity recognition. In particular, it is motivated by the fact that different occurrences of the same token or similar tokens in the same document are likely to have the same label.
• The model, called skip-chain CRF, extends linear-chain CRFs by adding links (skip-edges) between similar tokens. To quantify these links, they add a second type of potential to the model.
• With these skip edges, exact inference becomes intractable because there's no longer an efficient dynamic programming algorithm. The authors resort to loopy belief propagation, which is an approximate inference method that's not guaranteed to converge but has been found to work well in practice.
• Evaluated in the CMU seminar announcement dataset, for speaker tokens, skip-chain CRFs were found to have higher recall and F1 scores than linear-chain CRFs, and the two had similar precision. They didn't perform any better on the location field.
• I like the motivation of this work, which is how to exploit non-local dependencies. It is certainly an issue that's worth investigating. The paper offers a good solution to this problem.