Sgardine reviews Borthwick 1998

This is a review of Borthwick_1998_exploiting_diverse_knowledge_sources_via_maximum_entropy_in_named_entity_recognition by user:sgardine.

Summary

The MENE system used maximum entropy estimation over a set of features including binary features of the tokens, lexical features, location within structural aspects of the document such as sections, membership in certain obvious or off-the-shelf dictionaries of candidate concepts, and features given by adopting the output of previous systems. The model labeled the tokens as other or as begin, continue, end, or unique for each of the desired entity classes. (It was unclear whether the unique tag was a notational convenience, a tag stipulated by the evaluation framework, or something else ... would re-labeling the unique tag as a standalone begin tag affect anything?). Compound features were considered but rejected/deferred for complexity/performance reasons. Features were selected by simply observing those which fired more than a small number (3), and then pruning away features which bloated the model size and were heuristically judged to contribute little in recompense. The model was then Viterbi decoded to realize a tag sequence. The resulting system was competitive on the data provided by MUC-7, but fared less well on data from a novel (though similar) domain.

Comments

I liked the examination (Table 3) of how the amount of data affected MENE's ability to absorb other systems.

I am not sure why the Identifinder system was not presented alongside the other systems in Table 2. Also, the paper states that Identifinder made use of more training data, but it is unclear where that extra training data came from. Finally, it seems like the Identifinder could also have been incorporated as a feature, perhaps that team would not share its output?

• The name used for Identifinder at the time was "Nymble", I believe. It was trained on data tagged at BBN - not many teams tried to do any additional tagging beyond what was provided in the MUC competition.