Sha 2003 shallow parsing with conditional random fields write up

This is a review of the paper Sha_2003_shallow_parsing_with_conditional_random_fields by user:sgopal1.

• Contributions of the paper
  • The paper applies the CRF proposed by Lafferty ICML 2001 to NP chunking. NP Chunking refers to the problem of tagging the base NP.
  • Proposed small changes in the way inference is done. Instead of performing GIS proposed in the orignial Lafferty paper, they resort to using reasonable approximations to the Inverse of the Hessian. This is ofcourse not tractable for general CRFs on graphs.
  • One of the first papers to apply CRF and show that it is as good as other methods available. And they also show it is fast and converges relatively swiftly.

• Criticism
  • Given that they are not introducing 'CRF' and one of their main contributions being 'a faster way to do inference', they could have explained a little more about why their approximations of the inverse of the Hessian was good, and how it would have compared to finding the complete Hessian. ( may be just report at a smaller scale ).
  • I would have liked to see some more intuitive features to be added to the model than just throw all the previous-2 words or tags etc
  • I think that the use of McNemar's significance test on labeling errors is not convincing. Although the use of this test on F-scores is valid, It is not clear why they had to use the same test for labeling errors. They could have resorted to the normal paired-t-test