Class meeting for 10-605 Phrase Finding

This is one of the class meetings on the schedule for the course Machine Learning with Large Datasets 10-605 in Fall 2016.

Slides

• Slides in Powerpoint, Slides in Powerpoint. The slides also cover Spark and other map-reduce languages.

Quiz

• Quiz

Sample Code

• A tarball with code for phrase-finding in PIG.
• Some sample data for this is on HDFS on opencloud under /user/wcohen/phrases/data

Readings for the Class

• A Language Model Approach to Keyphrase Extraction, Takashi Tomokiyo and Matthew Hurst, MWE '03 Proceedings of the ACL 2003 workshop on Multiword expressions: analysis, acquisition and treatment.

Also discussed

• Locating complex named entities in web text, Downey et al, IJCAI 2007.
• Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews, Turney, ACL 2002
• Answering subcognitive Turing Test questions: A reply to French, Turney, Journal of Experimental and Theoretical Artificial Intelligence, 13 (4), 409-419.

Things to Remember

• The request-and-answer pattern
• BLRT and pointwise KL-divergence statistics