Turney, ACL 2002

Citation

Jure Leskovec, Lars Backstrom, Jon M. Kleinberg: Meme-tracking and the dynamics of the news cycle. KDD 2009: 497-506.

Online version

[1]

Summary

This is a highly cited KDD paper paper presenting an unsupervised approach to Temporal information extraction. Specifically it introduces a framework for tracking topic shifts, e.g. news and events, over short time scales.

The key idea is to track news by short, distinctive phrase, which acts as the analogue of "genetic signatures" for different topics. It also produces quantitative analysis of the news cycle on their representative data set.

The method

Some words alternation in a phrase during the quotes called (textual mutation) could inhibit the accurate tracking. To solve this problem, the authors propose a robust method to cluster textual variants of quotes consisting of two stages namely phrase graph construction and clustering.

First of all, pre-processing is conducted to eliminate the noisy phrases within the data set including:

1. remove the phrases whose word-length is less than 4.

2. remove the phrases whose term-frequency is less than 10.

3. eliminate the phrases whose domain-frequency is at least 25% (avoid spammers).

Each node $n\in G$ in the phrase graph $G$ represents a phrase extracted from the corpus. An edge $(p,q)$ is included for every pair of phrases p and q, which always points from shorter phrases to longer phrases. The weight on an edge is calculated from:

$Insertformulahere$

where operator NEAR means that the two phrases should be appeared close to each other in the corpus. Using the above formula they have calculated the average semantic orientation for a review. They have shown that the value of average semantic orientation for phrases in the items that are tagged as "recommended" by the users are usually positive and those that are tagged as "not recommended" are usually negative.

Data set

90 million news and blog articles collected over the final three months of the 2008 U.S. Presidential Election.

Experimental Result

This approach was fairly successful on a range of review-classification tasks: it achieved accuracy of between 65% and 85% in predicting an author-assigned "recommended" flag for Epinions ratings for eight diverse products, ranging from cars to movies. Many later writers used several key ideas from the paper, including: treating polarity prediction as a document-classification problem; classifying documents based on likely-to-be-informative phrases; and using unsupervised or semi-supervised learning methods.

Related papers

The widely cited Pang et al EMNLP 2002 paper was influenced by this paper - but considers supervised learning techniques. The choice of movie reviews as the domain was suggested by the (relatively) poor performance of Turney's method on movies.

An interesting follow-up paper is Turney and Littman, TOIS 2003 which focuses on evaluation of the technique of using PMI for predicting the semantic orientation of words.