Turney,2002

Citation

Turney, P., 2002, Thumbs up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews, ACL'02

Online version

[[1]]

Summary

This paper presents a simple unsupervised learning algorithm for Opinion mining problem. The system is able to classify the reviews as recommended ("thumbs up") or not-recommended ("thumbs down"). The idea is to measure the semantic orientation of phrases in a review and classify it to an appropriate class based on the average semantic orientation. The semantic orientation is measured by mutual information between the given phrase and the word "excellent" minus the mutual information between the input phrase and the word "poor".

Description of the method

The algorithm takes a written review as an input. First it assigns a POS tag to each word in the review to identify adjective or adverb phrases in the input review. They have used PMI-IR algorithm to estimate the semantic orientation of a phrase. The Pointwise Mutual Information (PMI) between two words $w_{1}$ and $w_{2}$ is defined as follow:

$PMI(w_{1},w_{2})=log_{2}(p(w_{1}\ and\ w_{2})/p(w_{1})p(w_{2}))$

where $p(w_{1},w_{2})$ is the probability that $w_{1}$ and $w_{2}$ co-occur. They have defined the semantic orientation of a phrase as follow:

$SO(phrase)=PMI(phrase,'excellent')-PMI(phrase,'poor')$

We can modify the above definition to obtain the following formula:

$SO(phrase)=log_{2}({\frac {hits(phrase\ NEAR\ 'excellent')hits('excellent')}{hits(phrase\ NEAR\ 'poor')hits('excellent')}})$

where operator NEAR means that the two phrases should be appeared close to each other in the corpus. Using the above formula we can calculate the average semantic orientation for a review. They have shown that the value of average semantic orientation for phrases in the items that are tagged as "recommended" by the users are usually positive and those that are classified as "not recommended" are usually negative.

Evaluation Results

To evaluate their technique they have chosen 410 reviews from Epinions. The accuracy of a classifier that guesses the majority class is 59% while PMI-IR technique achieves 75% accuracy.