Akcora et al, SOMA 2010

Citation

Cuneyt Gurcan Akcora, Murat Ali Bayir, Murat Demirbas, Hakan Ferhatosmanoglu, "Identifying BreakPoints in Public Opinion", SOMA 2010, SIGKDD Workshop on Social Media Analytics, Washington DC, USA.

Online version

Upinion

Summary

The aim of this paper is to identify breakpoints in public opinion by capturing trends of public opinion from twitter data. The authors approach to this opinion detection problem by using vector space models.

Methodology

After some observations, the authors claim that the emotion pattern and word pattern of tweets change as a result of a change in public opinion. With this aim in mind, authors developed an Emotion Corpus (Upinion) to detect emotions in tweets.

Two methods are used to detect opinions

• Vector Space Model : A binary vector has been created for each tweet. Each class of the Emotion Corpus is represented as a dimension in the vector and the value of each dimension is determined by the existence of any emotion word from the related class in the tweet. Centroid of vectors are calculated to represent an interval. Cosine similarity is applied to centroid vectors to find the opinion similarity between two intervals.

• Set Space Model : Each time interval is represented by a single document which is the union of tweets posted in that particular time interval. Jaccard similarity is used to find the similarity between two intervals.

The authors combine these two methods to detect a change and report a breakpoint.

In addition to detecting these changes, the authors also propose a tf-idf based scoring method to represent the breakpoints. They find the keywords by looking at the tfidf of the words while making sure that a word from the current time do not increase the prominence of the same word from an older time period.

The algorithm takes a written review as an input. First it assigns a POS tag to each word in the review to identify adjective or adverb phrases in the input review. They have used PMI-IR algorithm to estimate the semantic orientation of a phrase. The Pointwise Mutual Information (PMI) between two words $w_{1}$ and $w_{2}$ is defined as follow:

$PMI(w_{1},w_{2})=log_{2}(p(w_{1}\ and\ w_{2})/p(w_{1})p(w_{2}))$

where $p(w_{1},w_{2})$ is the probability that $w_{1}$ and $w_{2}$ co-occur. They have defined the semantic orientation of a phrase as follow:

$SO(phrase)=PMI(phrase,'excellent')-PMI(phrase,'poor')$

We can modify the above definition to obtain the following formula:

$SO(phrase)=log_{2}({\frac {hits(phrase\ NEAR\ 'excellent')hits('excellent')}{hits(phrase\ NEAR\ 'poor')hits('excellent')}})$

where operator NEAR means that the two phrases should be appeared close to each other in the corpus. Using the above formula they have calculated the average semantic orientation for a review. They have shown that the value of average semantic orientation for phrases in the items that are tagged as "recommended" by the users are usually positive and those that are tagged as "not recommended" are usually negative.

Experimental Result

This approach was fairly successful on a range of review-classification tasks: it achieved accuracy of between 65% and 85% in predicting an author-assigned "recommended" flag for Epinions ratings for eight diverse products, ranging from cars to movies. Many later writers used several key ideas from the paper, including: treating polarity prediction as a document-classification problem; classifying documents based on likely-to-be-informative phrases; and using unsupervised or semi-supervised learning methods.

Related papers

The widely cited Pang et al EMNLP 2002 paper was influenced by this paper - but considers supervised learning techniques. The choice of movie reviews as the domain was suggested by the (relatively) poor performance of Turney's method on movies.

An interesting follow-up paper is Turney and Littman, TOIS 2003 which focuses on evaluation of the technique of using PMI for predicting the semantic orientation of words.