Ling and He Joint Sentiment Topic Model for Sentiment Analysis

Citation

author = {Lin, Chenghua and He, Yulan},
title = {Joint sentiment/topic model for sentiment analysis},
booktitle = {Proceedings of the 18th ACM conference on Information and knowledge management},
series = {CIKM '09},
year = {2009},
isbn = {978-1-60558-512-3},
location = {Hong Kong, China},
pages = {375--384},
numpages = {10},
url = {http://doi.acm.org/10.1145/1645953.1646003},
doi = {10.1145/1645953.1646003},
acmid = {1646003},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {joint sentiment/topic model, latent dirichlet allocation, opinion mining, sentiment analysis}

Online Version

Joint Sentiment/Topic Model for Sentiment Analysis

Summary

This paper proposes a probabilistic modeling framework based on Latent Dirichlet Allocation (LDA) which detects sentiment and topics simultaneously from text. Most approaches to sentiment analysis require labeled corpora for training and inference. The proposed model, however, is fully unsupervised i.e. does not require any labeled data as such.

The authors hint towards why a joint model might is being sought. Sentiment polarities, intuitively, are dependent on topics or domains. For instance, though the adjective ‘unpredictable’ in a phrase such as ‘unpredictable steering’ has negative orientation in an automobile review, it has a positive orientation in a phrase like ‘unpredictable plot’ in a movie review.

The Joint Sentiment Topic (JST) model is similar in flavor to LDA the only major difference being an addition of a sentiment layer between the document and the topic layer. Each document in the Joint Sentiment Topic (JST) model is associated with distinct topic-document distributions corresponding to each of the $S$ sentiment labels. Each of these distributions correspond to a sentiment label $l$ with the same number of topics. The distribution defined by the topic and the sentiment label is then used to draw the word.

 1. For each document $d$, choose a distribution $\pi _{d}$ ~ Dirichlet($\gamma$)
 2. For each sentiment label $l$ under document $d$, choose a distribution $\theta _{d,l}$ ∼ Dir($\alpha$).
 3. For each word $w_{i}$ in document $d$
   a. Choose a sentiment label $l_{i}$ ∼ $\pi _{d}$
   b. Choose a topic $z_{i}$ ~ $\theta _{d,l_{i}}$
   c. Choose a word $w_{i}$ from the distribution over words defined by the topic $z_{i}$ and sentiment label $l_{i},\phi _{z_{i}}^{l_{i}}$ 

Inference

Gibbs sampling algorithm is provided for estimating the posterior distribution of the latent variables given a document.

Tying-JST model

One has to choose a topic-document distribution $\theta _{d}$ for every document under the JST model, whereas in tying-JST there is only one topic-document distribution $\theta$ which accounts for all the documents in the corpus.

Experiments

The authors used a corpus of preprocessed movie reviews for evaluating the performance of the JST model.

Study Plan

1. Blei_et_al_Latent_Dirichlet_Allocation