Ling and He Joint Sentiment Topic Model for Sentiment Analysis

Citation

author = {Lin, Chenghua and He, Yulan},
title = {Joint sentiment/topic model for sentiment analysis},
booktitle = {Proceedings of the 18th ACM conference on Information and knowledge management},
series = {CIKM '09},
year = {2009},
isbn = {978-1-60558-512-3},
location = {Hong Kong, China},
pages = {375--384},
numpages = {10},
url = {http://doi.acm.org/10.1145/1645953.1646003},
doi = {10.1145/1645953.1646003},
acmid = {1646003},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {joint sentiment/topic model, latent dirichlet allocation, opinion mining, sentiment analysis}

Online Version

Joint Sentiment/Topic Model for Sentiment Analysis

Summary

This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet Allocation (LDA) which detects sentiment and topics simultaneously from text. Unlike other machine learning approaches to sentiment classification which often require labeled corpora for classifier training, the proposed model is fully unsupervised.

Each document in the Joint Sentiment Topic (JST) model is associated with $S$ (number of sentiment labels) topic-document distributions, each of which corresponds to a sentiment label $l$ with the same number of topics. Finally, one draws a word from distribution over words defined by the topic and sentiment label.

 1. For each document $d$, choose a distribution $\pi _{d}$ ~ Dirichlet($\gamma$)
 2. For each sentiment label $l$ under document $d$, choose a distribution $\theta _{d,l}$ ∼ Dir($\alpha$).
 3. For each word $w_{i}$ in document $d$
   a. Choose a sentiment label $l_{i}$ ∼ $\pi _{d}$
   b. Choose a topic $z_{i}$ ~ $\theta _{d,l_{i}}$
   c. Choose a word $w_{i}$ from the distribution over words defined by the topic $z_{i}$ and sentiment label $l_{i},\phi _{z_{i}}^{l_{i}}$ 

Inference

Gibbs sampling algorithm is provided for estimating the posterior distribution of the latent variables given a document.

Tying-JST model

One has to choose a topic-document distribution $\theta _{d}$ for every document under the JST model, whereas in tying-JST there is only one topic-document distribution $\theta$ which accounts for all the documents in the corpus.

Experiments

The authors used a corpus of preprocessed movie reviews for evaluating the performance of the JST model.