Yano et al NAACL 2009

Citation

Tae Yano, William Cohen, and Noah A. Smith. Predicting Response to Political Blog Posts with Topic Models. In Proc of NAACL 2009.

Online Version

Predicting Response to Political Blog Posts with Topic Models.

Summary

This Paper describes a topic model based approach in modeling the generation of blog text (posts and comments).

Brief description of the method

This paper expands upon LinkLDA, presented in Erosheva et al. (2004).

Here, $\theta$ is a distribution over topics, $\beta$ is a multinomial distribution over post words, and $\gamma$ is a multinomial distribution over (comment) users. $z$ is the topic for post words, while $z^{\prime }$ is the topic for comment users. $w$ represents the specific post word, and $u$ represents the user that commented on that particular post (hence, $w$ and $u$ are both observed during training). $N$ and $M$ are the words counts in the post and all of its comments, respectively.

Although LinkLDA can model which users are likely to respond to a post, it does not model the comment text they will write. The authors expand on this by proposing CommentLDA, as shown below.

$w^{\prime }$ represents the specific comment word, and is also observed during training. In CommentLDA, note that the comment text is modeled by the distribution over comment words given topics, $\beta ^{\prime }$.

The authors provide three different variations on how to count the comments (counting by verbosity, response, or comments).

Experimental Result

Task: given a training dataset consisting of a collection of blog posts and their commenters and comments, and an unseen test dataset from a later time period, predict who is going to comment on a new blog post from the test set. The evaluation is done using precision (macro-averaged across posts) of predictions at various cut-offs ($n$) .

The authors have released the Yano & Smith blog dataset, which was used for this evaluation. This dataset contains post and comments from the following 5 sites: CarpetBagger (CB), Daily Kos (DK), Matthew Yglesias (MY), Red State (RS), and Right Wing News (RWN).

The compared models were:

• Baseline: post-independent prediction that ranks users by their comment frequency
• Naive Bayes: with word counts in the post's main entry as features
• LinkLDA: 3 variations (verbosity, response, comments)
• CommentLDA: 3 variations (verbosity, response, comments)

Results:

• Some improvement over both the baseline and Naive Bayes for 3 out of the 5 sites
• LinkLDA usually works slightly better than CommentLDA
• Varying the counting method can bring as much as 10% performance gain
• Results suggest that commenters on different sites behave differently

Discussion

While the proposed method (CommentLDA) achieves mixed results on the prediction task, it provides a way to understand and summarize the data.

Related Papers

• Erosheva et al. (2004)
• Nallapati et al. (2008)

Study Plan

Papers/articles/blogs/videos you may want to read to understand this paper.

• David Blei, Anrew Ng, Michael Jordan, "Latent Dirichlet Allocation" JMLR, vol.3, pp.993-1022 (2003) pdf
  • Wikipedia article on LDA
  • Topic Models - videolectures.net
  • Gibbs Sampling for the Uninitiated
  • Parameter estimation for text analysis
  • Latent Dirichlet Allocation in Python