Chang and Blei, AOAS2010

Citation

J. Chang and D. Blei. Hierarchical relational models for document networks. Annals of Applied Statistics, 4(1):124–150, 2010

Online version

D.Blei's papers

Motivation

• Network data

 - social networks of friends
 - citation networks of documents
 - hyperlinked networks of web pages

• “Predictive Models”

 - point social network members toward new friends
 - point scientific papers toward relevant citations
 - point web pages toward other related pages

• “Descriptive statistics”

 - uncover the hidden community structure

This paper developed a hierarchical model of both network structure and node attributes, based on Latent Dirichlet Allocation.

Methodology

1. For each document $d$:

 (a) Draw topic proportions $\theta _{d}|\alpha \sim Dir(\alpha )$
 (b) For each word $w_{d,n}$:
     i.  Draw assignment $z_{d,n}|\theta _{d}\sim Mult(\theta _{d})$.
     ii. Draw word $w_{d,n}|z_{d,n},\beta _{1:K}\sim Mult(\beta _{z_{d,n}})$.

2. For each pair of documents $d$,$d'$:

 (a) Draw binary link indicator   
              $y_{d,d'}|z_{d},z_{d'}\sim \psi (.|z_{d},z_{d'},\eta )$
     where $z_{d}=\{z_{d,1},z_{d,2},...,z_{d,n}\}$

Inference, Estimation and Prediction

• Inference
• Estimation
• Prediction

 - Link prediction from words
   $p(y_{d,d'}|w_{d},w_{d'})\approx E_{q}[p(y_{d,d'}|{\bar {z_{d}}},{\bar {z_{d'}}})]$
 - Words prediction from link
   $p(w_{d,i}|y_{d})\approx E_{q}[p(w_{d,i}|z_{d,i})]$

Data

• Cora: abstracts + citation link
• WebKB: web pages + hyperlinks
• PNAS: abstracts + intra-PNAS citation
• LocalNews: local news of each state in U.S + geographical adjacency

Results