Chang and Blei, AOAS2010

Citation

J. Chang and D. Blei. Hierarchical relational models for document networks. Annals of Applied Statistics, 4(1):124–150, 2010

Online version

D.Blei's papers

Motivation

Network data

 - social networks of friends
 - citation networks of documents
 - hyperlinked networks of web pages

“Predictive Models”

 - point social network members toward new friends
 - point scientific papers toward relevant citations
 - point web pages toward other related pages

“Descriptive statistics”

 - uncover the hidden community structure

This paper developed a hierarchical model of both network structure and node attributes, based on Latent Dirichlet Allocation.

Methodology

1. For each document $d$ :

 (a) Draw topic proportions  $\theta _{d}|\alpha \sim Dir(\alpha )$ 
 (b) For each word  $w_{d,n}$ :
     i.  Draw assignment  $z_{d,n}|\theta _{d}\sim Mult(\theta _{d})$ .
     ii. Draw word  $w_{d,n}|z_{d,n},\beta _{1:K}\sim Mult(\beta _{z_{d,n}})$ .

2. For each pair of documents $d$ , $d'$ :

 (a) Draw binary link indicator   
               $y_{d,d'}|z_{d},z_{d'}\sim \psi (.|z_{d},z_{d'},\eta )$ 
     where  $z_{d}=\{z_{d,1},z_{d,2},...,z_{d,n}\}$

Inference, Estimation and Prediction

Inference
Estimation
Prediction

 - Link prediction from words
    $p(y_{d,d'}|w_{d},w_{d'})\approx E_{q}[p(y_{d,d'}|{\bar {z_{d}}},{\bar {z_{d'}}})]$ 
 - Words prediction from link
    $p(w_{d,i}|y_{d})\approx E_{q}[p(w_{d,i}|z_{d,i})]$