# Chang and Blei, AOAS2010

## Citation

J. Chang and D. Blei. Hierarchical relational models for document networks. Annals of Applied Statistics, 4(1):124–150, 2010

## Motivation

For Network data, such as social networks of friends, citation networks of documents or hyperlinked networks of web pages, people want to point social network members toward new friends, scientific papers toward relevant citations or web pages toward other related pages. They also want to uncover the hidden community structure. This paper developed a hierarchical model of both network structure and node attributes, based on Latent Dirichlet Allocation.

## Methodology

1. For each document ${\displaystyle d}$:

 (a) Draw topic proportions ${\displaystyle \theta _{d}|\alpha \sim Dir(\alpha )}$
(b) For each word ${\displaystyle w_{d,n}}$:
i.  Draw assignment ${\displaystyle z_{d,n}|\theta _{d}\sim Mult(\theta _{d})}$.
ii. Draw word ${\displaystyle w_{d,n}|z_{d,n},\beta _{1:K}\sim Mult(\beta _{z_{d,n}})}$.


2. For each pair of documents ${\displaystyle d}$,${\displaystyle d'}$:

 (a) Draw binary link indicator
${\displaystyle y_{d,d'}|z_{d},z_{d'}\sim \psi (.|z_{d},z_{d'},\eta )}$
where ${\displaystyle z_{d}=\{z_{d,1},z_{d,2},...,z_{d,n}\}}$


## Inference, Estimation and Prediction

Prediction

   ${\displaystyle p(y_{d,d'}|w_{d},w_{d'})\approx E_{q}[p(y_{d,d'}|{\bar {z_{d}}},{\bar {z_{d'}}})]}$

   ${\displaystyle p(w_{d,i}|y_{d})\approx E_{q}[p(w_{d,i}|z_{d,i})]}$


## Data

• Cora: abstracts + citation link
• WebKB: web pages + hyperlinks
• PNAS: abstracts + intra-PNAS citation
• LocalNews: local news of each state in U.S + geographical adjacency

## Results

• Evaluating the predictive distribution