Teh et, JASA2006

Citation

Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 2006

Online version

Mathew Beal's papers

Summary

This paper proposed a nonparametric Bayes approach to decide the number of mixture components in grouped data, the basic idea is:

• Develop analogs for the Hierarchical Dirichlet process with representations of both a stick-breaking and a "Chinese restaurant franchise”.

• Use MCMC algorithm for posterior inference under hierarchical Dirichlet process mixtures.

Methodology

A hierarchical Dirichlet process is a distribution over a set of random probability measures over $(\theta ;B)$. The process defines a set of random probability measures $G_{j}$, one for each group, and a global random probability measure $G_{0}$. The global measure $G_{0}$ is distributed as a Dirichlet process with concentration parameter and base probability measure H:

$G_{0}|\gamma ,H~DP(\gamma ,H)$

and the random measures Gj are conditionally independent given G0, with distributions given by a Dirichlet process with base probability measure G0:

$G_{j}|\alpha _{0},G_{0}~DP(\alpha _{0},G_{0})$.

A hierarchical Dirichlet process can be used as the prior distribution over the factors for grouped data. For each j let $\theta _{j1},\theta _{j2},...$ be i.i.d. random variables distributed as $G_{j}$ . Each $\theta _{j}i$ is a factor corresponding to a single observation $x_{ji}$. The likelihood is given by:

$\theta _{ji}|G_{j}~G_{j}$

$x_{ji}|\theta _{ji}~F(\theta _{ji})$.

The hierarchical Dirichlet process can readily be extended to more than two levels. That is, the base measure H can itself be a draw from a DP, and the hierarchy can be extended for as many levels as are deemed useful.