View source for MCMC algorithm for posterior inference under hierarchical Dirichlet process mixtures

The Chinese restaurant franchise can yield a Gibbs sampling scheme for posterior sampling given observations x. Rather than dealing with the <math>\theta_{ji}</math>'s and <math>\psi_{jt}</math>'s, we shall sample their index variables <math>t_{ji}</math> and <math>k_jt</math> instead.


* '''Sampling t.''' The prior probability that <math>t_{ji}</math> takes on a particular previously used value t is proportional to <math>n_{jt.}^{-ji}</math>, whereas the probability that it takes on a new value (say <math>t^{new} = m_{j.} + 1</math>) is proportional to <math>\alpha_0</math>. The likelihood due to <math>x_{ji}</math> given <math>t_{ji} = t</math> for some previously used t is <math>f_k^{-x_{ji}} (x_ji)</math>, then

[[File:hdp_eq1.png]]

If the sampled value of <math>t_ji</math> is <math>t^{new}</math>, we obtain a sample of <math>k_{jt^{new}}</math> by sampling from

[[File:hdp_eq2.png]]

If as a result of updating <math>t_{ji}</math> some table t becomes unoccupied, i.e., <math>n_{jt.} = 0</math>, then the probability that this table will be reoccupied in the future will be zero, since this is always proportional to <math>n_{jt.}</math>. As a result, we may delete the corresponding <math>k_{jt}</math> from the data structure. If as a result of deleting
<math>k_{jt}</math> some mixture component k becomes unallocated, we delete this mixture component as well.

* '''Sampling k'''. Since changing <math>k_{jt}</math> actually changes the component membership of all data items in table t, the likelihood obtained by setting <math>k_{jt} = k</math> is given by <math>f_{k}^{-x_{jt}}(x_{jt})</math>, so that the conditional
probability of <math>k_{jt}</math> is

[[File:hdp_eq3.png]]