MCMC algorithm for posterior inference under hierarchical Dirichlet process mixtures

The Chinese restaurant franchise can yield a Gibbs sampling scheme for posterior sampling given observations x. Rather than dealing with the $\theta _{ji}$'s and $\psi _{jt}$'s, we shall sample their index variables $t_{ji}$ and $k_{j}t$ instead.

• Sampling t. The prior probability that $t_{ji}$ takes on a particular previously used value t is proportional to $n_{jt.}^{-ji}$, whereas the probability that it takes on a new value (say $t^{new}=m_{j.}+1$) is proportional to $\alpha _{0}$. The likelihood due to $x_{ji}$ given $t_{ji}=t$ for some previously used t is $f_{k}^{-x_{ji}}(x_{j}i)$, then

If the sampled value of $t_{j}i$ is $t^{new}$, we obtain a sample of $k_{jt^{new}}$ by sampling from

If as a result of updating $t_{ji}$ some table t becomes unoccupied, i.e., $n_{jt.}=0$, then the probability that this table will be reoccupied in the future will be zero, since this is always proportional to $n_{jt.}$. As a result, we may delete the corresponding $k_{jt}$ from the data structure. If as a result of deleting $k_{jt}$ some mixture component k becomes unallocated, we delete this mixture component as well.

• Sampling k. Since changing $k_{jt}$ actually changes the component membership of all data items in table t, the likelihood obtained by setting $k_{jt}=k$ is given by $f_{k}^{-x_{jt}}(x_{jt})$, so that the conditional

probability of $k_{jt}$ is