Posterior Regularization for Expectation Maximization

Summary

This is a method to impose contraints on posteriors in the Expectation Maximization algorithm, allowing a finer-level control over these posteriors.

Method Description

For a given set x of observed data, a set of latent data z and a set of parameters $\theta$, the Expectation Maximization algorithm can be viewed as the alternation between two maximization steps. Where the E-step is defined as:

$q^{t+1}=argmax_{q}F(q,\theta ^{t})=argmax_{q}-D_{KL}(q||p_{\theta ^{t}}(z|x))$