Difference between revisions of "Posterior Regularization for Expectation Maximization"

From Cohen Courses
Jump to navigationJump to search
Line 10: Line 10:
 
q^{t+1} = argmax_{q} F(q,\theta^t) = argmax_{q} [-D_{KL}(q||p_{\theta^t}(z|x))]
 
q^{t+1} = argmax_{q} F(q,\theta^t) = argmax_{q} [-D_{KL}(q||p_{\theta^t}(z|x))]
 
</math>
 
</math>
 +
 
where <math>D_{KL}</math> is the Kullback-Leibler divergence given by <math>D_{KL}(q||p) = E_q[log \frac{q}{p}]</math>
 
where <math>D_{KL}</math> is the Kullback-Leibler divergence given by <math>D_{KL}(q||p) = E_q[log \frac{q}{p}]</math>

Revision as of 17:20, 29 September 2011

Summary

This is a method to impose contraints on posteriors in the Expectation Maximization algorithm, allowing a finer-level control over these posteriors.

Method Description

For a given set x of observed data, a set of latent data z and a set of parameters , the Expectation Maximization algorithm can be viewed as the alternation between two maximization steps. Where the E-step is defined as:

where is the Kullback-Leibler divergence given by