Difference between revisions of "Posterior Regularization for Expectation Maximization"
From Cohen Courses
Jump to navigationJump to searchLine 10: | Line 10: | ||
q^{t+1} = argmax_{q} F(q,\theta^t) = argmax_{q} [-D_{KL}(q||p_{\theta^t}(z|x))] | q^{t+1} = argmax_{q} F(q,\theta^t) = argmax_{q} [-D_{KL}(q||p_{\theta^t}(z|x))] | ||
</math> | </math> | ||
+ | |||
where <math>D_{KL}</math> is the Kullback-Leibler divergence given by <math>D_{KL}(q||p) = E_q[log \frac{q}{p}]</math> | where <math>D_{KL}</math> is the Kullback-Leibler divergence given by <math>D_{KL}(q||p) = E_q[log \frac{q}{p}]</math> |
Revision as of 17:20, 29 September 2011
Summary
This is a method to impose contraints on posteriors in the Expectation Maximization algorithm, allowing a finer-level control over these posteriors.
Method Description
For a given set x of observed data, a set of latent data z and a set of parameters , the Expectation Maximization algorithm can be viewed as the alternation between two maximization steps. Where the E-step is defined as:
where is the Kullback-Leibler divergence given by