Difference between revisions of "Posterior Regularization for Expectation Maximization"
From Cohen Courses
Jump to navigationJump to search| Line 4: | Line 4: | ||
== Method Description == | == Method Description == | ||
| − | For a given set x of observed data, a set of latent data z and a set of parameters <math>\theta</math>, the [[Expectation Maximization]] algorithm can be viewed as the alternation between two maximization steps. | + | For a given set x of observed data, a set of latent data z and a set of parameters <math>\theta</math>, the [[Expectation Maximization]] algorithm can be viewed as the alternation between two maximization steps of the function <math>F(q,\theta)</math>. |
| − | + | ||
| + | The E-step is defined as: | ||
<math> | <math> | ||
| Line 12: | Line 13: | ||
where <math>D_{KL}</math> is the Kullback-Leibler divergence given by <math>D_{KL}(q||p) = E_q[log \frac{q}{p}]</math> | where <math>D_{KL}</math> is the Kullback-Leibler divergence given by <math>D_{KL}(q||p) = E_q[log \frac{q}{p}]</math> | ||
| + | |||
| + | The M-step is defined as: | ||
Revision as of 18:25, 29 September 2011
Summary
This is a method to impose contraints on posteriors in the Expectation Maximization algorithm, allowing a finer-level control over these posteriors.
Method Description
For a given set x of observed data, a set of latent data z and a set of parameters , the Expectation Maximization algorithm can be viewed as the alternation between two maximization steps of the function .
The E-step is defined as:
![{\displaystyle q^{t+1}=argmax_{q}F(q,\theta ^{t})=argmax_{q}[-D_{KL}(q||p_{\theta ^{t}}(z|x))]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e1e66cb100264131555a54cdef16b4c71009b5bf)
where is the Kullback-Leibler divergence given by
![{\displaystyle D_{KL}(q||p)=E_{q}[log{\frac {q}{p}}]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/080d8fe6c8d590b57eeded290d911f717cd878b2)
The M-step is defined as:
