IBM Model 2

Citation

Brown, P. F., Pietra, V. J. D., Pietra, S. A. D., & Mercer, R. L. (1993). The mathematics of statistical machine translation: parameter estimation. Comput. Linguist., 19, 263–311.

Online version

pdf

Summary

IBM Model 2 is an extension to [IBM Model 1].

This model addressed the weak reordering properties of IBM Model 1 by modeling the absolution distortion between the words in parallel sentence.

Model

One of the problems of the IBM Model 1 is that it is very weak to reordering, since $p(f,a|s)$ is calculated using only the lexical translation probabilities $tr(t|s)$. Because of this, if the model is presented with 2 translations candidates $t_{1}$ and $t_{2}$ with the same lexical translations, but with different reordering of the translated words, the model scores both translations with the same score.

Mixture-based Alignment models~(IBM Model 2) addresses this problem by modeling the absolute distortion in the word positioning between the 2 languages, introducing an alignment probability distribution $Pr_{a}(i|j,J,I)$, where $i$ and $j$ are the word positions in the source and target sentences. Thus the equation for $Pr(t,a|s)$ becomes:

$Pr(t,a|s)={\frac {\epsilon }{(J+1)^{I}}}\prod _{j=1}^{J}{tr(t_{j}|s_{a(j)})Pr_{a}(a(j)|j,J,I)}$

Where the alignment probability distribution $Pr_{a}(a(j)|j,J,I)$ models the probability of a word in the position $i$ in the source sentence of being reordered into the position $j$ in the target sentence.