IBM Model 1

Citation

Brown, P. F., Pietra, V. J. D., Pietra, S. A. D., & Mercer, R. L. (1993). The mathematics of statistical machine translation: parameter estimation. Comput. Linguist., 19, 263–311.

Online version

pdf

Summary

IBM Model 1 is a word alignment model, which uses an Expectation Maximization Algorithm to compute the lexical translation probabilities in parallel texts.

Model

IBM Model 1 defines the probability of a sentence $s_{1}^{J}$, with length $J$, being translated to a sentence $t_{1}^{I}$, with length $I$, with the alignment $a_{1}^{J}$ as:

$Pr(t,a|s)={\frac {\epsilon }{(J+1)^{I}}}\prod _{j=1}^{J}{tr(t_{j}|s_{a(j)})}$

Where the alignment $a_{1}^{J}$ is a function that maps each word $t_{j}$ to a word $s_{i}$, by their indexes. These alignments can be viewed as an object for indicating the corresponding words in a parallel text. We can see that the sentence translation probability $Pr(t,s)$, is decomposed into the product of the lexical translation probabilities $tr(t_{j}|s_{a(j)})$ of each word in the target $t_{1},...,t_{J}$ with the word that it is aligned to in the source $s_{a(j)}$. Additionally, target words that are not aligned with any source word are aligned with the null token, with the a lexical translation probability given by $tr(t_{j}|null)$. These are referred as null insertions. The normalizing factor ${\frac {\epsilon }{(I+1)^{J}}}$ ensures that $Pr(t,a|s)$ is a probability and is normalized over all possible alignments $a$ and all possible translations $t$.

Related Work

One of the problems of the IBM Model 1 is that it is very weak to reordering, since $p(f,a|s)$ is calculated using only the lexical translation probabilities $tr(t|s)$. Because of this, if the model is presented with 2 translations candidates $t_{1}$ and $t_{2}$ with the same lexical translations, but with different reordering of the translated words, the model scores both translations with the same score.

This problem is addressed in subsequent IBM Models, such as IBM Model 2.