Ravi and Knight, ACL 2011

Citation

S. Ravi and K. Knight. 2011. Deciphering Foreign Language. In Proceedings of ACL.

Online version

pdf

Summary

This work addresses the Machine Translation problem without resorting to parallel training data.

This is done by looking at the Machine Translation task from the decipherment perspective, where a sentence in the source language is viewed as the sentence target, but encoded in some unknown symbols.

Experimental showed that, while the results using monolingual data were considerably lower than those using bilingual data if the same amount of data is used, large amounts of monolingual data can be used to create models that perform similarly to systems that use smaller amounts of bilingual data. This is encouraging, since bilingual data is a scarce resource for most language pairs and domains, while monolingual data is much more abundant.

Description of the Method

Word Alignments using parallel corpora is viewed as a maximization problem with latent word alignments $a$ for a set of sentence pairs $(s,t)$, given by:

$argmax_{\theta }\prod _{s,t}P_{\theta }(s|t)=argmax_{\theta }\prod _{s,t}\sum _{a}P_{\theta }(s,a|t)$

where $\theta$ are the translation parameters of the model.

When only monolingual corpora is used, for each source sentence $s$, there isn't an exact target sentence that is aligned to the source sentence. Thus, like the word alignments, this work views the hidden target sentence as an additional latent variable. Hence, the previous equation can be rewritten as:

$argmax_{\theta }\prod _{s}\sum _{t}P(t)\sum _{a}P_{\theta }(s,a|t)$

where P(t) is the probability of a target sentence $t$, modeled by the a language model.

The problem of using the target sentence as a hidden variable is that all possible sequences of target words must be tested during EM, which is intractable due to the vocabulary size.