Hierarchical phrase-based translation

This method improves statistical phrase-based machine translation models by using hierarchical phrases (phrases that contain subphrases). The model is formally syntax-based because it uses Synchronous Context-Free Grammars (synchronous CFG) but not linguistically syntax-based because the grammar is learned from parallel text without using any linguistic annotations or assumptions.

Motivation

The hierarchical model is motivated by the inability of conventional phrase-based models to learn reorderings of phrases (they only learn local reorderings of words). For example, considering the following Mandarin sentence:

Aozhou    shi yu   Bei   Han   you  bangjiao             de   shaoshu guojia    zhiyi
Australia is  with North Korea have diplomatic relations that few     countries one of

(Australia is one of the few countries that have diplomatic relations with North Korea)

the typical output of a conventional phrase-based system would be:

Australia is diplomatic relations with North Korea is one of the few countries

because it is able to do the local reorderings of "diplomatic ... Korea" and "one ... countries" but fails to perform the inversion of the two groups.

To solve this problem, the proposal is to have pairs of hierarchical phrases that consist of both words and subphrases. These pairs are formally defined as productions of a synchronous CFG. Then, in the previous example, the following productions are sufficient to translate the sentence correctly:

Definition

Based on the definition of synchronous CFGs, the basic elements of the model are weighted rewrite rules with aligned pairs of right-handed sides, of the form: $X\rightarrow \left\langle \gamma ,\alpha ,\sim \right\rangle$, where $\,X\,$ is a non-terminal, $\,\gamma \,$ and $\,\alpha \,$ are strings of terminals and non-terminals (one in the source-language and the other in the target-language), and $\,\sim \,$ is a one-to-one correspondence between non-terminal occurences in $\,\gamma \,$ and $\,\alpha \,$. The weight of each rule is determined by a log-linear model:

where $\,\phi _{i}\,$ are features defined on rules. Additionally, the model uses two special "glue" rules which enables the model to build only partial translations with hierarchical phrases and then serially combine them:

The weight of the first special rule is $exp(-\lambda _{g})$ which controls the preference for hierarchical phrases over serial combination of phrases, and the weight of the second one is always one. The following partial derivation of a synchronous CFG shows how the "glue" rules and the standard ones are combined together: