Difference between revisions of "Chiang 2005"
| Line 24: | Line 24: | ||
To solve this problem, the proposal is to have pairs of hierarchical phrases that consist of both words and subphrases. For example, the following pairs along with conventional phrase pairs are sufficient to translate the previous sentence correctly: | To solve this problem, the proposal is to have pairs of hierarchical phrases that consist of both words and subphrases. For example, the following pairs along with conventional phrase pairs are sufficient to translate the previous sentence correctly: | ||
| − | <math> | + | <math>X \rightarrow \left \langle yu \; X_{[1]} \; you \; X_{[2]} \; , \; have \; X_{[1]} \; with \; X_{[1]} \right \rangle</math> |
| − | X \rightarrow \left \langle yu \; X_{[1]} \; you \; X_{[2]} \; , \; have \; X_{[1]} \; with \; X_{[1]} \right \rangle | + | <math>X \rightarrow \left \langle X_{[1]} \; de \; X_{[2]} , \; the \; X_{[2]} \; that \; X_{[1]} \right \rangle</math> |
| − | X \rightarrow \left \langle X_{[1]} \; de \; X_{[2]} , \; the \; X_{[2]} \; that \; X_{[1]} \right \rangle | + | <math>X \rightarrow \left \langle X_{[1]} \; zhiyi \; , \; one of \; X_{[1]} \right \rangle</math> |
| − | X \rightarrow \left \langle X_{[1]} \; zhiyi \; , \; one of \; X_{[1]} \right \rangle | ||
| − | </math> | ||
These pairs are formally defined as productions of a synchronous CFG. | These pairs are formally defined as productions of a synchronous CFG. | ||
Revision as of 19:33, 1 November 2011
Contents
Citation
Chiang, D. 2005. A Hierarchical Phrase-Based Model for Statistical Machine Translation. In Proceedings of the 43rd Annual Meeting of the ACL, pp. 263–270, Ann Arbor. Association for Computational Linguistics.
Online version
Information Sciences Institute, University of Southern California
Summary
This paper presents a statistical phrase-based machine translation model that uses hierarchical phrases (phrases that contain subphrases). The model is formally syntax-based because it uses Synchronous Context-Free Grammars (synchronous CFG) but not linguistically syntax-based because the grammar is learned from a parallel text without using any linguistic annotations or assumptions. Using BLEU as a metric, it is shown to outperform previous state-of-the-art phrase-based systems.
Motivation
The hierarchical model is motivated by the inability of conventional phrase-based models to learn reorderings of phrases (and not only local reorderings of words). For example, considering the following Mandarin sentence:
Aozhou shi yu Bei Han you bangjiao de shaoshu guojia zhiyi Australia is with North Korea have diplomatic relations that few countries one of (Australia is one of the few countries that have diplomatic relations with North Korea)
the typical output of a conventional phrase-based system would be:
Australia is diplomatic relations with North Korea is one of the few countries
because it is able to do the local reorderings of "diplomatic ... Korea" and "one ... countries" but fails to perform the inversion of the two groups.
To solve this problem, the proposal is to have pairs of hierarchical phrases that consist of both words and subphrases. For example, the following pairs along with conventional phrase pairs are sufficient to translate the previous sentence correctly:
![{\displaystyle X\rightarrow \left\langle yu\;X_{[1]}\;you\;X_{[2]}\;,\;have\;X_{[1]}\;with\;X_{[1]}\right\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/93d8ca39d141036756568d0378bb9dafb8b65927)
![{\displaystyle X\rightarrow \left\langle X_{[1]}\;de\;X_{[2]},\;the\;X_{[2]}\;that\;X_{[1]}\right\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/8dafb3a475cf054f506a74474fc4da52dea9c70e)
![{\displaystyle X\rightarrow \left\langle X_{[1]}\;zhiyi\;,\;oneof\;X_{[1]}\right\rangle }](https://wikimedia.org/api/rest_v1/media/math/render/svg/e5a7738241abac35c4a9e1dd0c8cb8420965e703)
These pairs are formally defined as productions of a synchronous CFG.
