Difference between revisions of "Training SMT Systems with Latent SVMs"
| (6 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
== Proposal == | == Proposal == | ||
Large-scale discriminative training of MT systems has been a long standing goal in statistical machine translation. One of the first attempts ([http://cs.stanford.edu/~pliang/papers/discriminative-mt-acl2006.pdf Laing et al 2006] | Large-scale discriminative training of MT systems has been a long standing goal in statistical machine translation. One of the first attempts ([http://cs.stanford.edu/~pliang/papers/discriminative-mt-acl2006.pdf Laing et al 2006] | ||
| − | ) used the structured perceptron to train weights for each phrase in a phrase-based system as well as features shared between phrases. The approach can be viewed as an instance of the Latent Structured SVM ([http://www.cs.cornell.edu/~cnyu/papers/icml09_latentssvm.pdf Yu & Joachims ICML 2009]) but with no regularizer and no cost function. Regularization is shown to be important in | + | ) used the structured perceptron to train weights for each phrase in a phrase-based system as well as features shared between phrases. The approach can be viewed as an instance of the Latent Structured SVM ([http://www.cs.cornell.edu/~cnyu/papers/icml09_latentssvm.pdf Yu & Joachims ICML 2009]) but with no regularizer and no cost function. Regularization is shown to be important in discriminative training of SMT systems ([http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.164.9399&rep=rep1&type=pdf Blumsom]). We propose to generalize the perceptron training of SMT systems to the Latent SSVM to allow for a regularizer and cost function, and to apply the method to large-scale training of systactic SMT systems as well as a phrase-based system. |
| − | Our original project was to incorporate binary feedback into MT systems, but we arrived at the current proposal after we realized nobody had tried this important | + | Our original project was to incorporate binary feedback into MT systems, but we arrived at the current proposal after we realized nobody had tried this important training method. So if we have time we may try to extend our latent SSVM model to the recently introduced Structured Output Learning with Indirect Supervision, [http://www.icml2010.org/papers/522.pdf M. Chang et al, ICML 2010]. |
== Dataset(s) == | == Dataset(s) == | ||
| Line 19: | Line 19: | ||
== Baseline System == | == Baseline System == | ||
| − | The baseline systems will be a phrase-based system and a Hiero system, optimized using MERT with | + | The baseline systems will be a phrase-based system and a Hiero system, optimized using MERT with gamut of usual features. |
== Related Work == | == Related Work == | ||
| − | [http://cs.stanford.edu/~pliang/papers/discriminative-mt-acl2006.pdf Laing et al 2006] | + | * An end-to-end discriminative approach to machine translation, [http://cs.stanford.edu/~pliang/papers/discriminative-mt-acl2006.pdf Laing et al 2006] |
| − | [http://www.cs.cornell.edu/~cnyu/papers/icml09_latentssvm.pdf Yu & Joachims ICML 2009] | + | * Learning Structural SVMs with Latent Variables, [http://www.cs.cornell.edu/~cnyu/papers/icml09_latentssvm.pdf Yu & Joachims ICML 2009] |
Latest revision as of 22:55, 18 October 2011
(was Improving SMT word alignment with binary feedback)
Team Member(s)
Proposal
Large-scale discriminative training of MT systems has been a long standing goal in statistical machine translation. One of the first attempts (Laing et al 2006 ) used the structured perceptron to train weights for each phrase in a phrase-based system as well as features shared between phrases. The approach can be viewed as an instance of the Latent Structured SVM (Yu & Joachims ICML 2009) but with no regularizer and no cost function. Regularization is shown to be important in discriminative training of SMT systems (Blumsom). We propose to generalize the perceptron training of SMT systems to the Latent SSVM to allow for a regularizer and cost function, and to apply the method to large-scale training of systactic SMT systems as well as a phrase-based system.
Our original project was to incorporate binary feedback into MT systems, but we arrived at the current proposal after we realized nobody had tried this important training method. So if we have time we may try to extend our latent SSVM model to the recently introduced Structured Output Learning with Indirect Supervision, M. Chang et al, ICML 2010.
Dataset(s)
We will primarily use one dataset for the purposes of this project, which is the IWSLT 2009 Chinese-English btec task parallel corpus.
- The training set is > 500,000 parallel sentences.
- There are 9 development (tuning) sets, each with ~500 sentences (total of 4,250 sentences)
- The test set consists of 200 aligned sentences
Of course, we can always decide to use one of the tuning sets as a test set and vice versa.
Baseline System
The baseline systems will be a phrase-based system and a Hiero system, optimized using MERT with gamut of usual features.
Related Work
- An end-to-end discriminative approach to machine translation, Laing et al 2006
- Learning Structural SVMs with Latent Variables, Yu & Joachims ICML 2009
