View source for Margin Infused Relaxed Algorithm

This [[Category::method]] is used by [[RelatedPaper::Watanabe_et_al.,_EMNLP_2007._Online_Large-Margin_Training_for_Statistical_Machine_Translation|Watanabe et al., EMNLP 2007]] to train an MT system a with a very large number of features of the order of millions. The training step was performed using a specific algorithm called the Margin Infused Relaxed Algorithm (MIRA) proposed by [[RelatedPaper::Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, Yoram Singer. Online Passive-Aggressive Algorithms. JMLR 7(Mar):551--585, 2006. | Crammer et al., 2006]]

== Summary ==
MIRA is an online large-margin training algorithm which updates the weight vector according to certain margin constraints and loss function. 

It is used to learn the weights of features after processing each training instance similar to [[UsesMethod::Voted_Perceptron | structured perceptron algorithm]] with an additional loss function and margin constraint in its update rule.

== Definition ==
A general definition of online training algorithms can be written down as follows:

[[File:Online-training.jpg|center|400px]]

<math>\forall t, \quad (f^{t}, \mathbf{e}^t) \in  \mathcal{T} </math> and a list of <math> m </math> -best oracles <math> \mathcal{O}^t </math>:
A <math>k</math> -best list of candidates is generated by <math> best_k(\cdot) </math> using the current weight vector <math>\mathbf{w}_i</math>. Each training instance <math>f_{t}</math> can have a multiple number of correct outputs or references, <math>\mathbf{e}^t</math>, in this case target translations.


Using the <math>k</math>-best list, <math>m</math>-best oracle translations <math>\mathcal{O}^t</math> is updated by <math>oracle_{m}(\cdot)</math> in each iteration.