Martins et al 2010

Citation and Online Link

A. F. T. Martins, K. Gimpel. N. A. Smith, E. P. Xing, P. M. Q. Aguiar, M. A. T. Figueiredo, 2010. Aggressive Online Learning of Structured Classifiers. Technical report CMU-ML-10-109.

Summary

This paper generalizes the loss function of CRFs, structured SVMs, structured perceptron, and Softmax-margin CRFs into a single loss function, and then derives an online learning algorithm that can be used to learn with that more general loss function. For the hinge loss, the learning algorithm reduces to MIRA.

Method

The general loss function is:

Different choices of $\mathrm {B}$ and $\gamma$ correspond to various well known loss functions. They are:

The function minimized is the loss function with a regularizer:

The online learning algorithm proposed to minimize this function is called Dual Coordinate Ascent (DCA):

The parameters can be updated using algorithm 2:

Related Papers

MIRA CRF Softmax-margin CRFs

In progress by User:Jmflanig