Nlao writeup of Cohen 2005

This is a review of Cohen_2005_stacked_sequential_learning by user:Nlao.

Stacking is a very cool idea: learners are not trying to solve problem individually but to count on the output of each other.

A very interesting connection is with CRFs inference with Mean Field (MF) approximation. MF can be seen as a special case of Stacking, where the base learners are fixed to the mean field update equation, and there are many levels of stacking. From this point of view, stacking can be seen as a smart way of inference, where the update equation is allowed to vary between different update iterations, but there are only a few iterations.

Daume III (2009) hint on the connection between stacking and Reinforcement Learning (RL), which suggests the possibility of an efficient and effective stacking (or inference) method based on RL. However, except borrowing some terminology from RL, no actual RL technology was used.

[minor points]