Generalized Iterative Scaling

From Cohen Courses
Jump to navigationJump to search

The method

The Generalized Iterative Scaling (GIS) is a method that searches the exponential family of a Maximum Entropy solution of the form:

where the 's are some unknown constants to be found. The 's of the solution would be such that will make satisfy all the constraints , of the equation:

The Algorithm

GIS starts with arbitrary values, wich define the initial probability estimate:

Each next iteration is intended to create an estimate, that will match the constraints better than the last one. Each iteration follows the steps:

  • (1) Compute the expectations of all the 's under the current estimate function, i.e.

the circumstances under which it is meant to be used


you are expected to explain clearly what the method is

and list papers that use it


things the method is comparable to.

Explain what motivations or assumptions underlie the method

Intrinsic characteristics

GIS has three advantages when compared to other methods: it is able to incorporate feature selection, scales up well in numbers of features and is resilient to feature dependence.

On the other hand GIS has problems with smoothing and is relatively slow in training when compared to other classification methods

Related Papers