Pointwise mutual information
If X and Y are random variables, the pointwise mutual information between two possible outcomes X=x and Y=y is
This quantity is zero if x and y are independent, positive if they are positively correlated, and negative if they are negatively correlated.
In more detail, Turney interpreted "X=x and Y=y" as an event where two words x and y occur nearby in the same document, and "X=x" as an event where word x occurs in a document. After some simplification, SO(x) can then be written as
This means that SO(x) can be computed quickly - with just two queries to a search engine.
|Turney, ACL 2002||Review classification|