View source for Pointwise mutual information

This is a [[category::method]] discussed in [[Social Media Analysis 10-802 in Spring 2010]].

If X and Y are random variables, the pointwise mutual information between two possible outcomes X=x and Y=y is 

: <math>
PMI(x,y) = \log\frac{Pr(X=x,Y=y)}{Pr(X=x)Pr(Y=y)}.
</math>

This quantity is zero if x and y are independent, positive if they are positively correlated, and negative if they are negatively correlated.

In [[Turney, ACL 2002]] this was used as a way of assessing the [[semantic orientation of words]] or phrases.  Specifically the semantic orientation of x was defined as  

: <math>
 SO(x) = PMI(x,\textit{'excellent'}) - PMI(x,\textit{'poor'})
</math>

In more detail, Turney interpreted "X=x and Y=y" as an event where two words x and y occur nearby in the same document, and "X=x" as an event where word x occurs in a document.  After some simplification, SO(x) can then be written as

: <math>
 \log \frac{\textit{Hits(x~near~'excellent')} \cdot \textit{Hits('poor')}}{\textit{Hits(x~near~'poor')}\cdot \textit{Hits('excellent')}}
</math>

This means that SO(x) can be computed quickly - with just two queries to a search engine.


== Relevant Papers ==

{{#ask: [[UsesMethod::Pointwise mutual information]]
| ?AddressesProblem
| ?UsesDataset
}}