KL Divergence

Its a metric which is used to measure the similarity between two probability distribution "P" and "Q".

For probability distributions P and Q of a discrete random variable their K–L divergence is defined to be

$D_{\mathrm {KL} }(P\|Q)=\sum _{i}P(i)\log {\frac {P(i)}{Q(i)}}.\!$