David M. Blei and Pedro J. Moreno, Topic Segmentation with an Aspect Hidden Markov Model, SIGIR 2001

Citation

David M. Blei and Pedro J. Moreno, Topic Segmentation with an Aspect Hidden Markov Model, SIGIR 2001

Online version

Link to paper

Summary

This paper addresses the problem of Topic Segmentation on unstructured text. The paper is an extension to the previous approach that uses hidden Markov model (HMM) for modeling sequence of words in a document as being generated by a latent topic variable in a time series. The paper extends this idea by adding Hofmann's Aspect model to HMM to also model topical dependency between words and create a more cohesive segmentation model.

The paper applies this aspect HMM (AHMM) model to segment unbroken streams of New York Times articles as well as noisy transcripts of radio programs on SpeechBot. They show that AHMM outperforms HMM for this task.

Description of the method

In HMM model to segment a document, the document is treated as a collection of mutually independent sets of words. Each set is probabilistically generated by a hidden topic variable in a time series. Transition probabilities determine the value of the next hidden topic variable in the series.

Given the observed distribution in Flickr data of the probability of friendship over number of co-occurrences k, cell size s and temporal time t, a probabilistic model is proposed to fit the observed distribution. A simple model supposes that the world is divided into N geographic cells, with M people (each having one social tie). Each day each pair of friends chooses to visit a place jointly with probability β and independently with probability $1-\beta$. The choice of location itself is made randomly. Using Bayes' Law, the probability of friendship between two people F given that they visit the same cells on k consecutive days ($C_{k}$) is:

$P(F|C_{k})={\frac {P(F)P(C_{k}|F)}{P(C_{k})}}$

where prior probability of friendship between two people, $P(F)$:

$P(F)={\frac {1}{M-1}}$

and

$P(C_{k})=P(C_{k}|F)P(F)+P(C_{k}|{\bar {F}})P({\bar {F}})=p_{1}^{k}.{\frac {1}{M-1}}+p_{2}^{k}.{\frac {M-2}{M-1}}$

where $p_{1}$, probability of two friends being at the same place on 1 given day:

$p_{1}=\beta +{\frac {1-\beta }{N}}$

and $p_{2}$, probability of co-occurrence between two non-friends:

$p_{2}={\frac {1}{N}}$

Hence, $P(F|C_{k})$:

$P(F|C_{k})={\frac {p_{1}^{k}}{p_{1}^{k}+p_{2}^{k}(M-2)}}$

In a more complex model, each pair of friends is randomly chosen a "home" cell drawn from the empirical distribution of Flickr photographs (approximately a power law with exponent 2.45). When they choose a cell to visit on a given day, they sample from a distribution which is not uniform over all cells, but peaked around the home cell and decays with distance according to power law distribution (with exponent γ). Each day, a person independently decides whether to visit a cell with probability α or to do nothing. When two friends each choose to visit a cell (an event with probability $\alpha _{2}$), with probability β they end up in the same cell and with probability $1-\beta$, their cell selection is independent.

Datasets used

Using Flickr's public API interface, a dataset of about 85 million geo-tagged photographs is collected from Flickr. Photos with imprecise geo-tags and/or missing time stamps are removed. About 38 million photos taken by about 490,000 users remained. The social contacts of each of these users are then collected (if they are made public by the user). The dataset contains photos taken by Flickr users as well as their social contacts.

Experimental Results

Social Network Attribute

Using the dataset of 38 million geo-tagged photos from Flickr, the paper discovers that the probability of a social tie increases sharply as the number of co-occurrences k increases and the temporal range t decreases. Specifically, two randomly chosen Flickr users have 0.0134% chance of being friends, but when they have multiple spatio-temporal co-occurrences, this probability increases significantly: for example, two people have a 60% chance of having a social tie when they have k = 5 co-occurrences at t = 1 and s = 1° latitude-longitude. The observed log-scale probability of friendship (y-axis) over number of co-occurrences k (x-axis) at s = 1° is shown below:

In developing the model to qualitatively fit this interesting observed distribution, the complex model (described above) with parameters M = 7500, N = 64800, α = 0.29, β = 0.12, γ = 1.8 is found to match the observed distribution well. The model's log-scale probability of friendship (y-axis) over number of co-occurrences k (x-axis) at s = 1° is shown below:

Related Papers

The main contribution of this paper is to provide an analytical framework that quantify the "power" of spatio-temporal coincidences (no matter how sparse) and its effect in predicting probability of social ties. Other earlier works that attempt to expose such social network structure have been done using less sparse information on:

- Anonymized versions of the network itself: Backstrom L, Dwork C, Kleinberg J (2007) Wherefore art thou R3579X? Anonymized social networks, hidden patterns, and structural steganography. Proceedings of the 16th International World Wide Web Conference: link to paper discussed on the slides during Class Meeting for 10-802 04/07/2011, Narayanan A, Shmatikov V (2009) De-anonymizing social networks. Proceedings of the 30th IEEE Symposium on Security and Privacy pp 173–187: link to paper

- Commonalities in online behavior such as co-visitations to web sites: Provost F, Dalessandro B, Hook R, Zhang X, Murray A (2009) Audience selection for online brand advertising: Privacy-friendly social network targeting. Proceedings of the International Conference on Knowledge Discovery and Data Mining pp 707–716: link to paper and tagging shared content with similar keywords Schifanella R, Barrat A, Cattuto C, Markines B, Menczer F (2010) Folks in folksonomies: Social link prediction from shared metadata. Proceedings of the Third ACM International Conference on Web Search and Data Mining pp 271–280: link to paper

- Detailed time series of physical co-presence: Eagle N, Pentland A, Lazer D (2009) Inferring social network structure using mobile phone data. Proc Natl Acad Sci USA, 106 pp:15274–15278: link to paper

Discussion

The novelty of the paper lies on its quantitative treatment of spatio-temporal coincidences between people and how they are related to the likelihood of social ties. The paper does not address the question on whether or not friendship manifests themselves in pattern of repeated spatio-temporal coincidences. Rather, the strength of the paper lies in the opposite implication: that when two people exhibit multiple spatio-temporal coincidences, this is a strong indicator of a social tie, relative to the baseline frequency of such ties.

Unfortunately, although the paper proposes a probabilistic model to fit the distribution of friendship observed in Flickr data, the paper falls short in providing a quantitative evaluation of its proposed model, aside from that the model matches the observed distribution. No testing (prediction using the model) is conducted. An interesting next direction is perhaps to use the model to try and predict social ties among people based on their spatio-temporal coincidences. Another interesting direction is to explore this framework and model on another dataset, to discover whether or not the same model applies to different datasets, one that is not photography-related, for example.

Another interesting future direction is perhaps to explore whether it is possible to qualify the type of social ties between two persons, from its spatio-temporal coincidences - i.e. whether it is possible to differentiate strong ties from weak ones. For example, in photography, people may take pictures often with their friends (indicating strong ties). However, they may also take pictures often in popular public events or tourist destinations in which they are part of large crowds - hence decreasing the possibility of strong ties among observed co-occurrences. Differentiating such ties will be an interesting further direction.