Yang et al, SIGIR 98

From Cohen Courses
Revision as of 20:04, 26 September 2012 by Ysim (talk | contribs) (Created page with 'This [[Category::Paper]] is relevant to our project on detecting controversial events in Twitter. == Citation == Yiming Yang, Thomas Pierce, and Jaime Carbonell. A study on ret…')
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

This Paper is relevant to our project on detecting controversial events in Twitter.

Citation

Yiming Yang, Thomas Pierce, and Jaime Carbonell. A study on retrospective and online event detection. In Proc. ACM SIGIR, pages 28–36, Melbourne, 1998.

Online version

A study on retrospective and online event detection

Summary

This paper addresses the problems of detecting events in news stories. They present solutions for retrospective event detection and online event detection using clustering techniques: group average clustering and single pass clustering. They addressed the problem of the streaming nature of their data by doing incremental IDF, where the IDF values of terms in the corpus is incrementally updated as a new document is observed. Furthermore, they use a time window to limit the search space for similar news events to the last m received stories. They also tried reweighting similarity scores according to the temporal proximity of two documents.

They experimented with the Topic Detection and Tracking corpus.

Evaluation

They evaluated the ability of their systems to recover news events retrspectively, and also in an online setting. They compared their system's performance to human judgements for two specific events to analyse the behaviour of their algorithm.

Discussion

This paper presents a bag-of-words clustering approach to detecting new events in a news corpus. They showed how online detection is a more difficult problem than retrospective detection. This paper poses two important social problems related to bipartite social graphs and explained how those problems can be solved efficiently using random walks.

Related papers

There has been a lot of work on event detection.

Study plan

  • Article: Group average agglomerative clustering [1]
  • Article: Single pass clustering [2]