A Latent Variable Model for Geographic Lexical Variation
Contents
Citation
A Latent Variable Model for Geographic Lexical Variation. Jacob Eisenstein, Brendan O'Connor, Noah A. Smith, and Eric P. Xing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), Cambridge, MA, October 2010.
Online version
Summary
This paper aims to analyze the variation in the usage of words in vernacular wrt geography. In particular, it analyzes lexical variation by both topic and geography. It also separates regions into coherent linguistic communities. Also it can predict with some accuracy the location of the author from raw text.
They develop a model that incorporates two sources of lexical variation : topic and geographical region, both as latent variables. At the base level of the model are (as referred in the paper) "pure" topics (such as sports, weather, slangs) and these topics are used differently in different geographic regions.
Data
This work is based on the Twitter dataset which can be found here. Only GeoTagged data is used. Also they choose users based on certain criterias such as, they should be active on twitter (wrote atleast 20 messages over the period) and should follow less than 1000 people and have less than 1000 followers (so they are not celebrities or influential people)
