View source for Comparison: O'Connor et al. ICWSM 2010 & Widespread Worry and Stock Market

From Cohen Courses

← Comparison: O'Connor et al. ICWSM 2010 & Widespread Worry and Stock Market

You do not have permission to edit this page, for the following reason:

---

You can view and copy the source of this page.

== Papers == * [[OConnor et. al., ICWSM 2010]] Brendan O’Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010a. From tweets to polls: Linking text sentiment to public opinion time series. In Proc. of ICWSM. * [[Widespread Worry and the Stock Market]] Gilbert, E. and Karahalios, K., Widespread worry and the stock market, 2010, In Proceedings of the international conference on weblogs and social media (ICWSM 10). == Method == The two papers used different methodologies. The O'Conner et al. paper studied sentiment analysis of Twitter data by using a sentiment lexicon to mark positive or negative tweets, whereas the Gilbert et al. paper used supervised learning classifiers to automatically identify ''anxious'' LiveJournal blog posts. Furthermore, Gilbert et al. analyzed the usefulness of time series data from their Anxiety index in forecasting S&P 500 stock market data using Granger causality, whereas O'Connor et al. simply studied the smoothed correlation between public opinion polls and their aggregated Twitter sentiment. == Dataset Used == * Gilbert et al. used two data sets: [[UsesDataset::LiveJournal blog]] dataset and the [[UsesDataset::S&P 500]] dataset. * O'Connor et al. uses a dataset consisting of roughly 1 billion [[UsesDataset::Twitter]] posts collected between 2008-2009. == Problem == Gilbert et al. was concerned with finding a blog-based economic indicator that could be used to predict movements of the S&P 500. O'Connor et al. was concerned with using aggregated sentiment of Twitter posts as an alternative (or supplement) to traditional public opinion polling, for such areas as consumer confidence or presidential approval ratings. == Big Idea == The big idea common to both papers is that public sentiment can be roughly estimated from large natural language crowd-sourced textual data (LiveJournal blogs and Twitter). The high-level goal of both papers is to use the textual data to estimate the latent sentiment of the general population, and use these estimates as alternatives to costly social science polling. == Other == The correlations reported in the O'Connor et al. paper seemed fairly significant (> 75% in many cases), whereas the results in the Gilbert et al. paper were not as strong, with fairly large p-values in many cases. == Additional Questions == How much time did you spend reading the (new, non-wikified) paper you summarized? 1 hour How much time did you spend reading the old wikified paper? 1 hour How much time did you spend reading the summary of the old paper? 20 minutes How much time did you spend reading background material? 20 minutes Was there a study plan for the old paper? Yes If so, did you read any of the items suggested by the study plan? and how much time did you spend with reading them? No. Overall, I found the previous summary useful for getting a high-level understanding of what the O'Connor et al. paper was about, and it helped me know what to look for in terms of comparisons when reading the new paper. To get an understanding of the actual methodology and relevant facts concerning the old paper, however, required me to read it.

Return to Comparison: O'Connor et al. ICWSM 2010 & Widespread Worry and Stock Market.