Project Proposal Second Draft:Daniel and Sherry
Contents
Team Members
Dataset
Russian movie social network data: This dataset, which can only be used in this project and cannot be released publicly, consists of two files. The first one includes directed connection between users and the second one contains user ratings on Russian movies. The dataset contains approximately 65,000 users, and 250,000 friendship links, as well as 16 million user ratings of movies.
Project Ideas
We propose to explore the properties of a large social network tied with movie ratings. We will investigate a variety of topics within this, using a mix of supervised and unsupervised machine learning methods.
Tasks
- Predict user ratings of movies based on their ratings of other movies and ratings made by their friends in addition to using social structure of the network.
- Predict evolution of the social network using interest similarity and network structure.
- Detect hidden communities.
Evaluation
Using cross-validation, we can compare predicted user ratings with actual ratings. Prediction of change in the network can be directly measured using recall and precision of new links predicted. Hidden communities are harder to evaluate, but can potentially be used as features in other tasks.
Potential Methods
- Analyze data for correlation before doing anything else
- Linear classification for link prediction
- Graph clustering, such as spectral clustering
- Regression models
- Collaborative Filtering for rating prediction
Midterm Goals
- Complete analysis of data
- Implement baseline systems for all three tasks
- Begin work on final systems
References
Mehta, B. et al., 2007. Robust Collaborative Filtering. RecSys 2007.
Backstrom, L. et al., 2006. Group Formation in Large Social Networks: Membership, Growth, and Evolution. KDD 2006.
