Shmueli et. al. WWW2012

Citation

E Shmueli, A Kagian, Y Koren, R Lempel Care to Comment? Recommendations for Commenting on News Stories WWW2012

Summary

In this paper, the authors address the following problem, "How to recommend News Stories to recommend?"

They treat this problem as a classic recommendation problem. They use two approaches to recommend article to users. Memory based approach and Latent Factor Model.

Datasets

The author established two data sets: News Vine and Yahoo! For News Vine, the author crawled all the articles from May to September 2011, including all the tags and all the associated comments. They also crawled the friends relationships from for each commender.

And the Yahoo! data sets are constructed following the same way.

Approaches

1.Memory Based Approach. The authors defined a co-occurrence similarity function using both tag information and co-commenter information. Then rank stories using this function.

2.Latent Factor Model The authors define a K dimensional latent space for both users and stories and the product of two vectors plus a bias function as a rank function. Then the model is trained by minimizing a loss function defined later, using stochastic gradient descent.

Evaluation

The evaluation is performed as follows, the authors randomly selected pairs of stories, one from the stories the user made a commend and those stories the user didn't. Then using the measure defined above, to assign scores to pick one of the story over another, then using AUC metric to evaluate the performance.

As to the Latent Model approach, the user using one of the following metrics, to define the loss of misclassification.

1.Squared Error

2.Classification Error

3.Bayesian Personalized Ranking

4.Rank Loss

The result shows that on the Newsvine Datasets, Latent Factor model with BPR loss function performs the best, and on Yahoo! dataset the LFM+RL performs better.

Stratified AUC

The authors found some problem on the above methods have a common weakness: that is many stories are unlikely to be commend by any user. It makes more sense to distinguish between stories that has a similar popularity.

So the author put stories into different popularity bins, the stories in the same bin share the same popularity.

Then AUC is computed on each bins and averaged proportionally to the number of positive examples in each bin.

This time it shows that BPR has the best performance on Newsvine datasets while CE has the best performance on Yahoo.

Feature Selection and Social Data

As mentioned before the author combined tag information and co-comment information in the rank function. Then the author trained model using only one of the model to see which feature is more significant. They discover that the model using both features performs best.

They also introduced friends relationship data in the NewVine datasets and the boost on AUC and sAUC are both very small.

My Opinion

I think the approaches this paper proposes have the following weaknesses:

1. Ignore the freshness of a news article. Because news is different than products items, they only lived for a short period of time.

2. The text representation doesn't catch the features of news article. The author only use tag information in their paper.

3. Co-commending is not a good feature. The tag feature outperforms co-commend feature on sAUC, which means when the popularity are even, the user tends to commend stories based on the what he interested. And those models only leverage tag information, which doesn't leverage the richness of text information. If introducing those features, I believe the content feature will outperforms the co-commend features much more.