|
|
| Line 10: |
Line 10: |
| | The problem of SRL is usually broken into two parts: identifying candidate phrases for assigning semantic roles, and predicting the semantic role to be assigned to the identified phrase. The approach in this paper does both these things in a single pass over the syntactic tree structure. | | The problem of SRL is usually broken into two parts: identifying candidate phrases for assigning semantic roles, and predicting the semantic role to be assigned to the identified phrase. The approach in this paper does both these things in a single pass over the syntactic tree structure. |
| | | | |
| − | ==Sentiment Analysis of Rated Comments== | + | ==Dataset Used== |
| − | The authors first analyzed the comments for their sentiments to prove their hypothesis that positively rated comments have positive sentiment and vice-versa. They first categorized the comments into three categories "5Neg" (comments that have a negative rating of 5 or higher), "0Dist" (comments that have not got any rating) and "5Pos" (comments that have a positive rating of 5 or higher). Then the terms in these comments were assigned a sentiment score using SentiWordNet. SentiWordNet has a score triplet in the form of (positivity-score,negativity-score,objectivity-score) for each word present in WordNet. The authors just considered the adjectives present in the comments to be tagged for their sentiment scores. Experiments showed that negatively rated comments had more negative sentiment terms, and positively rated comments had more positive sentiment terms. Authors further did an "analysis of variance" test to prove that the mean of sentiment scores for the three categories varied significantly across any two categories. | + | The dataset used was the Propbank corpus, which is the Penn Treebank corpus with semantic role annotation. |
| | | | |
| − | ==Predicting Rating for Comments== | + | ==CRF Model== |
| − | After the above analysis, the authors did an SVM based classification of the comments. The comment was considered as a vector of sentiment values of the terms present in the comment. The classification was binary with the classes being positive/accepted or negative/unaccepted. For this experiment, authors considered distinct thresholds for the minimum and maximum ratings (above/below +2/-2, +5/-5, +7/-7) for comments to be considered accepted or unaccepted. The authors also chose different amounts of randomly chosen accepted and unaccepted comments (T=1000,10000,50000,200000) for training. At least 1000 comments in each of the classes were kept for testing. Three experiments were conducted. First was classification with accepted comments marked as accepted, and unaccepted comments marked as unaccepted. Second was classification with accepted comments marked as unaccepted, and unaccepted comments marked as accepted; this was done to find the "bad" or erroneous comments. The third experiment was with comments with high rating (positive or negative) and the ones with no rating. The three scenarios are labeled AC_POS, AC_NEG and THRES-0 in the results below.
| + | The CRF was defined over the tree structure of the sentence as:<br> |
| | + | <math>p(y|x) = w(\lambda)*(w_{com}\gamma_p + InfluenceFlow(p))</math><br> |
| | + | [[File:]] |
| | | | |
| | ==Results for Rating Prediction== | | ==Results for Rating Prediction== |
| | [[File:Results_youtube.jpg]] | | [[File:Results_youtube.jpg]] |
| − |
| |
| − | ==Variance of Comments Rating as Indicator of Polarizing topics==
| |
| − | The authors also analyzed the relation between variance of comments rating and the polarity of videos. 1413 tags from 50 videos were selected and average variance of comment ratings was calculated over all videos having a particular tag. The table below shows top-25 and bottom-25 tags according to the average variance. We can see that tags in top-25 videos tend to be related to more polarizing topics, and the ones in bottom-25 videos tend to be related to rather neutral topics.
| |
| − |
| |
| − | [[File:Results_youtube_variance.jpg]]
| |
| − |
| |
| − | ==Category Dependencies of Ratings==
| |
| − | Authors conducted the classification experiments separately for comments in three different categories: Music, Entertainment, and News & Politics. The results of these experiments are as shown in the figure below. While classification did comparably well for Entertainment and Music categories, it didn't do that well for News & Politics category. In addition, authors also found mean rating scores for comments in a variety of categories. The results are as in the figure below.
| |
| − |
| |
| − | [[File:Results_youtube_categories.jpg]]
| |
| − | [[File:Results_youtube_categories-2.jpg]]
| |
Citation
Trevor Cohn, Philip Blunsom, "Semantic Role Labeling with Conditional Random Fields", CoNLL 2005
Online version
Click here to download
Introduction
This paper aims at Semantic Role Labeling or SRL of sentences using Conditional Random Fields. This was the first attempt of solving the problem of SRL using CRF. The authors defined CRF over the tree structure of the syntactic parse tree of the sentence, rather than defining it on the linear sentence structure as is usually done for the tasks of Named Entity Recognition or Part-of-Speech tagging. The motivation behind this came from the very nature of semantic role labeling which is the task of labeling phrases with their semantic labels with respect to a particular constituent of the sentence, the predicate or the verb. The authors conjectured that for this reason, modeling linear chain CRF was not intuitive for SRL.
The problem of SRL is usually broken into two parts: identifying candidate phrases for assigning semantic roles, and predicting the semantic role to be assigned to the identified phrase. The approach in this paper does both these things in a single pass over the syntactic tree structure.
Dataset Used
The dataset used was the Propbank corpus, which is the Penn Treebank corpus with semantic role annotation.
CRF Model
The CRF was defined over the tree structure of the sentence as:
[[File:]]
Results for Rating Prediction
