Difference between revisions of "Jin et al, 2009"

From Cohen Courses
Jump to navigationJump to search
(Created page with '== Citation == Jin, W., Ho, H.,Srihari, R., 2009, OpinionMiner: A Novel Machine Learning System for Web Opinion Mining and Extraction, KDD'09 == Online version == [[http://por…')
 
 
(3 intermediate revisions by the same user not shown)
Line 9: Line 9:
 
== Summary ==
 
== Summary ==
  
−
This [[Category::paper]] introduces a system that mines customer reviews of a product and extract product features from the review. The system return opinion expression that are extracted from product review as well as opinion direction. [[AddressesProblem:: Opinion mining]] have been studied widely in machine learning and information extraction community. Most of these approaches have used statistical or rule-based learning to extract opinion expression. Jin et al. in this work have introduced a new technique that uses lexicalized HMM for opinion mining.  
+
This [[Category::paper]] introduces a system that mines customer reviews of a product and extracts product features from the review. The system returns opinion expression that are extracted from product review as well as opinion direction. [[AddressesProblem:: Opinion mining]] have been studied widely in machine learning and information extraction community. Most of these approaches have used statistical or rule-based learning to extract opinion expression. Jin et al. in this work have introduced a new technique that uses lexicalized HMM for opinion mining.  
  
 
== System Architecture ==
 
== System Architecture ==
 +
The architecture of their system is as follow:
  
 +
- Pre-processing: The system first crawls web pages from the Web. It cleans the HTML files that are crawled from the Web and segments all the sentences. The technique that is used to extract reviews of a product from the input web pages is not described in the paper. For the next parts of the architecture they have assumed that the reviews are extracted from the webpage and are given to the system.
  
 +
- Entity types and tag sets: They have defined four entity types for each product review: components (e.g. physical object of a camera), functions (e.g. zoom in a camera), features (e.g. color), and opinions (e.g. ideas and thoughts). For each if these types they have defined a set of tags that are used in annotation process.
  
−
The intuition behind their technique is to use global features to infer rules about the local features. For example suppose that we know the name of a set of books. Then by looking at webpages of Amazon.com and by searching the name of the books that we already have we can infer the position and font of the book title. We can then use these two features (position and font of book title in web pages) to extract new book titles from other web pages.  
+
- Lexicalized HMMs: Given a review of a product as an input of the system, the goal of the lexicalized HMM is to assign appropriate tag type to each part of the product review. For classification, they maximize conditional probability <math> P(T|W,S) </math> where T is the tags that should be assigned to different parts of a product review, <math> W </math> is a set of all the words in the review and <math> S </math> is the POS tag for each word. They have used MLE to learn parameters of the system.  
  
−
They have described both generative and discriminative approaches for classification and extraction tasks. Global features are governed by the parameters that are shared by all the data and local features are shared only by a subset of data. For example in information extraction task, all the words in a webpage (without considering formatting) can be considered as global features. On the other hand, features such as position of a text or color of text are local features.  
+
- Information propagation: The goal of this part is to decrease the number of training data that this system requires. Suppose that we have a sentence like "Good picture quality" as part of a review in the training data. Word "good" is tagged as "<opinion_pos_exp>" in the training data. The system then creates new training data by looking at a dictionary and substitute word "good" with it's synonyms. For example the new sentence "great picture quality" can be added as a new training data. This idea is applied to all the words in the training data to increase  the number of examples.
  
−
In generative model they have modeled each document by introducing a random variable that governs local features. The parameters of the model are:
+
- Bootstrapping: The main contribution of this paper is the bootstrapping part. The idea is to partition the training set to two different disjoint sets and train a HMM using each of these sets. Then for each instance of the test data (which is non annotated by the human), if two HMMs classify the input review as the same class and if the confidence value is above a threshold T then this new instance is added to the training set. This idea can significantly decrease the amount of time that human should spend to annotate training data.
  
−
- N words of documents are shown by <math> w=\{w_1,w_2,...,w_N\}</math>
+
== Evaluation Results ==  
−
 
+
They have tested their system on reviews of different cameras that are chosen from Amazon.com. They have manually annotated reviews of 6 cameras to use as the training data. The system is tested using 4-fold validation. They have used the system that is developed by [[Turney,2002]] as the baseline for comparisons. The results have shown that their system can increase accuracy of mining opinion expression by a factor of 2 comparing to the baseline system.
−
- Formatting features are shown by <math> f=\{f_1,f_2,...,f_N\} </math>
 
−
 
 
−
- Class labels are shown by <math> c=\{c_1,c_2,...,c_N\} </math>
 
−
 
 
−
The model can be shown by the following joint distribution over local parameters, class labels, words, and formatting features:
 
−
 
 
−
<math> p(\phi,c,w,f)=p(\phi)\prod_{i=1}^N p(c_n)p(w_n|c_n)p(f_n|c_n,\phi)</math>
 
−
 
 
−
The parameters are estimated using maximum likelihood estimation on a set of training documents. For inference, one approach is to approximate parameter <math> \phi </math> with a point estimation <math> \hat{\phi} </math> and infer the class label using MAP estimation. We can label each pair by the following formula:
 
−
 
 
−
<math> \hat{c_n}=argmax_{c_n}p(w_n|c_n)p(f_n|c_n,\hat{\phi})p(c_n) </math>
 
−
 
 
−
<math> \hat{\phi} </math> can be approximated by <math> \hat{\phi}=argmax_{\phi}p(\phi|f,w) </math>. They have used EM algorithm to maximize the expected log likelihood of formatting features.
 
−
 
 
−
They have tested their method on two different datasets. The first dataset contains 1000 HTML documents. Each document is automatically divided into a set of words with similar layout characteristics and then are hand-labeled as containing or not containing a job title. The local and global features for this domain are the same as what we explained above. The second dataset contains 42,548 web pages from 330 web sites which each web page is hand-labeled as if it is a press release or not press release. The global feature is a set of word in each webpage and local feature is the URL of the webpage. Their experimental result have shown that this approach can obtain high precision and low/moderate recall.
 

Latest revision as of 11:03, 2 December 2010

Citation

Jin, W., Ho, H.,Srihari, R., 2009, OpinionMiner: A Novel Machine Learning System for Web Opinion Mining and Extraction, KDD'09

Online version

[[1]]

Summary

This paper introduces a system that mines customer reviews of a product and extracts product features from the review. The system returns opinion expression that are extracted from product review as well as opinion direction. Opinion mining have been studied widely in machine learning and information extraction community. Most of these approaches have used statistical or rule-based learning to extract opinion expression. Jin et al. in this work have introduced a new technique that uses lexicalized HMM for opinion mining.

System Architecture

The architecture of their system is as follow:

- Pre-processing: The system first crawls web pages from the Web. It cleans the HTML files that are crawled from the Web and segments all the sentences. The technique that is used to extract reviews of a product from the input web pages is not described in the paper. For the next parts of the architecture they have assumed that the reviews are extracted from the webpage and are given to the system.

- Entity types and tag sets: They have defined four entity types for each product review: components (e.g. physical object of a camera), functions (e.g. zoom in a camera), features (e.g. color), and opinions (e.g. ideas and thoughts). For each if these types they have defined a set of tags that are used in annotation process.

- Lexicalized HMMs: Given a review of a product as an input of the system, the goal of the lexicalized HMM is to assign appropriate tag type to each part of the product review. For classification, they maximize conditional probability where T is the tags that should be assigned to different parts of a product review, is a set of all the words in the review and is the POS tag for each word. They have used MLE to learn parameters of the system.

- Information propagation: The goal of this part is to decrease the number of training data that this system requires. Suppose that we have a sentence like "Good picture quality" as part of a review in the training data. Word "good" is tagged as "<opinion_pos_exp>" in the training data. The system then creates new training data by looking at a dictionary and substitute word "good" with it's synonyms. For example the new sentence "great picture quality" can be added as a new training data. This idea is applied to all the words in the training data to increase the number of examples.

- Bootstrapping: The main contribution of this paper is the bootstrapping part. The idea is to partition the training set to two different disjoint sets and train a HMM using each of these sets. Then for each instance of the test data (which is non annotated by the human), if two HMMs classify the input review as the same class and if the confidence value is above a threshold T then this new instance is added to the training set. This idea can significantly decrease the amount of time that human should spend to annotate training data.

Evaluation Results

They have tested their system on reviews of different cameras that are chosen from Amazon.com. They have manually annotated reviews of 6 cameras to use as the training data. The system is tested using 4-fold validation. They have used the system that is developed by Turney,2002 as the baseline for comparisons. The results have shown that their system can increase accuracy of mining opinion expression by a factor of 2 comparing to the baseline system.

Retrieved from "http://curtis.ml.cmu.edu/w/courses/index.php?title=Jin_et_al,_2009&oldid=3100"