Difference between revisions of "Wu and Weld WWW 2008"
| (13 intermediate revisions by the same user not shown) | |||
| Line 9: | Line 9: | ||
== Summary == | == Summary == | ||
| − | This is a [[Category::paper]] that introduces an autonomous system for refining | + | This is a [[Category::paper]] that introduces an autonomous system for refining [[UsesDataset::Wikipedia]]’s |
infobox information schema to create a cleanly-structured ontology. Advanced query capability, improved information extractors and semiautomatic generation of new infobox templates are shown as advantages of a refined ontology. The [[AddressesProblem::ontology refinement]] problem is solved using both [[UsesMethod::Support Vector Machines]] (SVM) and a more powerful joint-inference approach expressed in [[UsesMethod::Markov Logic Networks]] (MLN). | infobox information schema to create a cleanly-structured ontology. Advanced query capability, improved information extractors and semiautomatic generation of new infobox templates are shown as advantages of a refined ontology. The [[AddressesProblem::ontology refinement]] problem is solved using both [[UsesMethod::Support Vector Machines]] (SVM) and a more powerful joint-inference approach expressed in [[UsesMethod::Markov Logic Networks]] (MLN). | ||
| − | The autonomous system, presented as Kylin Ontology Generator (KOG), is comprised of three modules: | + | The autonomous system, presented as Kylin Ontology Generator (KOG), is comprised of three modules: a schema cleaner, which merges duplicate classes and attributes and prunes rarely-used ones; a subsumption detector, which identifies '''[http://en.wikipedia.org/wiki/is-a is-a]''' relations between infobox classes (e.g. "volleyball player" is-a "athlete"); and a schema mapper, which builds attribute mappings between related infobox classes. |
| − | |||
| − | |||
| − | |||
| − | + | == Methods used == | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | Both the SVM classifier and the MLN model are trained using the features above, but | + | The subsumption detection task is modeled as a binary classification problem and several intuitive indicators are used as features to train two classifiers: one using SVM, and the other using MLN. Some of these features are similarity measures between infobox classes, based on the TF/IDF scores between bags of words taken from their attribute set and the first sentence of each of their instances (articles). Other features include category tags, class-name string inclusion, edit history and hearst patterns. Additionally, a bunch of heuristics is used to compute a mapping between an infobox class and a WordNet node, and whether a corresponding node in WordNet is subsumed by the corresponding node of another class is also used as a feature for classification. |
| + | |||
| + | === Joint-inference classification === | ||
| + | |||
| + | Both the SVM classifier and the MLN model are trained using the features above, but the MLN classifier exploits additional information. First, if "Class1 is-a Class2" and "Class2 is-a Class3", then it is likely that "Class1 is-a Class3". Also, the WordNet mapping and the is-a binary classification are treated as separate problems when actually the evidence from either one can help to reduce the uncertainty of the other. This knowledge is represented in the MLN model as additional logical implications with an attached measure of uncertainty: | ||
* <math>\text{is-a}(c_{1}, c_{2}) \wedge \text{is-a}(c_{2}, c_{3}) \Rightarrow \text{is-a}(c_{1}, c_{3})</math> | * <math>\text{is-a}(c_{1}, c_{2}) \wedge \text{is-a}(c_{2}, c_{3}) \Rightarrow \text{is-a}(c_{1}, c_{3})</math> | ||
| − | (the intuition that is-a is transitive) | + | (the intuition that is-a is transitive), |
* <math>map(c_{1}) \wedge map(c_{2}) \wedge \text{is-a-WN}(c_{1}, c_{2}) \Rightarrow \text{is-a}(c_{1}, c_{2})</math> | * <math>map(c_{1}) \wedge map(c_{2}) \wedge \text{is-a-WN}(c_{1}, c_{2}) \Rightarrow \text{is-a}(c_{1}, c_{2})</math> | ||
| − | (which means if two infobox classes have correct WordNet mappings and their mapped nodes are is-a according to WordNet, then they should also be in a subsumption relation in the ontology) | + | (which means that if two infobox classes have correct WordNet mappings and their mapped nodes are is-a according to WordNet, then they should also be in a subsumption relation in the ontology). |
== Experimental results == | == Experimental results == | ||
| − | A labeled dataset of 205 positive and 358 negative is-a pairs are used for training the classifiers. This dataset is constructed in part using | + | A labeled dataset of 205 positive and 358 negative is-a pairs are used for training the classifiers. This dataset is constructed in part using [[UsesDataset::DBpedia]]'s manually-created mapping from 287,676 Wikipedia articles to their corresponding WordNet nodes. The performance of three different classifiers is tested with five-fold cross validation on the dataset: the SVM classifier, a MLN classifier using only the exact same features as the SVM one, and a fully-functional MLN classifier (called MLN+) using the additional formulas for crosstalk between WordNet mapping and is-a classification. |
| − | The SVM classifier achieves a precision of 97.2% and recall of 88.6%. Altough the MLN model drops precision to 96.8%, it has better recall at 92.1%. Finally, MLN+ wins on both measures, increasing precision to 98.8% and recall to 92.5%, showing the | + | The SVM classifier achieves a precision of 97.2% and recall of 88.6%. Altough the MLN model drops precision to 96.8%, it has better recall at 92.1%. Finally, MLN+ wins on both measures, increasing precision to 98.8% and recall to 92.5%, showing the impact of joint inference in the task of subsumption detection, and therefore of ontology refinement. |
== Related papers == | == Related papers == | ||
| − | + | The autonomous system KOG is designed with the goal of situating semantic knowledge extracted from Wikipedia's natural language text (described in [[RelatedPaper::Wu and Weld CIKM 2007]]) in a clean and useful ontology. A follow-up paper [[RelatedPaper::Wu et al KDD 2008]] presents techniques | |
| + | for increasing recall while extracting information from Wikipedia's long tail of sparse classes, by applying the automatically-learned subsumption taxonomy. The refined ontology applied to Wikipedia's infobox schema can also provide training data to bootstrap open information extractors, such as the ones described in [[RelatedPaper::Weld et al SIGMOD 2009]] and [[RelatedPaper::Wu and Weld ACL 2010]]. | ||
Latest revision as of 01:04, 28 September 2011
Contents
Citation
Wu, F. and Weld, D. 2008. Automatically Refining the Wikipedia Infobox Ontology. In Proceedings of the 17th Conference of the World Wide Web, pp. 635-644, ACM, New York.
Online version
Summary
This is a paper that introduces an autonomous system for refining Wikipedia’s infobox information schema to create a cleanly-structured ontology. Advanced query capability, improved information extractors and semiautomatic generation of new infobox templates are shown as advantages of a refined ontology. The ontology refinement problem is solved using both Support Vector Machines (SVM) and a more powerful joint-inference approach expressed in Markov Logic Networks (MLN).
The autonomous system, presented as Kylin Ontology Generator (KOG), is comprised of three modules: a schema cleaner, which merges duplicate classes and attributes and prunes rarely-used ones; a subsumption detector, which identifies is-a relations between infobox classes (e.g. "volleyball player" is-a "athlete"); and a schema mapper, which builds attribute mappings between related infobox classes.
Methods used
The subsumption detection task is modeled as a binary classification problem and several intuitive indicators are used as features to train two classifiers: one using SVM, and the other using MLN. Some of these features are similarity measures between infobox classes, based on the TF/IDF scores between bags of words taken from their attribute set and the first sentence of each of their instances (articles). Other features include category tags, class-name string inclusion, edit history and hearst patterns. Additionally, a bunch of heuristics is used to compute a mapping between an infobox class and a WordNet node, and whether a corresponding node in WordNet is subsumed by the corresponding node of another class is also used as a feature for classification.
Joint-inference classification
Both the SVM classifier and the MLN model are trained using the features above, but the MLN classifier exploits additional information. First, if "Class1 is-a Class2" and "Class2 is-a Class3", then it is likely that "Class1 is-a Class3". Also, the WordNet mapping and the is-a binary classification are treated as separate problems when actually the evidence from either one can help to reduce the uncertainty of the other. This knowledge is represented in the MLN model as additional logical implications with an attached measure of uncertainty:
(the intuition that is-a is transitive),
(which means that if two infobox classes have correct WordNet mappings and their mapped nodes are is-a according to WordNet, then they should also be in a subsumption relation in the ontology).
Experimental results
A labeled dataset of 205 positive and 358 negative is-a pairs are used for training the classifiers. This dataset is constructed in part using DBpedia's manually-created mapping from 287,676 Wikipedia articles to their corresponding WordNet nodes. The performance of three different classifiers is tested with five-fold cross validation on the dataset: the SVM classifier, a MLN classifier using only the exact same features as the SVM one, and a fully-functional MLN classifier (called MLN+) using the additional formulas for crosstalk between WordNet mapping and is-a classification.
The SVM classifier achieves a precision of 97.2% and recall of 88.6%. Altough the MLN model drops precision to 96.8%, it has better recall at 92.1%. Finally, MLN+ wins on both measures, increasing precision to 98.8% and recall to 92.5%, showing the impact of joint inference in the task of subsumption detection, and therefore of ontology refinement.
Related papers
The autonomous system KOG is designed with the goal of situating semantic knowledge extracted from Wikipedia's natural language text (described in Wu and Weld CIKM 2007) in a clean and useful ontology. A follow-up paper Wu et al KDD 2008 presents techniques for increasing recall while extracting information from Wikipedia's long tail of sparse classes, by applying the automatically-learned subsumption taxonomy. The refined ontology applied to Wikipedia's infobox schema can also provide training data to bootstrap open information extractors, such as the ones described in Weld et al SIGMOD 2009 and Wu and Weld ACL 2010.
