Wu and Weld WWW 2008

Citation

Wu, F. and Weld, D. 2008. Automatically Refining the Wikipedia Infobox Ontology. In Proceedings of the 17th Conference of the World Wide Web, pp. 635-644, ACM, New York.

Online version

University of Washington

Summary

This is a paper that introduces an autonomous system for refining Wikipedia’s infobox information schema to create a cleanly-structured ontology. Advanced query capability, improved information extractors and semiautomatic generation of new infobox templates are shown as advantages of a refined ontology. The ontology refinement problem is solved using both Support Vector Machines (SVM) and a more powerful joint-inference approach expressed in Markov Logic Networks (MLN).

The autonomous system, presented as Kylin Ontology Generator (KOG), is comprised of three modules: a schema cleaner, which merges duplicate classes and attributes and prunes rarely-used ones; a subsumption detector, which identifies is-a relations between infobox classes (e.g. "volleyball player" is-a "athlete"); and a schema mapper, which builds attribute mappings between related infobox classes.

Methods used

The subsumption detection task is modeled as a binary classification problem and several intuitive indicators are used as features to train the classifiers.

Classification features

• Similarity measure: the similarity between two infobox classes, measured using the TF/IDF scores between bags of words taken from their attribute set, the first sentence of each of their instances (articles) and their category tags.
• Class-name string inclusion: whether the name of a class is a substring of another one (e.g. "English public school" is-a "public school").
• Category tags: whether the name of a class is found in the infobox template category tag.
• Edit history: the edit pattern of an instance, because a Wikipedia author tends to specialize rather than generalize when changing the type of an article.
• Hearst patterns: the number of Google hits for match phrases of the form "Class1, like Class2" or "Class1 such as Class2" (e.g. "...scientists such as chemists...").
• WordNet mapping: a bunch of heuristics is used to compute a mapping between a WordNet node and an infobox class, and whether a corresponding node in WordNet is subsumed by the corresponding node of another class is also used as a feature for classification.

Joint-inference approach

Both the SVM classifier and the MLN model are trained using the features above, but the MLN classifier exploits additional joint-inference information. First, if "Class1 is-a Class2" and "Class2 is-a Class3", then it is likely that "Class1 is-a Class3". Also, the WordNet mapping and the is-a binary classification are treated as separate problems when actually the evidence from either one can help to reduce the uncertainty of the other. This additional knowledge is represented in the MLN model as additional logical implications with an attached measure of uncertainty:

• ${\text{is-a}}(c_{1},c_{2})\wedge {\text{is-a}}(c_{2},c_{3})\Rightarrow {\text{is-a}}(c_{1},c_{3})$

(the intuition that is-a is transitive),

• $map(c_{1})\wedge map(c_{2})\wedge {\text{is-a-WN}}(c_{1},c_{2})\Rightarrow {\text{is-a}}(c_{1},c_{2})$

(which means that if two infobox classes have correct WordNet mappings and their mapped nodes are is-a according to WordNet, then they should also be in a subsumption relation in the ontology).

Experimental results

A labeled dataset of 205 positive and 358 negative is-a pairs are used for training the classifiers. This dataset is constructed in part using DBpedia's manually-created mapping from 287,676 Wikipedia articles to their corresponding WordNet nodes. The performance of three different classifiers is tested with five-fold cross validation on the dataset: the SVM classifier, a MLN classifier using exactly the same features as the SVM one, and a fully-functional MLN classifier (called MLN+) using the additional formulas for crosstalk between WordNet mapping and is-a classification.

The SVM classifier achieves a precision of 97.2% and recall of 88.6%. Altough the MLN model drops precision to 96.8%, it has better recall at 92.1%. Finally, MLN+ wins on both measures, increasing precision to 98.8% and recall to 92.5%, showing the impact of joint inference in the task of subsumption detection, and therefore of ontology refinement.

Related papers

The autonomous system KOG is designed with the goal of situating semantic knowledge extracted from Wikipedia's natural language text (described in Wu and Weld CIKM 2007) in a clean and useful ontology. This refined ontology applied to Wikipedia's infobox schema can provide training data to bootstrap, or improve even further, open information extractors such as the ones described in Weld et al SIGMOD 2009 and Wu and Weld ACL 2010.