Joint Inference in Information Extraction

Citation

Hoifung Poon & Pedro Domingos, "Joint Inference in Information Extraction", 27th National Conference on Artificial Intelligence (AAAI), 2007

Online version

Click here to download

Introduction

This paper aims at solving the problem of Citation Matching.
The approach makes use of Markov Logic Networks or MLN. The authors decided some rules for segmenting the citation into certain entities (title of the publication, authors and venue) and also identifying those entities, which were represented through an MLN. Three variants of solutions for inference were tried: an isolated inference for segmentation, a joint inference for segmentation, in which one inferred segmentation takes part in inferring another, and a joint inference of segmentation and recognition. The results were compared with existing baselines and were found to outdo baseline, although with a meager margin.

Dataset Used

The dataset used were Standard Citation Datasets Citeseer and Cora. These datasets contain a number of citations, with duplicate citations clustered together.

MLN for Joint Citation Matching

The main predicate used in the MLN is Token(t, i, c), which is true iff token t appears in the ith position of the cth citation. A token can be a word, date, number, etc. Punctuation marks are not treated as separate tokens; rather, the predicate HasPunc(c, i) is true iff a punctuation mark appears immediately after the ith position in the cth citation. Two "query" predicates are used (query predicates are the ones whose truth values are to be inferred):
InField(i, f, c), which is true iff i-th position of c-th citation is a field f, where f $\mathrm {E}$ {Title,Author,Venue}, and
SameCitation(c, c′) which is true iff citations c and c' represent the same publication, and inferring this predicate performs entity resolution.

Isolated Segmentation Model

The first model that the authors tried to solve the problem with, was to segment the citations without any kind of joint inference. Their segmentation model is essentially an HMM, where observation matrix and trnasition matrix are defined by certain logical formulas. For this model for identifying a segment, the observation matrix is defined by the logical formula:
Token(+t, i, c) ⇒ InField(i, +f, c)
The “+t, +f” notation signifies that the MLN contains an instance of this rule for each (token, field) pair. If this rule was learned in isolation, the weight of the (t,f)th instance would be log(p${_{t}f}$ /(1−p${_{t}f}$ )), where p${_{t}f}$ is the corresponding entry in the HMM observation matrix. The transition matrix, on the other hand, is defined by the below logical formula:
InField(i, +f, c) ⇒ InField(i + 1, +f',c')
The inclusion of token boundary in the above formulas for finding the token in a filed is as below:
InField(i, +f, c) ∧ ¬HasPunc(c, i) ⇒ InField(i + 1, +f, c)
In addition to the above rules, the following rules were also used: the first two positions of a citation are usually in the author field, and the middle one in the title; initials (e.g., “J.”) tend to appear in either the author or the venue field; positions preceding the last non-venue initial are usually not part of the title or venue; and positions after the first venue keyword (e.g., “Proceedings”, “Journal”) are usually not part of the author or title.

Entity Resolution Model

The Entity Resolution/Recognition model contains rules of the form: if two fields contain many common tokens, they are the same; if the fields of two citations match, the citations also match, and vice-versa; etc. Simply taking the output InField() predicates of the segmentation MLN as evidence to this MLN would constitute a standard pipeline model. Merging the two MLNs produces a joint model for segmentation and entity resolution.
However, the problem with this pipeline is that entity resolution often affects segmentation in a joint model. Since only a small fraction of citation pairs (c, c′) match, in the absence of strong evidence to the contrary the MLN will conclude that SameCitation(c, c′) is false. If SameCitation(c, c′) is the consequent of a rule (or rule chain) with InField() in the antecedent, the MLN may infer that InField() is false, even if segmentation alone would correctly predict it to be true.
Therefore, the authors defined additional rules that would not simply take InField as an antecedent rule.
A rule SimilarTitle(c, i, j, c′, i′, j′) was defined, which is true if citations c and c′ contain similar title like strings at positions i to j and i′ to j′, respectively. A string is title-like if it does not contain punctuation and does not match the “title exclusion” as defined above in isolated segmentation model. The authors held that if two citations have similar titles, yet different venues, they still represent the same citation. Hence, the rule:
SimilarTitle(c, i, j, c′, i′, j′) ∧ SimilarVenue(c, c′) ⇒ SameCitation(c, c′)

Joint Segmentation Model

This model aims at joint segmentation, where citations are segmented collectively, as opposed to in isolation. A predicate
JointInferenceCandidate(c, i, c′) was defined to be true if the trigram starting at position i in citation c also appears somewhere in citation c′, the trigrams do not match the “title exclusion” rules, and the trigram in c is not preceded by punctuation, while in c′ it is. This is used to extend the segmentation model by adding the following condition:
InField(i, +f, c) ∧ ¬HasPunc(c, i)∧(¬∃c′JointInferenceCandidate(c, i, c′)) ⇒ InField(i + 1, +f, c)
This rule says that a field inferred for position i is extended to the next position if that field is not a punctuation and does not satisfy a "title" (or JointInferenceCandidate) rule.
However, if one uses this rule to do a segmentation for the pair of citations given below, the titles won't match, since the title token started with "On" will not be able to match with title of the second citation, when both citations are essentially the same.
R. Schapire. On the strength of weak learnability. Proceedings of the 30th I.E.E.E. Symposium on the Foundations of Computer Science, 1989, pp. 28-33.
Robert E. Schapire. 5(2) The strength of weak learnability. Machine Learning, 1990 197-227
To take care of such cases, another rule was included, which could take advantage of the joint inference,ie, extend the segment to the next position if there's no matching citation for the given citation:
InField(i, +f, c) ∧ ¬HasPunc(c, i) ∧(¬∃c′ JointInferenceCandidate(c, i, c′)∧SameCitation(c, c′)) ⇒ InField(i + 1, +f, c)

Experiments and Results

For CiteSeer data, four-fold cross-validation was done, while for Cora data, three-fold cross-validation was done. The results in F1 for identification of individual and all entities (title,author,venue) taken together for CiteSeer and Cora datasets are as given in Table-1 and Table-2 below, respectively. "All" means all-citations, "Non-trivial" means citations that had at least one duplicate, "Potential" means citations with poor author-title boundary (e.g. with a punctuation missing after the author's last name and title's first word).

Table-1

Table-2