Difference between revisions of "Wikipedia Infobox Generator Using Cross Lingual Unstructured Text"
| Line 20: | Line 20: | ||
* Topic modelling to detect type of page (Eg, country, person, athlete, actor) | * Topic modelling to detect type of page (Eg, country, person, athlete, actor) | ||
* Training CRF models for fact identification and extraction | * Training CRF models for fact identification and extraction | ||
| − | * Clustering for aligning multiple language | + | * Clustering for aligning multiple language entity names based on page topic |
Revision as of 14:41, 8 September 2011
Team Members
Project Idea
Key facts about Wikipedia articles are often mentioned in short block sections called 'InfoBoxes'. For example, an article on Brazil would mention "Capital:Brasília", "Largest city:São Paulo", "Official language : Portuguese", etc. As these infoboxes are manually created & maintained, several articles have either missing or outdated information (that only got revised in the plain text). We also note that articles in languages closer to the native speakers have much more detailed information. For eg, articles on latin Soccer players have better facts like "Debut match, number of goals, Records wins" mentioned in Español version of Wikipedia whereas the English version lacks these crucial stats.
In our SPLODD project, we propose to achieve two objectives:
- Extract facts for unstructured wikipedia text to generate infobox
- Combine facts in multiple languages for an article to generate infoboxes with comprehensive information.
Methodology used:
- Topic modelling to detect type of page (Eg, country, person, athlete, actor)
- Training CRF models for fact identification and extraction
- Clustering for aligning multiple language entity names based on page topic
Challenges:
- Large scale data
- Noise : Same infobox entity name can be written in different ways in the same language. For eg, "Number of Goals","No. of Goals","Goals", "# Goals", etc.
Corpus
- Wikipedia XML Dumps (Current Revision only)
- http://en.wikipedia.org/wiki/Wikipedia_database#Other_languages
- English corpus size - 31 GB Uncompressed
- With 5 languages, approximately 200 GB total
Reference Papers
- Wu, Weld, Autonomously Semantifying Wikipedia, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management CIKM 2007
- Eytan Adar , Michael Skinner , Daniel S. Weld, Information arbitrage across multi-lingual Wikipedia, Proceedings of the Second ACM International Conference on Web Search and Data Mining, February 09-12, 2009, Barcelona, Spain
