Difference between revisions of "MUC"
From Cohen Courses
Jump to navigationJump to search| (9 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | These [[Category::Dataset|datasets]] were created by MUC, the Message Understanding Conference. | |
| − | + | The conference was not as important as the work needed to get into the conference. It was novel in that participants had to complete in the task in order to gain admission. The performance measure used in the tasks is F1, the geometric mean of precision and recall. | |
| + | |||
| + | Table partially taken from [http://en.wikipedia.org/wiki/Message_Understanding_Conference] | ||
{|border=1 | {|border=1 | ||
| Line 8: | Line 10: | ||
|| ''Text Source'' | || ''Text Source'' | ||
|| ''Topic (Domain)'' | || ''Topic (Domain)'' | ||
| + | || ''Task'' | ||
|- | |- | ||
|| MUC-1 | || MUC-1 | ||
| Line 13: | Line 16: | ||
|| Mil. reports | || Mil. reports | ||
|| Fleet Operations | || Fleet Operations | ||
| + | || Open ended | ||
|- | |- | ||
|| MUC-2 | || MUC-2 | ||
| Line 18: | Line 22: | ||
|| Mil. reports | || Mil. reports | ||
|| Fleet Operations | || Fleet Operations | ||
| + | || Template extraction (1 template; 10 slots) | ||
|- | |- | ||
|| MUC-3 | || MUC-3 | ||
| Line 23: | Line 28: | ||
|| News reports | || News reports | ||
|| Terrorist activities in Latin America | || Terrorist activities in Latin America | ||
| + | || Template extraction (1 template; 18 slots) | ||
|- | |- | ||
|| MUC-4 | || MUC-4 | ||
|| 1992 | || 1992 | ||
|| News reports | || News reports | ||
| − | || Terrorist activities in Latin America | + | || Terrorist activities in Latin America<br>Additional details can be found [http://malt.ml.cmu.edu/mw/index.php/Learning_Domain-Speci%EF%AC%81c_Information_Extraction_Patterns_from_the_Web#Dataset here] |
| + | |||
| + | || Template extraction (1 template; 24 slots) | ||
|- | |- | ||
|| MUC-5 | || MUC-5 | ||
| Line 33: | Line 41: | ||
|| News reports | || News reports | ||
|| Corporate Joint Ventures, Microelectronic production | || Corporate Joint Ventures, Microelectronic production | ||
| + | || Template extraction (11 templates; 47 slots total) Some templates were nested. ''Note:'' Only multilingual MUC; also in Japanese. | ||
|- | |- | ||
|| [http://www.cs.nyu.edu/cs/faculty/grishman/muc6.html MUC-6] | || [http://www.cs.nyu.edu/cs/faculty/grishman/muc6.html MUC-6] | ||
| Line 38: | Line 47: | ||
|| News reports | || News reports | ||
|| Negotiation of Labor Disputes and Corporate Management Succession | || Negotiation of Labor Disputes and Corporate Management Succession | ||
| + | || Named entity extraction, Coreference, Template element (this was mainly person and organization), Scenario template | ||
|- | |- | ||
|| [http://www.itl.nist.gov/iaui/894.02/related_projects/muc/proceedings/muc_7_toc.html MUC-7] | || [http://www.itl.nist.gov/iaui/894.02/related_projects/muc/proceedings/muc_7_toc.html MUC-7] | ||
| Line 43: | Line 53: | ||
|| News reports | || News reports | ||
|| Airplane crashes, and Rocket/Missile Launches | || Airplane crashes, and Rocket/Missile Launches | ||
| + | || Named entity extraction, Coreference, Template elements, Facts or Template relations, Events | ||
|- | |- | ||
|} | |} | ||
| + | MUC-7: [https://docs.google.com/viewer?url=http%3A%2F%2Fwww.itl.nist.gov%2Fiaui%2F894.02%2Frelated_projects%2Fmuc%2Fproceedings%2Fmuc_7_proceedings%2Fmarsh_slides.pdf overview of results] | ||
==References== | ==References== | ||
| − | [http://www-nlpir.nist.gov/related_projects/muc/muc_data/muc_data_index.html] | + | [http://www-nlpir.nist.gov/related_projects/muc/muc_data/muc_data_index.html General information on MUC] |
| + | [http://acl.ldc.upenn.edu/C/C96/C96-1079.pdf A history of the MUC conference] | ||
Latest revision as of 12:44, 3 October 2011
These datasets were created by MUC, the Message Understanding Conference.
The conference was not as important as the work needed to get into the conference. It was novel in that participants had to complete in the task in order to gain admission. The performance measure used in the tasks is F1, the geometric mean of precision and recall.
Table partially taken from [1]
| Conference | Year | Text Source | Topic (Domain) | Task |
| MUC-1 | 1987 | Mil. reports | Fleet Operations | Open ended |
| MUC-2 | 1989 | Mil. reports | Fleet Operations | Template extraction (1 template; 10 slots) |
| MUC-3 | 1991 | News reports | Terrorist activities in Latin America | Template extraction (1 template; 18 slots) |
| MUC-4 | 1992 | News reports | Terrorist activities in Latin America Additional details can be found here |
Template extraction (1 template; 24 slots) |
| MUC-5 | 1993 | News reports | Corporate Joint Ventures, Microelectronic production | Template extraction (11 templates; 47 slots total) Some templates were nested. Note: Only multilingual MUC; also in Japanese. |
| MUC-6 | 1995 | News reports | Negotiation of Labor Disputes and Corporate Management Succession | Named entity extraction, Coreference, Template element (this was mainly person and organization), Scenario template |
| MUC-7 | 1997 | News reports | Airplane crashes, and Rocket/Missile Launches | Named entity extraction, Coreference, Template elements, Facts or Template relations, Events |
MUC-7: overview of results
