ANERcorp
training and testing corpus annotated corpus manually annotated news wire and other web sources person location organization miscellaneous other
more than 150K tokens in which 11% of them are NE.
the corpus has been used in several papers the data is in standard CONLL format
In Arabic one word may written in several forms. In order to normalize these differences and decrease the data sparseness, the data has been normalized by reducing different forms of words are reduced into one form.
The corpus consists of 316 articles which have been selected from news wire and other type web sources.
- Person : 39%
- Location : 30.4%
- Organization : 20.6%
- Miscellaneous : 10%
ANERgazet consists of three different gazetteers, all built manually using web resources: (i) Location Gazetteer : this gazetteer consists of 1,950 names of continents, countries, cities, rivers and mountains found in the Arabic version of wikipedia16; (ii) Person Gazetteer : this was originally a list of 1,920 complete names of people found in wikipedia and other websites. Splitting the names into first names and last names and omitting the repeated names, the list contains finally 2,309 names; (iii) Organizations Gazetteer : the last gazetteer consists of a list of 262 names of companies, football teams and other organizations.
