ANERcorp

ANERcorp is a manually annotated corpus in Arabic which is created to be used in Arabic NER tasks. It consists of two parts; training and testing. It has been annotated by one person in order to guarantee the coherence of the annotation.

There are more than 150K tokens in the corpus and 11% of them are Named Entities. Every token in the corpus is annotated with one of the followings; person, location, organization, miscellaneous or other. The distribution of the Named Entities are given below:

• Person : 39%
• Location : 30.4%
• Organization : 20.6%
• Miscellaneous : 10%

The corpus consists of 316 articles which have been selected from news wire and other type web sources. Before tagging the corpus, a p reprocessing had been applied to the data. In Arabic one word may written in several forms. In order to normalize these differences and decrease the data sparseness, the data had been normalized by reducing different forms of words are reduced into one form.

The corpus is publicly available and can be downloaded from [download URL]. It is in standard CONLL format.

The corpus has been used in several papers such as Benajiba et al, CICLing 2007, Benajiba and Rosso, IICAI 2007, Benajiba and Rosso, LREC 2008.

ANERgazet consists of three different gazetteers, all built manually using web resources: (i) Location Gazetteer : this gazetteer consists of 1,950 names of continents, countries, cities, rivers and mountains found in the Arabic version of wikipedia16; (ii) Person Gazetteer : this was originally a list of 1,920 complete names of people found in wikipedia and other websites. Splitting the names into first names and last names and omitting the repeated names, the list contains finally 2,309 names; (iii) Organizations Gazetteer : the last gazetteer consists of a list of 262 names of companies, football teams and other organizations.