Evaluation of coreference resolution tools for Polish from the information extraction perspective

• Kaczmarek, Adam
• Marcińczuk, Michał

In this paper we discuss the performance of existing tools for coreference resolution for Polish from the perspective of information extraction tasks. We take into consideration the source of mentions, i.e., gold standard vs mentions recognized automatically. We evaluate three existing tools, i.e., IKAR, Ruler and Bartek on the KPWr corpus. We show that the widely used metrics for coreference evaluation (B, MUC, CEAF, BLANC) do not reflect the real performance when dealing with the task of semantic relations recognition between named entities. Thus, we propose a supplementary metric called PARENT, which measures the correctness of linking between referential mentions and named entities.

Research areas:

• information-extraction
• jednostki identyfikacyjne
• KPWr

Year:

2015

Type of Publication:

In Proceedings

Keywords:

Coreference resolution; information extraction; corpus; named entities recognition

Book title:

Proceedings of the Workshop The 5th Workshop on Balto-Slavic Natural Language Processing, BSNLP 2015

Month:

September