Evidence Reasoning and Curriculum Learning for Document-level Relation Extraction

Tianyu Xu, Jianfeng Qu, Wen Hua, Zhixu Li, Jiajie Xu, An Liu, Lei Zhao, Xiaofang Zhou

Research output: Journal article publicationJournal articleAcademic researchpeer-review


Document-level Relation Extraction (RE) is a promising task aiming at identifying relations of multiple entity pairs in a document. Compared with the sentence-level counterpart, it has raised two significant challenges: a) In most cases, a relational fact can be adequately expressed via a small subset of sentences from the document, namely evidence. But the traditional method cannot model such strong semantic correlations between evidence sentences that collaborate to describe a specific relation; b) The data of this task is extremely long-tail in terms of too many NA instances and imbalanced relational types. Such data can mislead the tail prediction bias to the head categories in the RE model. In this paper, we present a novel Evidence reasoning and Curriculum learning method for DocRE (DRE-EC) to address these challenges. Particularly, we first formulate evidence extraction as a sequential decision problem through a crafted reinforcement learning mechanism with an efficient path searching strategy to reduce the action space. Providing the evidence for each entity pair as a customized-filtered document in advance helps infer the relations better. To address the long-tail issue, we further develop a hybrid curriculum learning method at the NA-level (NC) and relation-level (RC) with our customized difficulty measure score. In NC, the NA samples are scheduled in an easy-to-hard scheme and gradually added, resulting in the data distribution from ideal and balanced to real and unbalanced. In RC, the scheme is switched into hard-to-easy to enhance the hard and tail samples. In addition, we propose a new Equalization adaptive Focal Loss(EFLoss) that can adjust to the changing data distribution and focus more on the tail categories. We conduct various experiments on two document-level RE benchmarks and achieve a remarkable improvement over previous competitive baselines. Furthermore, we provide detailed analyses of the advantages and effectiveness of our method.

Original languageEnglish
Pages (from-to)1-14
Number of pages14
JournalIEEE Transactions on Knowledge and Data Engineering
Publication statusAccepted/In press - Jul 2023


  • Curriculum Learning
  • Data mining
  • Data models
  • Document-level Relation Extraction
  • Evidence extraction
  • Long-tail problem
  • Predictive models
  • Semantics
  • Tail
  • Task analysis
  • Training

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics


Dive into the research topics of 'Evidence Reasoning and Curriculum Learning for Document-level Relation Extraction'. Together they form a unique fingerprint.

Cite this