Improving Human Annotation Effectiveness for Fact Collection by Identifying the Most Relevant Answers

This paper was accepted at the Workshops on Data Science with Human in the Loop at EMNLP 2022
Identifying and integrating missing facts is a crucial task for knowledge graph completion to ensure robustness towards downstream applications such as question answering. Adding new facts to a knowledge graph in real world system often involves human verification effort, where candidate facts are verified for accuracy by human annotators. This process is labor-intensive, time-consuming, and inefficient since only a small number of missing facts can be identified. This paper proposes a simple but…