Answering Science Exam Questions Using Query Reformulation with Background Knowledge
doi:10.24432/C5CC7R
TL;DR
We explore how using background knowledge with query reformulation can help retrieve better supporting evidence when answering multiple-choice science questions. Open-domain question answering (QA) is an important problem in AI and NLP that is emerging as a bellwether for progress on the generalizability of AI methods and techniques. Much of the progress in open-domain QA systems has been realized through advances in information retrieval methods and corpus construction. In this paper, we focus on the recently introduced ARC Challenge dataset, which contains 2,590 multiple choice questions authored for grade-school science exams. These questions are selected to be the most challenging for current QA systems, and current state of the art performance is only slightly better than random chance. We present a system that reformulates a given question into queries that are used to retrieve supporting text from a large corpus of science-related text. Our rewriter is able to incorporate background knowledge from ConceptNet and -- in tandem with a generic textual entailment system trained on SciTail that identifies support in the retrieved results -- outperforms several strong baselines on the end-to-end QA task despite only being trained to identify essential terms in the original source question. We use a generalizable decision methodology over the retrieved evidence and answer candidates to select the best answer. By combining query reformulation, background knowledge, and textual entailment our system is able to outperform several strong baselines on the ARC dataset.
Citation
@inproceedings{
musa2019answering,
title={Answering Science Exam Questions Using Query Reformulation with Background Knowledge},
author={Ryan Musa and Xiaoyan Wang and Achille Fokoue and Nicholas Mattei and Maria Chang and Pavan Kapanipathi and Bassem Makni and Kartik Talamadupula and Michael Witbrock},
booktitle={Automated Knowledge Base Construction (AKBC)},
year={2019},
url={https://openreview.net/forum?id=HJxYZ-5paX},
doi={10.24432/C5CC7R}
}
