Abg-CoQA: Clarifying Ambiguity in Conversational Question Answering
doi:10.24432/C5F30Z
TL;DR
We introduce Abg-CoQA, a novel dataset for clarifying ambiguity in Conversational Question Answering systems. Effective communication is about the dissemination of properly worded meaningful ideas/messages that are comprehensible to both sender and receiver and which ultimately can attract the desired response or feedback. For machines to engage in a conversation, it is therefore essential to enable them to clarify ambiguity and achieve a common ground. We introduce Abg-CoQA, a novel dataset for clarifying ambiguity in Conversational Question Answering systems. Our dataset contains 9k questions with answers where 1k questions are ambiguous, obtained from 4k text passages from five diverse domains. For ambiguous questions, a clarification conversational turn is collected. We evaluate strong language generation models and conversational question answering models on Abg-CoQA. The best-performing system achieves a BLEU-1 score of 12.9% on generating clarification question, which is 27.9 points behind human performance (40.8%); and a F1 score of 40.1% on question answering after clarification, which is 35.1 points behind human performance (75.2%), indicating there is ample room for improvement.
Citation
@inproceedings{
guo2021abgcoqa,
title={Abg-Co{QA}: Clarifying Ambiguity in Conversational Question Answering},
author={Meiqi Guo and Mingda Zhang and Siva Reddy and Malihe Alikhani},
booktitle={3rd Conference on Automated Knowledge Base Construction},
year={2021},
url={https://openreview.net/forum?id=SlDZ1o8FsJU},
doi={10.24432/C5F30Z}
}