Normative Documents Interactive Question Answering Dataset (NDIQAD)

Answers are exact pieces of document texts. The searched area of one question is always one whole document. Each question-answer pair is also accompanied with a path leading over headings of the document from the document's root to the section containing the answer.

This structural information enables testing of interactive question answering when the QA system is asking supplementary questions to limit the number of answers by disambiguating the document section. The dataset contains the original documents in PDF together with questions and answers in CSV files. The CSV files use semicolon [;] as the separator, and optionally double quotes ["] to escape strings.

The double quote is doubled when used inside a text [""]. Each row has four attributes, see an example of a dataset item below. Document: Study Rules of Mendel University in Brno Path: Study Rules / 2 Study in Bachelor’s and Master’s Degree Programs / 2.11 Study interruption Question: How can I interrupt my study?

Answer: Student’s study may be interrupted at the student’s request or ex officio. You can find all questions twice in the dataset. The first version is the original question as written by the annotator.

The second version marked "optimized" contains questions where the first person ("I", "me", "my") has been replaced by the actor of the document (a student or a client). This replacement is done automatically and can improve QA performance. This dataset is unique mainly due to the type of the documents.

Our focus is on normative documents with strict consistent formatting, numbered headings and paragraphs, low use of pronouns, etc. We do not focus on national laws as the majority of other research in this field, rather we focus on documents used to support internal processes in large organisations. We found that current QA methods do not perform well on this kind of documents.

Hence, we have introduced this dataset to address the issues and we have also presented a new approach to QA on normative documents, see sciencedirect.com/science/article/pii/S0957417421016158?dgcid=coauthor