Data · dataset · 2022
XSum Hallucination Annotations
Listed in Hugging Face Datasets
Neural abstractive summarization models are highly prone to hallucinate content that is unfaithful to the input document.
Description
The popular metric such as ROUGE fails to show the severity of the problem. The dataset consists of faithfulness and factuality annotations of abstractive summaries for the XSum dataset.
We have crowdsourced 3 judgements for each of 500 x 5 document-system pairs. This will be a valuable resource to the abstractive summarization community.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/google-research-datasets/xsum_factuality ↗
landing page · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/google-research-datasets/xsum_factuality ↗
metadata API · from Hugging Face
Topics
- Stated by source
- summarization
- From keywords
- Computer Science & AI
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | google-research-datasets/xsum_factuality | 12 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:summarization | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |