Data · dataset · 2022
GEM
Listed in Hugging Face Datasets
GEM is a benchmark environment for Natural Language Generation with a focus on its Evaluation, both through human annotations and automated Metrics.
Description
GEM aims to: - measure NLG progress across 13 datasets spanning many NLG tasks and languages. - provide an in-depth analysis of data and models presented via data statements and challenge sets. - develop standards for evaluation of generated text using both automated and human metrics.
It is our goal to regularly update GEM and to encourage toward more inclusive practices in dataset development by extending existing data or developing datasets for additional languages.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/GEM/gem ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2102.01672 arxiv.org/abs/2102.01672 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/GEM/gem ↗
metadata API · from Hugging Face
Topics
- Stated by source
- fill mask · summarization · table to text · tabular to text · text generation
- From keywords
- Computer Science & AI
- Inferred from text
- Artificial intelligence 69% · Text 75%
Provenance · 1 source records, 15 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | GEM/gem | 10 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].anzsrc:group:4602 | enrichment · Hugging Face | taxonomy-embedding@1.1.0 | title+keywords+description (69%) |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].local:modality:text | enrichment · Hugging Face | keyword-concept-rules@1.0.0 | title+description (75%) |
| concepts[task].hf_task:fill-mask | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:summarization | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:table-to-text | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:tabular-to-text | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license_text | source · Hugging Face | connector:huggingface@1.0.0 | |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |