Imaging · dataset · 2022
HuggingFaceM4/LocalizedNarratives
Listed in Hugging Face Datasets
Localized Narratives, a new form of multimodal image annotations connecting vision and language.
Description
We ask annotators to describe an image with their voice while simultaneously hovering their mouse over the region they are describing. Since the voice and the mouse pointer are synchronized, we can localize every single word in the description.
This dense visual grounding takes the form of a mouse trace segment per word and is unique to our data. We annotated 849k images with Localized Narratives: the whole COCO, Flickr30k, and ADE20K datasets, and 671k images of Open Images, all of which we make publicly available.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/HuggingFaceM4/LocalizedNarratives ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:1912.03098 arxiv.org/abs/1912.03098 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/HuggingFaceM4/LocalizedNarratives ↗
metadata API · from Hugging Face
Topics
- Stated by source
- image
- From keywords
- Computer Science & AI
- Inferred from text
- Artificial intelligence 70% · Image 75%
Provenance · 1 source records, 11 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | HuggingFaceM4/LocalizedNarratives | 12 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].anzsrc:group:4602 | enrichment · Hugging Face | taxonomy-embedding@1.0.0 | title+keywords+description (70%) |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:image | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].local:modality:image | enrichment · Hugging Face | keyword-concept-rules@1.0.0 | title+description (75%) |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |