Imaging · dataset · 2024
Lin-Chen/MMStar
Listed in Hugging Face Datasets
Description
MMStar (Are We on the Right Way for Evaluating Large Vision-Language Models?) 🌐 Homepage | 🤗 Dataset | 🤗 Paper | 📖 arXiv | GitHub Dataset Details As shown in the figure below, existing benchmarks lack consideration of the vision dependency of evaluation samples and potential data leakage from LLMs' and LVLMs' training data. Therefore, we introduce MMStar: an elite vision-indispensible multi-modal benchmark, aiming to ensure each curated sample exhibits… See the full description on the dataset page: huggingface.co/datasets/Lin-Chen/MMStar.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/Lin-Chen/MMStar ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2403.20330 arxiv.org/abs/2403.20330 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/Lin-Chen/MMStar ↗
metadata API · from Hugging Face
Topics
- Stated by source
- image · multiple choice · question answering · text · visual question answering
- From keywords
- Computer Science & AI
- Inferred from text
- Computer vision and multimedia computation 71%
Provenance · 1 source records, 13 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | Lin-Chen/MMStar | 8 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].anzsrc:group:4603 | enrichment · Hugging Face | taxonomy-embedding@1.1.0 | title+keywords+description (71%) |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:image | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:text | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:multiple-choice | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:question-answering | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:visual-question-answering | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |