Data · dataset · 2025
RoleEval
Listed in ScienceDB
RoleEval is a bilingual benchmark designed to assess the memorization, utilization, and reasoning capabilities of role knowledge for large language models.
Description
RoleEval comprises RoleEval-Global (including internationally recognized characters) and RoleEval-Chinese (including characters popular in China), with 6,000 Chinese-English parallel multiple-choice questions focusing on 300 influential people and fictional characters drawn from a variety of domains including celebrities, anime, comics, movies, TV series, games, and fiction.
These questions cover basic knowledge and multi-hop reasoning abilities, aiming to systematically probe various aspects such as personal information, relationships, abilities, and experiences of the characters. To maintain high standards, we perform a hybrid quality check process combining automatic and human verification, ensuring that the questions are diverse, challenging, and discriminative.
Links
Where it is published
- DOI doi.org/10.57760/sciencedb.28967 ↗
DOI / persistent id · from scidb cn
Catalogue records · 1
- OAI-PMH record scidb.cn/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=10.57760%2… ↗
metadata API · from scidb cn
Topics
- From keywords
- Computer Science & AI · Earth & Environmental Science · Engineering · Humanities · Life Sciences · Social Science
Provenance · 1 source records, 11 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| ScienceDB | 10.57760/sciencedb.28967 | 10 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:computer-science-ai | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:engineering | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:humanities | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:social-science | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| description | source · scidb cn | connector:scidb_cn@1.0.0 | /metadata/dc/description |
| license | source · scidb cn | connector:scidb_cn@1.0.0 | /metadata/dc/rights |
| publication_date | source · scidb cn | connector:scidb_cn@1.0.0 | |
| title | source · scidb cn | connector:scidb_cn@1.0.0 | /metadata/dc/title |