Data · dataset · 2024
Prompts generated from ChatGPT3.5, ChatGPT4, LLama3-8B, and Mistral-7B with NYT and HC3 topics in different roles and parameters configurations
Listed in e-cienciaDatos
Description Prompts generated from ChatGPT3.5, ChatGPT4, Llama3-8B, and Mistral-7B with NYT and HC3 topics in different roles and parameter configurations.
Description
The dataset is useful to study lexical aspects of LLMs with different parameters/roles configurations. The 0_Base_Topics.xlsx file lists the topics used for the dataset generation The rest of the files collect the answers of ChatGPT to these topics with different configurations of parameters/context: Temperature (parameter): Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
Frequency penalty (parameter): Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model s likelihood to repeat the same line verbatim. Top probability (parameter): An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass.
Read the rest (6 more)
Presence penalty (parameter): Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model s likelihood to talk about new topics. Roles (context) Default: No role is assigned to the LLM, the default role is used.
Child: The LLM is requested to answer as a five-year-old child. Young adult male: The LLM is requested to answer as a young male adult. Young adult female: The LLM is requested to answer as a young female adult.
Elderly adult male: The LLM is requested to answer as an elderly male adult. Elderly adult female: The LLM is requested to answer as an elderly female adult. Affluent adult male: The LLM is requested to answer as an affluent male adult.
Affluent adult female: The LLM is requested to answer as an affluent female adult. Lower-class adult male: The LLM is requested to answer as a lower-class male adult. Lower-class adult female: The LLM is requested to answer as a lower-class female adult.
Erudite: The LLM is requested to answer as an erudite who uses a rich vocabulary. Paper Paper: Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study Cite: @article{10.1145/3696459,author = {Mart\ {\i}nez, Gonzalo and Hern\ {a}ndez, Jos\ {e} Alberto and Conde, Javier and Reviriego, Pedro and Merino-G\ {o}mez, Elena},title = {Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study},year = {2024},publisher = {Association for Computing Machinery},address = {New York, NY, USA},issn = {2157-6904},url = {doi.org/10.1145/3696459},doi = {10.1145/3696459},abstract = ,note = {Just Accepted},journal = {ACM Trans.
Intell. Syst. Technol.},month = sep,keywords = {LLM, Lexical diversity, ChatGPT, Evaluation}}
Links
Where it is published
- Dataverse dataset page edatos.consorciomadrono.es/dataset.xhtml?persistentId=doi%3A10.5281%2FZENODO.10646081 ↗
landing page · from edatos consorciomadrono es
- DOI doi.org/10.5281/zenodo.10646081 ↗
DOI / persistent id · from edatos consorciomadrono es
Catalogue records · 1
- Dataverse API edatos.consorciomadrono.es/api/datasets/:persistentId/?persistentId=doi%3A10.5281%2FZENOD… ↗
metadata API · from edatos consorciomadrono es
Topics
- From keywords
- Computer Science & AI · Mathematics & Statistics
- Inferred from text
- Artificial intelligence 70% · Text 75%
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| e-cienciaDatos | doi:10.5281/ZENODO.10646081 | 9 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| concepts[field].anzsrc:group:4602 | enrichment · edatos consorciomadrono es | taxonomy-embedding@1.1.0 | title+keywords+description (70%) |
| concepts[field].local:field:computer-science-ai | mapping · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 | /subjects |
| concepts[field].local:field:mathematics-statistics | mapping · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 | /subjects |
| concepts[modality].local:modality:text | enrichment · edatos consorciomadrono es | keyword-concept-rules@1.0.0 | title+description (75%) |
| created_date | source · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 | |
| description | source · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 | /description |
| publication_date | source · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 | |
| title | source · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 | /name |
| updated_date | source · edatos consorciomadrono es | connector:edatos_consorciomadrono_es@1.0.0 |