nlpllmeval/NLP-Course-LLM-Reasoning-Eval-May2025

Overview of LLM Reasoning Eval Dataset This dataset contains evaluation of multiple large language models (LLMs) over 918 MCQ reasoning questions created by 184 students. Each question was used to test 3 LLMs (each 3 times): GPT-4o, Claude Sonnet 3.x (3.5 or 3.7), and Deepseek R1. The questions target various reasoning areas (i.e., Math, Logic, Temporal, Commonsense) and are included only if 3 seperate attempts (in a new session) by ChatGPT (GPT-4o) fail at giving the correct… See the full description on the dataset page: huggingface.co/datasets/nlpllmeval/NLP-Course-LLM-Reasoning-Eval-May2025.