Data · dataset · 2026
Evaluation Dataset for the Symbolic Constraint-Guided Reasoning (SCGR) Framework
Listed in ScienceDB
This dataset is designed to evaluate the effectiveness and accuracy of the Symbolic Constraint-Guided Reasoning (SCGR) framework in medical intervention recommendation tasks.
Description
The dataset generation is based on 100 anonymized clinical patient cases, covering core physiological metrics such as gender, age, and BMI. The processing involves three main stages: First, the DeepSeek-chat large language model was utilized with four distinct inference strategies (LLM-only, CoT, RAG, and SCGR) to generate medical intervention reports for the cases.
Second, the m3e-base model under the SentenceTransformer architecture was employed to calculate the semantic similarity between the generated reports and standardized interventions derived from evidence-based guidelines (e.g., Guideline for Hypertension Management in Chinese Patients with Chronic Kidney Disease (2023 Edition), Diagnosis and Treatment Guidelines for Obesity (2024 Edition), and relevant WHO guidelines); paired sample t-tests were then performed to analyze the statistical significance of performance across strategies.
Read the rest (3 more)
Finally, multi-dimensional quantitative assessments were conducted using models including GPT-3.5-Turbo, DeepSeek-V3, Qwen-Plus, GLM-4.5-Flash, and Hunyuan-Pro. The experimental environment is based on the Python 3.13 programming language, utilizing core tools such as Pandas for data processing, Scikit-learn for similarity measurement, and Scipy for statistical inference.The dataset covers a time span primarily from May to September 2025 during the experimental execution phase.
The data is organized in tabular format, encompassing model-generated reports, multi-dimensional scores for various systems across different models, semantic similarity results, t-test results, user evaluation data, and the final multi-model review results.The dataset structure is complete with no missing data. Regarding potential errors, since semantic similarity calculation relies on the pre-trained vector embedding model m3e-base, there may be model biases in capturing subtle nuances of specific medical terminology; however, these errors remain within statistically controlled ranges (e.g., p-values in the t-tests are significantly lower than 0.05, demonstrating high statistical significance).
Supporting data is stored within the supporting_data.zip archive.
Links
Where it is published
- DOI doi.org/10.57760/sciencedb.j00133.00673 ↗
DOI / persistent id · from scidb cn
Catalogue records · 1
- OAI-PMH record scidb.cn/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=10.57760%2… ↗
metadata API · from scidb cn
Topics
- From keywords
- Earth & Environmental Science · Engineering · Humanities · Life Sciences · Medicine & Health · Social Science
Provenance · 1 source records, 13 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| ScienceDB | 10.57760/sciencedb.j00133.00673 | 10 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[disease].local:disease:disease | enrichment · scidb cn | keyword-concept-rules@1.0.0 | title+description (75%) |
| concepts[field].local:field:earth-environmental | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:engineering | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:humanities | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:medicine-health | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[field].local:field:social-science | mapping · scidb cn | connector:scidb_cn@1.0.0 | |
| concepts[modality].local:modality:tabular | enrichment · scidb cn | keyword-concept-rules@1.0.0 | title+description (75%) |
| description | source · scidb cn | connector:scidb_cn@1.0.0 | /metadata/dc/description |
| license_text | source · scidb cn | connector:scidb_cn@1.0.0 | |
| publication_date | source · scidb cn | connector:scidb_cn@1.0.0 | |
| title | source · scidb cn | connector:scidb_cn@1.0.0 | /metadata/dc/title |