Constarium
← Search

Data · dataset · 2026

Evaluation Dataset for the Symbolic Constraint-Guided Reasoning (SCGR) Framework

Listed in ScienceDB

This dataset is designed to evaluate the effectiveness and accuracy of the Symbolic Constraint-Guided Reasoning (SCGR) framework in medical intervention recommendation tasks.

Description

The dataset generation is based on 100 anonymized clinical patient cases, covering core physiological metrics such as gender, age, and BMI. The processing involves three main stages: First, the DeepSeek-chat large language model was utilized with four distinct inference strategies (LLM-only, CoT, RAG, and SCGR) to generate medical intervention reports for the cases.

Second, the m3e-base model under the SentenceTransformer architecture was employed to calculate the semantic similarity between the generated reports and standardized interventions derived from evidence-based guidelines (e.g., Guideline for Hypertension Management in Chinese Patients with Chronic Kidney Disease (2023 Edition), Diagnosis and Treatment Guidelines for Obesity (2024 Edition), and relevant WHO guidelines); paired sample t-tests were then performed to analyze the statistical significance of performance across strategies.

Read the rest (3 more)

Finally, multi-dimensional quantitative assessments were conducted using models including GPT-3.5-Turbo, DeepSeek-V3, Qwen-Plus, GLM-4.5-Flash, and Hunyuan-Pro. The experimental environment is based on the Python 3.13 programming language, utilizing core tools such as Pandas for data processing, Scikit-learn for similarity measurement, and Scipy for statistical inference.The dataset covers a time span primarily from May to September 2025 during the experimental execution phase.

The data is organized in tabular format, encompassing model-generated reports, multi-dimensional scores for various systems across different models, semantic similarity results, t-test results, user evaluation data, and the final multi-model review results.The dataset structure is complete with no missing data. Regarding potential errors, since semantic similarity calculation relies on the pre-trained vector embedding model m3e-base, there may be model biases in capturing subtle nuances of specific medical terminology; however, these errors remain within statistically controlled ranges (e.g., p-values in the t-tests are significantly lower than 0.05, demonstrating high statistical significance).

Supporting data is stored within the supporting_data.zip archive.

Links

Where it is published

Catalogue records · 1

Topics

Inferred from text
Disease 75% · Tabular 75%
Provenance · 1 source records, 13 field assertions
SourceKeyLast seenRaw
ScienceDB10.57760/sciencedb.j00133.0067310 d agoJSON v1
FieldAssertionExtractorEvidence
access_levelsource · scidb cnconnector:scidb_cn@1.0.0
concepts[disease].local:disease:diseaseenrichment · scidb cnkeyword-concept-rules@1.0.0title+description (75%)
concepts[field].local:field:earth-environmentalmapping · scidb cnconnector:scidb_cn@1.0.0
concepts[field].local:field:engineeringmapping · scidb cnconnector:scidb_cn@1.0.0
concepts[field].local:field:humanitiesmapping · scidb cnconnector:scidb_cn@1.0.0
concepts[field].local:field:life-sciencesmapping · scidb cnconnector:scidb_cn@1.0.0
concepts[field].local:field:medicine-healthmapping · scidb cnconnector:scidb_cn@1.0.0
concepts[field].local:field:social-sciencemapping · scidb cnconnector:scidb_cn@1.0.0
concepts[modality].local:modality:tabularenrichment · scidb cnkeyword-concept-rules@1.0.0title+description (75%)
descriptionsource · scidb cnconnector:scidb_cn@1.0.0/metadata/dc/description
license_textsource · scidb cnconnector:scidb_cn@1.0.0
publication_datesource · scidb cnconnector:scidb_cn@1.0.0
titlesource · scidb cnconnector:scidb_cn@1.0.0/metadata/dc/title