<p>Inter-rater reliability for expert evaluations.</p>

This study examined whether a combined text-and-image AI-assisted workflow differed from human-only brainstorming in expert-rated ideation outputs, preliminary design refinement, and subjective task experience. A counterbalanced within-subjects experiment was conducted with 32 undergraduate design students, each of whom completed both conditions using the same design brief.

Ideation outputs were evaluated by three expert raters across six task-specific analytic criteria, participant-matched concept sheets were assessed across five design attributes, and participants completed a task-specific post-task questionnaire after each condition. After Holm–Bonferroni correction within each outcome family, AI-assisted outputs received higher ratings for problem framing, mechanism, creator perspective, and evaluator perspective, whereas human-only outputs received higher ratings for innovation and proposed solution strategies.