Replication and Discussion Data: User study for Copilot for constraint-driven generation of architectural design evaluation and suggestions

The user study involved architects and BIM specialists performing different compliance checks with and without the copilot’s assistance to assess its performance, as well as participating in discussions regarding the interaction and workflow integration implemented for the copilot.

The raw data contains transcripts of the discussions held during the user study as text files, along with the data for each task conducted by the architects during the study. It includes: Ground truth for the compliance checks to be performed (.JSON) Evaluation results from the copilot (.JSON) Evaluation results from the architects (.csv) A Python script to compare the copilot’s and architects’ results to the ground truth The processed data contains a report of the user study by architectural company as text data.