Estimating the accuracy of user surveys for assessing the impact of HPC systems

However, with a modest response rate and reliance on selfreporting by users, the accuracy of the survey is uncertain as is the degree of that uncertainty.

To quantify this uncertainty, CISL undertook a project that attempted to provide statistically supported limits on the accuracy and precision of the survey approach. We discovered limitations related to the range of users' HPC usage in our modeling phase, and several methods were attempted to adjust the model to fit the usage data. The resulting statistical models leverage data about the HPC usage associated with survey invitees to quantify the degree to which the survey undercounts the relevant publications.