Tweet Annotation Sensitivity Experiment 1

The stratification was done with respect to majority-voted class and level of disagreement. We then recruited 1000 Prolific workers to annotate each of the 20 tweets.

Annotators were randomly selected into one of six experimental conditions, as shown in the following figures. In these conditions, they were asked to assign the labels Hate Speech / Offensive Language / Neither. In addition, we collected a variety of demographic variables (e.g. age and gender) and some para data (e.g. duration of the whole task, duration per screen).