Gold Standard and Annotation Dataset for CO2 Emissions Annotation

Annotation of the pipeline output was done in 3 phases: first by non-experts (see columns prefixed with non_expert_ in annotation_dataset ), then by expert groups (columns prefixed with exp_group_ in annotation_dataset ) in case of disagreement of non-experts and finally in a discussion of all experts (columns prefixed with exp__disc in annotation_dataset ) in case of disagreement between expert groups. The annotation guidelines for the non-experts and experts are also included in this repository.

The annotation results from all three phases are combined to form the final benchmark dataset: gold_standard . Codebooks detailing each variable of each of the two datasets are also provided. More details about the annotation template or the data wrangling scripts can be found in the GitHub repository .

Merging of datasets Users can match the two datasets ( gold_standard and annotation_dataset ) using the variable combination of company_name , report_year and merge_id (index column). The merge_id already includes the company name and report year implicitly, but to avoid column duplication in the join operation, it should be included as join variables. For example this is useful when comparing LLM extractions to gold standard data.