Bias in expert product reviews

This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we know the reviewer’s conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review.

The review resembles other non-blind tests of product quality. We find evidence of a sizable bias in the reviewers’ ratings. Our findings suggest that reviewers’ ad hoc relationships with producers, often dismissed as ‘coming with the job’, can be very harmful.