Using photographic records to quantify accuracy of bird identifications in citizen science data

Abstract

Citizen science data are increasingly used for biodiversity monitoring. However, concerns are often raised over the accuracy of species identifications in citizen science databases, as data are collected mostly by non-professionals. Misidentifications can simultaneously generate two error types: false positives (erroneous reports of a species) and false negatives (lack of reports of the misidentified species). Large-scale assessments of identification errors should provide insights into the strengths and weaknesses of citizen science data. Here we show that citizen science photographic data for birds are trustworthy overall, although problems arise in hard-to-identify bird groups. We reviewed over 104 000 images of 377 passerine species from the southern Neotropics (Argentina) stored in eBird – a large citizen science platform – and quantified erroneous reports to calculate precision and recall metrics as measures for data accuracy. Precision increases with fewer false positives and recall increases with fewer false negatives; hence, high values of precision and recall will mirror a higher data accuracy. We found that 97% of the photos of all species were correctly identified. Most species (77%; n = 291) showed high accuracy in their identifications (precision and recall > 95%), with 122 species showing no errors. A few hard-to-identify species (10%; n = 40) showed low levels of data quality (63–90% precision or recall). Similarly, few species (12%; n = 46) exhibited intermediate precision or recall scores (90–95%). Further, we uncovered the existence of a complex network of cross-identifications composed of 272 species, with a predominance of tyrant flycatchers and ovenbirds, reflecting the strong traffic of errors that occurs within these families. To our knowledge, our study provides the first large-scale quantification of identification errors in photos submitted by citizen science contributors. We underscore the relevance of performing such assessments to understand how identification errors are distributed across a database before analysing data, and provide tools for citizen science stakeholders to direct more specific efforts toward species that need an improvement in data quality.