Crowdsourcing annotations for visual object detection

Hao Su, Jia Deng, Li Fei-Fei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

94 Scopus citations


A large number of images with ground truth object bounding boxes are critical for learning object detectors, which is a fundamental task in compute vision. In this paper, we study strategies to crowd-source bounding box annotations. The core challenge of building such a system is to effectively control the data quality with minimal cost. Our key observation is that drawing a bounding box is significantly more difficult and time consuming than giving answers to multiple choice questions. Thus quality control through additional verification tasks is more cost effective than consensus based algorithms. In particular, we present a system that consists of three simple sub-tasks - a drawing task, a quality verification task and a coverage verification task. Experimental results demonstrate that our system is scalable, accurate, and cost-effective.

Original languageAmerican English
Title of host publicationHuman Computation - Papers from the 2012 AAAI Workshop, Technical Report
Number of pages7
StatePublished - 2012
Externally publishedYes
Event2012 AAAI Workshop - Toronto, ON, Canada
Duration: Jul 23 2012Jul 23 2012

Publication series

NameAAAI Workshop - Technical Report


Other2012 AAAI Workshop
CityToronto, ON

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'Crowdsourcing annotations for visual object detection'. Together they form a unique fingerprint.

Cite this