Blind Users Accessing Their Training Images in Teachable Object Recognizers

Jonggi Hong, Jaina Gandhi, Ernest Essuah Mensah, Farnaz Zamiri Zeraati, Ebrima Jarjue, Kyungjun Lee, Hernisa Kacorri

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Teachable object recognizers provide a solution for a very practical need for blind people-instance level object recognition. They assume one can visually inspect the photos they provide for training, a critical and inaccessible step for those who are blind. In this work, we engineer data descriptors that address this challenge. They indicate in real time whether the object in the photo is cropped or too small, a hand is included, the photos is blurred, and how much photos vary from each other. Our descriptors are built into open source testbed iOS app, called MYCam. In a remote user study in (N = 12) blind participants' homes, we show how descriptors, even when error-prone, support experimentation and have a positive impact in the quality of training set that can translate to model performance though this gain is not uniform. Participants found the app simple to use indicating that they could effectively train it and that the descriptors were useful. However, many found the training being tedious, opening discussions around the need for balance between information, time, and cognitive load.

Original languageEnglish
Title of host publicationASSETS 2022 - Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450392587
DOIs
StatePublished - Oct 22 2022
Event24th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2022 - Athens, Greece
Duration: Oct 23 2022Oct 26 2022

Publication series

NameASSETS 2022 - Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility

Conference

Conference24th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2022
Country/TerritoryGreece
CityAthens
Period10/23/2210/26/22

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications
  • Hardware and Architecture
  • Human-Computer Interaction
  • Software

Keywords

  • blind
  • machine teaching
  • object recognition
  • participatory machine learning
  • visual impairment

Cite this