Incentivizing peer grading in MOOCS: An audit game approach

Alejandro Carbonara, Anupam Datta, Arunesh Sinha, Yair Zick

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In Massively Open Online Courses (MOOCs) TA resources are limited; most MOOCs use peer assessments to grade assignments. Students have to divide up their time between working on their own homework and grading others. If there is no risk of being caught and penalized, students have no reason to spend any time grading others. Course staff want to incentivize students to balance their time between course work and peer grading. They may do so by auditing students, ensuring that they perform grading correctly. One would not want students to invest too much time on peer grading, as this would result in poor course performance. We present the first model of strategic auditing in peer grading, modeling the student's choice of effort in response to a grader's audit levels as a Stackelberg game with multiple followers. We demonstrate that computing the equilibrium for this game is computationally hard. We then provide a PTAS in order to compute an approximate solution to the problem of allocating audit levels. However, we show that this allocation does not necessarily maximize social welfare; in fact, there exist settings where course auditor utility is arbitrarily far from optimal under an approximately optimal allocation. To circumvent this issue, we present a natural condition that guarantees that approximately optimal TA allocations guarantee approximately optimal welfare for the course auditors.

Original languageAmerican English
Title of host publicationIJCAI 2015 - Proceedings of the 24th International Joint Conference on Artificial Intelligence
EditorsMichael Wooldridge, Qiang Yang
PublisherInternational Joint Conferences on Artificial Intelligence
Pages497-503
Number of pages7
ISBN (Electronic)9781577357384
StatePublished - 2015
Externally publishedYes
Event24th International Joint Conference on Artificial Intelligence, IJCAI 2015 - Buenos Aires, Argentina
Duration: Jul 25 2015Jul 31 2015

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
Volume2015-January

Other

Other24th International Joint Conference on Artificial Intelligence, IJCAI 2015
Country/TerritoryArgentina
CityBuenos Aires
Period7/25/157/31/15

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Incentivizing peer grading in MOOCS: An audit game approach'. Together they form a unique fingerprint.

Cite this