MeLoDy: A long-term dynamic quality-aware incentive mechanism for crowdsourcing

Hongwei Wang, Song Guo, Jiannong Cao, Minyi Guo

Research output: Journal article publicationJournal articleAcademic researchpeer-review

38 Citations (Scopus)

Abstract

Crowdsourcing allows requesters to allocate tasks to a group of workers on the Internet to make use of their collective intelligence. Quality control is a key design objective in incentive mechanisms for crowdsourcing as requesters aim at obtaining high-quality answers under a limited budget. However, when measuring workers' long-term quality, existing mechanisms either fail to utilize workers' historical information, or treat workers' quality as stable and ignore its temporal characteristics, hence performing poorly in a long run. In this paper we propose MeLoDy, a long-term dynamic quality-aware incentive mechanism for crowdsourcing. MeLoDy models interaction between requesters and workers as reverse auctions that run continuously. In each run of MeLoDy, we design a truthful, individual rational, budget feasible and quality-aware algorithm for task allocation with polynomial-time computation complexity and O(1) performance ratio. Moreover, taking into consideration the long-term characteristics of workers' quality, we propose a novel framework in MeLoDy for quality inference and parameters learning based on Linear Dynamical Systems at the end of each run, which takes full advantage of workers' historical information and predicts their quality accurately. Through extensive simulations, we demonstrate that MeLoDy outperforms existing work in terms of both quality estimation (reducing estimation error by 17.6 ∼ 24.2) and social performance (increasing requester's utility by 18.2 ∼ 46.6) in long-term scenarios.

Original languageEnglish
Pages (from-to)901-914
Number of pages14
JournalIEEE Transactions on Parallel and Distributed Systems
Volume29
Issue number4
DOIs
Publication statusPublished - 1 Apr 2018

Keywords

  • approximation algorithm
  • Crowdsourcing
  • incentive mechanism
  • inference and learning
  • quality control

ASJC Scopus subject areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Cite this