TY - GEN
T1 - MELODY
T2 - 37th IEEE International Conference on Distributed Computing Systems, ICDCS 2017
AU - Wang, Hongwei
AU - Guo, Song
AU - Cao, Jiannong
AU - Guo, Minyi
PY - 2017/7/13
Y1 - 2017/7/13
N2 - Crowdsourcing allows requesters to allocate tasks to a group of workers on the Internet to make use of their collective intelligence. Quality control is a key design objective in incentive mechanisms for crowdsourcing as requesters aim at obtaining answers of high quality under a given budget. However, when measuring workers' long-term quality, existing mechanisms either fail to utilize workers' historical information, or treat workers' quality as stable and ignore its temporal characteristics, hence performing poorly in a long run. In this paper we propose MELODY, a long-term dynamic quality-aware incentive mechanism for crowdsourcing. MELODY models interaction between requesters and workers as reverse auctions that run continuously. In each run of MELODY, we design a truthful, individual rational, budget feasible and quality-aware algorithm for task allocation with polynomial-time computation complexity and O(1) performance ratio. Moreover, taking into consideration the long-term characteristics of workers' quality, we propose a novel framework in MELODY for quality inference and parameters learning based on Linear Dynamical Systems at the end of each run, which takes full advantage of workers' historical information and predicts their quality accurately. Through extensive simulations, we demonstrate that MELODY outperforms existing work in terms of both quality estimation (reducing estimation error by 17.6% ~ 24.2%) and social performance (increasing requester's utility by 18.2% ~ 46.6%) in long-term scenarios.
AB - Crowdsourcing allows requesters to allocate tasks to a group of workers on the Internet to make use of their collective intelligence. Quality control is a key design objective in incentive mechanisms for crowdsourcing as requesters aim at obtaining answers of high quality under a given budget. However, when measuring workers' long-term quality, existing mechanisms either fail to utilize workers' historical information, or treat workers' quality as stable and ignore its temporal characteristics, hence performing poorly in a long run. In this paper we propose MELODY, a long-term dynamic quality-aware incentive mechanism for crowdsourcing. MELODY models interaction between requesters and workers as reverse auctions that run continuously. In each run of MELODY, we design a truthful, individual rational, budget feasible and quality-aware algorithm for task allocation with polynomial-time computation complexity and O(1) performance ratio. Moreover, taking into consideration the long-term characteristics of workers' quality, we propose a novel framework in MELODY for quality inference and parameters learning based on Linear Dynamical Systems at the end of each run, which takes full advantage of workers' historical information and predicts their quality accurately. Through extensive simulations, we demonstrate that MELODY outperforms existing work in terms of both quality estimation (reducing estimation error by 17.6% ~ 24.2%) and social performance (increasing requester's utility by 18.2% ~ 46.6%) in long-term scenarios.
UR - http://www.scopus.com/inward/record.url?scp=85027273973&partnerID=8YFLogxK
U2 - 10.1109/ICDCS.2017.28
DO - 10.1109/ICDCS.2017.28
M3 - Conference article published in proceeding or book
AN - SCOPUS:85027273973
T3 - Proceedings - International Conference on Distributed Computing Systems
SP - 933
EP - 943
BT - Proceedings - IEEE 37th International Conference on Distributed Computing Systems, ICDCS 2017
A2 - Lee, Kisung
A2 - Liu, Ling
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 5 June 2017 through 8 June 2017
ER -