A user behavior based cheat detection mechanism for crowdtesting

Ricky K P Mok, Weichao Li, Kow Chuen Chang

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

1 Citation (Scopus)

Abstract

Crowdtesting is increasingly popular among researchers to carry out subjective assessments of different services. Experimenters can easily assess to a huge pool of human subjects through crowdsourcing platforms. The workers are usually anonymous, and they participate in the experiments independently. Therefore, a fundamental problem threatening the integrity of these platforms is to detect various types of cheating from the workers. In this poster, we propose cheat-detection mechanism based on an analysis of the workers' mouse cursor trajectories. It provides a jQuery-based library to record browser events. We compute a set of metrics from the cursor traces to identify cheaters. We deploy our mechanism to the survey pages for our video quality assessment tasks published on Amazon Mechanical Turk. Our results show that cheaters' cursor movement is usually more direct and contains less pauses.
Original languageEnglish
Title of host publicationSIGCOMM 2014 - Proceedings of the 2014 ACM Conference on Special Interest Group on Data Communication
PublisherAssociation for Computing Machinery
Pages123-124
Number of pages2
ISBN (Print)9781450328364
DOIs
Publication statusPublished - 1 Jan 2014
Event2014 ACM Conference on Special Interest Group on Data Communication, SIGCOMM 2014 - Chicago, IL, United States
Duration: 17 Aug 201422 Aug 2014

Conference

Conference2014 ACM Conference on Special Interest Group on Data Communication, SIGCOMM 2014
Country/TerritoryUnited States
CityChicago, IL
Period17/08/1422/08/14

Keywords

  • cheat-detection
  • crowdsourcing
  • cursor submovement

ASJC Scopus subject areas

  • Computer Science Applications

Cite this