A user behavior based cheat detection mechanism for crowdtesting

Ricky K P Mok, Weichao Li, Kow Chuen Chang

Research output: Journal article publicationConference articleAcademic researchpeer-review

Abstract

Crowdtesting is increasingly popular among researchers to carry out subjective assessments of different services. Experimenters can easily assess to a huge pool of human subjects through crowdsourcing platforms. The workers are usually anonymous, and they participate in the experiments independently. Therefore, a fundamental problem threatening the integrity of these platforms is to detect various types of cheating from the workers. In this poster, we propose cheat-detection mechanism based on an analysis of the workers' mouse cursor trajectories. It provides a jQuery-based library to record browser events. We compute a set of metrics from the cursor traces to identify cheaters. We deploy our mechanism to the survey pages for our video quality assessment tasks published on Amazon Mechanical Turk. Our results show that cheaters' cursor movement is usually more direct and contains less pauses.
Original languageEnglish
Pages (from-to)123-124
Number of pages2
JournalComputer Communication Review
Volume44
Issue number4
DOIs
Publication statusPublished - 1 Jan 2015
EventACM SIGCOMM 2014 Conference - Chicago, United States
Duration: 17 Aug 201422 Aug 2014

Keywords

  • Cheat-detection
  • Crowdsourcing
  • Cursor submovement

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this