MTurk Accessibility Evaluation

Image of the logo of Amazon Mechanical Turk.

Introduction

Many people with disabilities work from home because of commuting challenges. Due to the increase of new technologies, working from home is even easier for them because they just need a computer and an Internet connection. Amazon Mechanical Turk (MTurk) website allows users to get small quantities of money when they complete tasks. Thus, this tool could be a good way for people with disabilities to obtain financial support. In order to understand and find the barriers that people with disabilities face, the MTurk website was evaluated from the point of view of Web Content Accessibility Guidelines 2.0 guidelines and workers with disabilities.

Evaluation

The evaluation followed the Website Accessibility Conformance Evaluation Methodology 1.0 provided by the W3C. Then, the steps followed were: specify the target and objective of the evaluation, establish the evaluation environment and participants, select the sampling websites, audit the selected websites and obtain the results of the evaluation. In this case, five MTurk administrative tasks and eleven Human Intelligence Tasks (HITs) of the most common requesters were evaluated in a desktop computer using Firefox and Chrome and Windows operating system through the AA priority level of the WCAG 2.0 guidelines.

Problems

After carrying out the evaluation, it was found that people with disabilities could experience problems when they desire to use MTurk to work at home. A complete list of the barriers faced by people when they use MTurk is detailed in the Output section.

Duration

Three months (2013)

Team

Rocío Calvo
Shaun K. Kane
Amy Hurst

Applied Skills and Methods

Heuristic Evaluation
Semi-automatic Tools
Automatic Tools
Screen Readers
Flesch Reading Ease and Flesch-Kincaid

Outputs

Conference Article: "Evaluating the Accessibility of Crowdsourcing Tasks on Amazon’s Mechanical Turk"