Creating tasks that work well on Amazon Mechanical Turk

Amazon Mechanical Turk (Mechanical Turk) can be used for an exceptionally wide range of tasks. Tasks that work well on Mechanical Turk generally meet the following criteria:

Most tasks that meet these criteria can be completed on Mechanical Turk, assuming you provide workers with a task interface that allows them to successfully perform the task. You should also keep in mind that Mechanical Turk workers excel at tasks that rely on general human knowledge and skills. While some workers have specialized experience such as legal or medical backgrounds, most do not. As a result, while Mechanical Turk can enable tasks such as labeling the location of people or animals in images, you are likely to have less success asking workers to apply expertise that would be associated with a radiologist.

Note that tasks must also conform to the rules in the Mechanical Turk Acceptable Use Policy. Prohibited uses cover a range of activities such as violating the privacy or security of workers or others, abusive behavior, or any illegal activities.

Tasks can be completed within a web browser

Mechanical Turk tasks are built using HTML and presented to workers via the Mechanical Turk website. Most workers complete tasks on their computer without the need to use other devices or specialized software. Tasks that require workers to visit physical locations or leverage other devices aren't recommended.

Work can be broken into distinct, bite-sized tasks

Most Mechanical Turk tasks take less than five minutes to complete and almost all can be completed within an hour. This lets workers try new tasks without needing to commit a lot of time. Most workers appreciate the flexibility that Mechanical Turk provides in moving from task to-task without being locked in for an extended period of time.

Task supports clear instructions and outcomes

The most successful tasks on Mechanical Turk are those that provide the necessary information for a worker to imagine what a successful response would look like. Avoid tasks that are open-ended and could have multiple possible outcomes. For example, a task that asks workers to identify all of the competitors of company X would be frustrating for workers. By specifying that you want all competitors, workers are left wondering at what point they should draw a line and stop their research. It would also leave them wondering if you will reject their work if they aren't as comprehensive as you want them to be. In this example, you should instead be specific about the data that you need by describing your task as identify the top 5 competitors of company X.

Examples of common uses of Mechanical Turk

The following are examples of common Mechanical Turk use-cases: