Notification Handling Using Amazon SQS

Your application can use the Amazon Simple Queue Service (Amazon SQS) to handle Mechanical Turk notifications. By using Amazon SQS, your notifications are guaranteed to be delivered at least once. For more information about guaranteed delivery of notifications, see Guaranteed Delivery. For more information about, see Amazon SQS.

Creating an SQS Queue

You must create an Amazon SQS queue before using the SQS transport type in notification-related calls. Mechanical Turk does not create an Amazon SQS queue for you. An SQS queue can be created through the Amazon SQS API or by using the AWS Console. For more information, see the Amazon SQS documentation.

Configuring an SQS Queue

Your Amazon SQS queue permissions must be configured to allow a Mechanical Turk system account to call the sqs:SendMessage action on your queue. Whether you use the management console UI or the API to configure permissions, consider the following:

Amazon SQS Policy Document Example

The following example policy document only creates the SendMessage permission for the Mechanical Turk account. You can add additional restrictions. For more information about policy documents, see the Amazon SQS Developer Guide.

{
  "Version": "2012-10-17",
  "Id": "arn:aws:sqs:region:aws-account-id:queue-name/MTurkOnlyPolicy",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "mturk-requester.amazonaws.com"
      },
      "Action": "SQS:SendMessage",
      "Resource": "arn:aws:sqs:region:aws-account-id:queue-name",
      "Condition": {
        "Bool": {
          "aws:SecureTransport":"true"
        }
      }
    }
  ]
}

Configuring Permissions Using the AWS Console

Configuring Permissions Using the Amazon SQS API

Call the Amazon SQS SetQueueAttributes action with the Attribute.Name parameter set to Policy. You can call SetQueueAttributes with a policy document similar to the example policy document. Do not use the Amazon SQS AddPermission action for configuring permissions on this queue. If you programmatically create a queue and apply a policy document to it, you must ensure the Resource value in the policy document is updated with the correct queue name.

Testing Your Queue

To test your permissions, call the Mechanical Turk SendTestEventNotification operation with a Transport of SQS and your queue URL as the Destination.

Guaranteed Delivery

Using Amazon SQS provides a guaranteed at-least-once delivery of each message. Mechanical Turk ensures that it calls SendMessage at least once for each message. SQS then provides guarantees regarding message persistence and message delivery.

Rarely, the same message may show up twice in the queue. This is an attribute of Amazon SQS's nature as a distributed system.

If you take action on your queue that prevents Mechanical Turk from publishing to it, we cannot guarantee delivery of the messages that would have been sent to your queue. For instance, such actions may include:

SQS Message Ordering

You should expect that messages may arrive out of order. For information about message ordering behavior, see the SQS documentation.

Multiple SQS Queues

You may use a different queue for each HITType that you configure with notifications.

Mechanical Turk does not provide the ability to route events within a HITType to different queues. For example, you might prefer to have AssignmentSubmitted events for a HITType delivered to a different queue than HITReviewable events for that same HITType. Mechanical Turk will publish both events to the same queue. You can split the events into different queues by running an SQS client that pulls the messages and republishes them to different queues depending on the event type.

SQS Message Payload

The body of each SQS message is a JSON-encoded structure that provides support for multiple events in each message.

The JSON-encoded structure contains the following:

The Event structure contains the following:

Double Delivery

Amazon SQS already provides a MessageId value that enables double-delivery detection in the typical SQS case. However, when receiving messages from Mechanical Turk, we recommend that you use the EventDocId value for double-delivery detection. This will cover an additional scenario in which you may see the same EventDocId in two messages with distinct MessageIds.

Most messages are safe to process twice, since they represent independent one-way state changes. Consider whether detection of repeated messages is important for your application. You may be able to simply process the message and ignore it if it appears to have been applied already.