Application Integration | Amazon Simple Queue Service (SQS) Flashcards

Question

How does Amazon SQS identify messages? Features, Functionality, and Interfaces Amazon Simple Queue Service (SQS) | Application Integration

Answer 1

All messages have a global unique ID that Amazon SQS returns when the message is delivered to the message queue. The ID isn’t required to perform any further actions on the message, but it is useful for tracking the receipt of a particular message in the message queue. When you receive a message from the message queue, the response includes a receipt handle that you must provide when deleting the message.

Answer 2

In Amazon SQS, you can use the API or the console to configure dead letter queues, which are queues that receive messages from other source queues. If you make a queue into a dead letter queue, it receives messages after a maximum number of processing attempts cannot be completed. You can use dead letter queues to isolate messages that can't be processed for later analysis. For more information, see "Can I use a dead letter queue with FIFO queues?" on this page and Using Amazon SQS Dead Letter Queues in the Amazon SQS Developer Guide.

Answer 3

The visibility timeout is a period of time during which Amazon SQS prevents other consuming components from receiving and processing a message. For more information, see Visibility Timeout in the Amazon SQS Developer Guide.

Answer 4

Every Amazon SQS queue has a configurable visibility timeout. A message is not visible to any other reader for a designated amount of time when it is read from a message queue. As long as the amount of time it takes to process the message is less than the visibility timeout, every message is processed and deleted. If the component processing of the message fails or becomes unavailable, the message again becomes visible to any component reading the message queue once the visibility timeout ends. This allows multiple components to read messages from the same message queue, each one working to process different messages.

Answer 5

The maximum visibility timeout for an Amazon SQS message is 12 hours.

Answer 6

Yes. An Amazon SQS message can contain up to 10 metadata attributes. You can use message attributes to separate the body of a message from the metadata that describes it. This helps process and store information with greater speed and efficiency because your applications don't have to inspect an entire message before understanding how to process it. Amazon SQS message attributes take the form of name-type-value triples. The supported types include string, binary, and number (including integer, floating-point, and double). For more information, see Using Amazon SQS Message Attributes in the Amazon SQS Developer Guide.

Answer 7

To determine the time-in-queue value, you can request the SentTimestamp attribute when receiving a message. Subtracting that value from the current time results in the time-in-queue value.

Answer 8

Typical latencies for SendMessage, ReceiveMessage, and DeleteMessage API requests are in the tens or low hundreds of milliseconds.

Answer 9

When the AWS account ID is not available (for example, when an anonymous user sends a message), Amazon SQS provides the IP address.

Answer 10

Amazon SQS long polling is a way to retrieve messages from your Amazon SQS queues. While the regular short polling returns immediately, even if the message queue being polled is empty, long polling doesn’t return a response until a message arrives in the message queue, or the long poll times out. Long polling makes it inexpensive to retrieve messages from your Amazon SQS queue as soon as the messages are available. Using long polling might reduce the cost of using SQS, because you can reduce the number of empty receives. For more information, see Amazon SQS Long Polling in the Amazon SQS Developer Guide.

Answer 11

No. Long-polling ReceiveMessage calls are billed exactly the same as short-polling ReceiveMessage calls.

Answer 12

In almost all cases, Amazon SQS long polling is preferable to short polling. Long-polling requests let your queue consumers receive messages as soon as they arrive in your queue while reducing the number of empty ReceiveMessageResponse instances returned. Amazon SQS long polling results in higher performance at reduced cost in the majority of use cases. However, if your application expects an immediate response from a ReceiveMessage call, you might not be able to take advantage of long polling without some modifications to your application. For example, if your application uses a single thread to poll multiple queues, switching from short polling to long polling will probably not work, because the single thread will wait for the long-poll timeout on any empty queues, delaying the processing of any queues that might contain messages. In such an application, it is a good practice to use a single thread to process only one queue, allowing the application to take advantage of the benefits that Amazon SQS long polling provides.

Answer 13

In general, you should use maximum 20 seconds for a long-poll timeout. Because higher long-poll timeout values reduce the number of empty ReceiveMessageResponse instances returned, try to set your long-poll timeout as high as possible. If the 20-second maximum doesn't work for your application (see the example in the previous question), set a shorter long-poll timeout, as low as 1 second. All AWS SDKs work with 20-second long polls by default. If you don't use an AWS SDK to access Amazon SQS, or if you configured your AWS SDK to specifically have a shorter timeout, you might need to modify your Amazon SQS client to allow longer requests or to use a shorter long-poll timeout.

Answer 14

The AmazonSQSBufferedAsyncClient for Java provides an implementation of the AmazonSQSAsyncClient interface and adds several important features: Automatic batching of multiple SendMessage, DeleteMessage, or ChangeMessageVisibility requests without any required changes to the application Prefetching of messages into a local buffer that allows your application to immediately process messages from Amazon SQS without waiting for the messages to be retrieved Working together, automatic batching and prefetching increase the throughput and reduce the latency of your application while reducing your costs by making fewer Amazon SQS requests. For more information, see Client-Side Buffering and Request Batching in the Amazon SQS Developer Guide. The Amazon SQS Buffered Asynchronous Client does not currently support FIFO queues.

Answer 15

You can download the AmazonSQSBufferedAsyncClient as part of the AWS SDK for Java.

Answer 16

No. The AmazonSQSBufferedAsyncClient for Java is implemented as a drop-in replacement for the existing AmazonSQSAsyncClient. If you update your application to use the latest AWS SDK and change your client to use the AmazonSQSBufferedAsyncClient for Java instead of the AmazonSQSAsyncClient, your application will receive the added benefits of automatic batching and prefetching.

Answer 17

In the Amazon SQS console, select an Amazon SQS standard queue. Under Queue Actions, select Subscribe Queue to SNS Topic from the drop-down list. In the dialog box, select the topic from the Choose a Topic drop-down list, and click Subscribe. For more information, see Subscribing a Queue to an Amazon SNS Topic in the Amazon SQS Developer Guide.

Answer 18

Use Amazon SNS to create a topic. Create and subscribe multiple Amazon SQS standard queues to the Amazon SNS topic. Whenever a message is sent to the Amazon SNS topic, it is fanned out to the Amazon SQS message queues. Amazon SNS delivers the message to all Amazon SQS message queues subscribed to the topic.

Answer 19

Amazon SNS is designed such that each message is delivered at least once to Amazon SQS standard queues.

Answer 20

Yes. You can delete all messages in an Amazon SQS message queue using the PurgeQueue action. When you purge a message queue, all the messages previously sent to the message queue are deleted. Because your message queue and its attributes remain, there is no need to reconfigure the message queue; you can continue using it. To delete only specific messages, use the DeleteMessage or DeleteMessageBatch actions.

Answer 21

Standard Queues Unlimited Throughput: Standard queues support a nearly unlimited number of transactions per second (TPS) per API action. At-Least-Once Delivery: A message is delivered at least once, but occasionally more than one copy of a message is delivered. Best-Effort Ordering: Occasionally, messages might be delivered in an order different from which they were sent. FIFO Queues High Throughput: By default, FIFO queues support up to 300 messages per second (300 send, receive, or delete operations per second). When you batch 10 messages per operation (maximum), FIFO queues can support up to 3,000 messages per second. To request a limit increase, file a support request. Exactly-Once Processing: A message is delivered once and remains available until a consumer processes and deletes it. Duplicates aren't introduced into the queue. First-In-First-Out Delivery: The order in which messages are sent and received is strictly preserved (i.e. First-In-First-Out). sqs-what-is-sqs-standard-queue-diagram sqs-what-is-sqs-fifo-queue-diagram You can use standard message queues in many scenarios, as long as your application can process messages that arrive more than once and out of order, for example: Decouple live user requests from intensive background work: Let users upload media while resizing or encoding it. Allocate tasks to multiple worker nodes: Process a high number of credit card validation requests. Batch messages for future processing: Schedule multiple entries to be added to a database. FIFO queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can't be tolerated, for example: Ensure that user-entered commands are executed in the right order. Display the correct product price by sending price modifications in the right order. Prevent a student from enrolling in a course before registering for an account.

Answer 22

FIFO queues are currently available in the US West (Oregon), US East (Ohio), US East (N. Virginia), and EU (Ireland) regions. This feature will be available in more regions in the coming months.

Answer 23

FIFO queues are designed to never introduce duplicate messages. However, your message producer might introduce duplicates in certain scenarios: for example, if the producer sends a message, does not receive a response, and then resends the same message. Amazon SQS APIs provide deduplication functionality that prevents your message producer from sending duplicates. Any duplicates introduced by the message producer are removed within a 5-minute deduplication interval. For standard queues, you might occasionally receive a duplicate copy of a message (at-least-once delivery). If you use a standard queue, you must design your applications to be idempotent (that is, they must not be affected adversely when processing the same message more than once).

Answer 24

No. Amazon SQS standard queues (the new name for existing queues) remain unchanged, and you can still create standard queues. These queues continue to provide the highest scalability and throughput; however, you will not get ordering guarantees and duplicates might occur. Standard queues are appropriate for many scenarios, such as work distribution with multiple idempotent consumers.

Answer 25

No. You must choose the queue type when you create it. However, it is possible to move to a FIFO queue. For more information, see Moving From a Standard Queue to a FIFO Queue in the Amazon SQS Developer Guide.

Answer 26

To take advantage of FIFO queue functionality, you must use the latest AWS SDK. FIFO queues use the same API actions as standard queues, and the mechanics for receiving and deleting messages and changing the visibility timeout are the same. However, when sending messages, you must specify a message group ID. For more information, see FIFO Queue Logic in the Amazon SQS Developer Guide. Important: You can't convert an existing standard queue into a FIFO queue. To make the move, you must either create a new FIFO queue for your application or delete your existing standard queue and recreate it as a FIFO queue. For more information, see Moving from a Standard Queue to a FIFO Queue in the Amazon SQS Developer Guide.

Answer 27

Some AWS or external services that send notifications to Amazon SQS might not be compatible with FIFO queues, despite allowing you to set a FIFO queue as a target. The following features of AWS services aren't currently compatible with FIFO queues: Amazon CloudWatch Events Amazon S3 Event Notifications Amazon SNS Topic Subscriptions Auto Scaling Lifecycle Hooks AWS IoT Rule Actions AWS Lambda Dead Letter Queues For information about compatibility of other services with FIFO queues, see your service documentation.

Answer 28

FIFO queues aren't currently compatible with the Amazon SQS Buffered Asynchronous Client. FIFO queues are compatible with the Amazon SQS Extended Client Library for Java and the Amazon SQS Java Message Service (JMS) client.

Answer 29

FIFO queues support all metrics that standard queues support. For FIFO queues, all approximate metrics return accurate counts. For example, the following AWS CloudWatch metrics are supported: ApproximateNumberOfMessagesDelayed - The number of messages in the queue that are delayed and not available for reading immediately. ApproximateNumberOfMessagesVisible - The number of messages available for retrieval from the queue. ApproximateNumberOfMessagesNotVisible - The number of messages that are in flight (sent to a client but have not yet been deleted or have not yet reached the end of their visibility window).

Answer 30

Messages are grouped into distinct, ordered "bundles" within a FIFO queue. For each message group ID, all messages are sent and received in strict order. However, messages with different message group ID values might be sent and received out of order. You must associate a message group ID with a message. If you don't provide a message group ID, the action fails. If multiple hosts (or different threads on the same host) send messages with the same message group ID are sent to a FIFO queue, Amazon SQS delivers the messages in the order in which they arrive for processing. To ensure that Amazon SQS preserves the order in which messages are sent and received, ensure that multiple senders send each message with a unique message group ID.

Answer 31

Yes. One or more producers can send messages to a FIFO queue. Messages are stored in the order that they were successfully received by Amazon SQS. If multiple producers send messages in parallel, without waiting for the success response from SendMessage or SendMessageBatch actions, the order between producers might not be preserved. The response of SendMessage or SendMessageBatch actions contains the final ordering sequence that FIFO queues use to place messages in the queue, so your multiple-parallel-producer code can determine the final order of messages in the queue.

Answer 32

By design, Amazon SQS FIFO queues don't serve messages from the same message group to more than one consumer at a time. However, if your FIFO queue has multiple message groups, you can take advantage of parallel consumers, allowing Amazon SQS to serve messages from different message groups to different consumers.

Answer 33

Yes. However, you must use a FIFO dead letter queue with a FIFO queue. (Similarly, you can use only a standard dead letter queue with a standard queue.)

Answer 34

Without batching, FIFO queues can support up to 300 messages per second (300 SendMessage, ReceiveMessage, or DeleteMessage operations per second). If you take advantage of the maximum batching of 10 messages per operation, FIFO queues can support up to 3,000 messages per second. Amazon SQS standard queues have unlimited throughput.

Answer 35

The name of a FIFO queue must end with the .fifo suffix. The suffix counts towards the 80-character queue name limit. To determine whether a queue is FIFO, you can check whether the queue name ends with the suffix.

Answer 36

Amazon SQS stores all message queues and messages within a single, highly-available AWS region with multiple redundant Availability Zones (AZs), so that no single computer, network, or AZ failure can make messages inaccessible. For more information, see Regions and Availability Zones in the Amazon Relational Database Service User Guide.

Answer 37

Authentication mechanisms ensure that messages stored in Amazon SQS message queues are secured against unauthorized access. You can control who can send messages to a message queue and who can receive messages from a message queue. For additional security, you can build your application to encrypt messages before they are placed in a message queue. Amazon SQS has its own resource-based permissions system that uses policies written in the same language as AWS Identity and Access Management (IAM) policies: for example, you can use variables, just like in IAM policies. For more information, see Amazon SQS Policy Examples in the Amazon SQS Developer Guide. Amazon SQS supports the HTTP over SSL (HTTPS) and Transport Layer Security (TLS) protocols. Most clients can automatically negotiate to use newer versions of TLS without any code or configuration change. Amazon SQS supports versions 1.0, 1.1, and 1.2 of the Transport Layer Security (TLS) protocol in all regions.

Answer 38

When Amazon SQS returns a message to you, the message stays in the message queue whether or not you actually receive the message. You're responsible for deleting the message and the deletion request acknowledges that you’re done processing the message. If you don’t delete the message, Amazon SQS will deliver it again on when it receives another receive request. For more information, see Visibility Timeout in the Amazon SQS Developer Guide.

Answer 39

No. FIFO queues never introduce duplicate messages. For standard queues, under rare circumstances, you might receive a previously-deleted message a second time. This can happen in the rare situation when a DeleteMessage operation doesn’t delete all copies of a message because one of the servers in the distributed Amazon SQS system isn’t available at the time of deletion. This message copy can be delivered again. If you use standard queues, design your application to be idempotent (that is, no errors or inconsistencies occur if you receive a deleted message a second time).

Answer 40

When you issue a DeleteMessage request on a previously-deleted message, Amazon SQS returns a success response.

Answer 41

Server-side encryption (SSE) lets you transmit sensitive data in encrypted queues. SSE protects the contents of messages in Amazon SQS queues using keys managed in the AWS Key Management Service (AWS KMS). SSE encrypts messages as soon as Amazon SQS receives them. The messages are stored in encrypted form and Amazon SQS decrypts messages only when they are sent to an authorized consumer. AWS KMS combines secure, highly available hardware and software to provide a key management system scaled for the cloud. The following are benefits of using AWS KMS: You can create and manage customer master keys (CMKs) yourself. You can also use the AWS-managed CMK for Amazon SQS, which is unique for each account and region. The AWS KMS security standards can help you meet encryption-related compliance requirements. For more information, see the following resources: Protecting Data Using Server-Side Encryption (SSE) and AWS KMS in the Amazon SQS Developer Guide What is AWS Key Management Service? in the AWS KMS Developer Guide The AWS Key Management Service Cryptographic Details whitepaper

Answer 42

SSE for Amazon SQS is available in the US East (N. Virginia and Ohio) and US West (Oregon) regions. This feature will be available in more regions over the coming months.

Answer 43

To enable SSE for a new or existing queue using the Amazon SQS API, specify the customer master key (CMK) ID: the alias, alias ARN, key ID, or key ARN of the an AWS-managed CMK or a custom CMK by setting the KmsMasterKeyId attribute of the CreateQueue or SetQueueAttributes action. For detailed instructions, see Creating an Amazon SQS Queue with Server-Side Encryption and Configuring Server-Side Encryption (SSE) for an Existing Amazon SQS Queue in the Amazon SQS Developer Guide.

Answer 44

Both standard and FIFO queues support SSE.

Answer 45

Before you can use SSE, you must configure AWS KMS key policies to allow encryption of queues and encryption and decryption of messages. To enable SSE for a queue, you can use the AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK. For more information, see Customer Master Keys in the AWS KMS Developer Guide. To send messages to an encrypted queue, the producer must have the kms:GenerateDataKey and kms:Decrypt permissions for the CMK. To receive messages from an encrypted queue, the consumer must have the kms:Decrypt permission for any CMK that is used to encrypt the messages in the specified queue. If the queue acts as a dead letter queue, the consumer must also have the kms:Decrypt permission for any CMK that is used to encrypt the messages in the source queue. For more information, see What Permissions Do I Need to Use SSE? in the Amazon SQS Developer Guide.

Answer 46

There are no additional Amazon SQS charges. However, there are charges for calls from Amazon SQS to AWS KMS. For more information, see AWS Key Management Service Pricing. The charges for using AWS KMS depend on the data key reuse period configured for your queues. For more information, see How Do I Estimate My AWS KMS Usage Costs? in the Amazon SQS Developer Guide.

Answer 47

SSE encrypts the body of a message in an Amazon SQS queue. SSE doesn't encrypt the following components: Queue metadata (queue name and attributes) Message metadata (message ID, timestamp, and attributes) Per-queue metrics Amazon SQS generates data keys based on the AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK to provide envelope encryption and decryption of messages for a configurable time period (from 1 minute to 24 hours).

Answer 48

SSE uses the AES-GCM 256 algorithm.

Answer 49

Encrypting a message makes its contents unavailable to unauthorized or anonymous users. Encrypting messages doesn't affect the normal functioning of Amazon SQS: A message is encrypted only if it is sent after the encryption of a queue is enabled. Amazon SQS doesn't encrypt backlogged messages. Any encrypted message remains encrypted even if the encryption of its queue is disabled.

Answer 50

Moving a message to a dead letter queue does not affect its encryption: If you move a message from an encrypted source queue to a unencrypted dead letter queue, the message remains encrypted. If you move a message from a unencrypted source queue to an encrypted dead letter queue, the message remains unencrypted.

Answer 51

SSE doesn't limit the throughput (TPS) of Amazon SQS. The number of SSE queues that you can create is limited by the following: The data key reuse period (1 minute to 24 hours). The AWS KMS per-account limit (100 TPS by default). The number of IAM users or accounts that access queues. The existence of a large backlog (a larger backlog requires more AWS KMS calls). For example, let's assume the following limits: You set your data key reuse period to 5 minutes (300 seconds). Your KMS account has a default AWS KMS TPS limit of 100 TPS. You use an Amazon SQS queue without a backlog and with 1 IAM user for SendMessage or ReceiveMessage actions to all queues. In this case, you can calculate the theoretical maximum of Amazon SQS queues with SSE as follows: 300 seconds × 100 TPS / 1 IAM user = 30,000 queues

Answer 52

To predict costs and better understand your AWS bill, you might want to know how often Amazon SQS uses your CMK. Note: Although the following formula can give you a very good idea of expected costs, actual costs might be higher because of the distributed nature of Amazon SQS. To calculate the number of API requests per queue (R), use the following formula: R = B / D \* (2 \* P + C) B is the billing period (in seconds) D is the data key reuse period (in seconds) P is the number of producing principals that send to the Amazon SQS queue. C is the number of consuming principals that receive from the Amazon SQS queue. Important: In general, producing principals incur double the cost of consuming principals. For more information, see How Does the Data Key Reuse Period Work? in the Amazon SQS Developer Guide. If the producer and consumer have different IAM users, the cost increases. For example calculations, see How Do I Estimate My Customer Master Key (CMK) Usage Costs? in the Amazon SQS Developer Guide. For exact pricing information, see AWS KMS Service Pricing.

Answer 53

Yes. Amazon SQS is PCI DSS Level 1 certified. For more information, see PCI Compliance.

Answer 54

Yes, AWS has expanded its HIPAA compliance program to include Amazon SQS as a HIPAA Eligible Service. If you have an executed Business Associate Agreement (BAA) with AWS, you can use Amazon SQS to build your HIPAA-compliant applications, store messages in transit, and transmit messages—including messages containing protected health information (PHI). If you already have an executed BAA with AWS, you can start using Amazon SQS right away. If you don't have a BAA or have other questions about using AWS for your HIPAA-compliant applications, contact us for more information. Note: If you prefer not to transfer PHI through Amazon SQS (or if you have messages larger than 256 KB), you can alternatively send Amazon SQS message payloads through Amazon S3 using the Amazon SQS Extended Client Library for Java (Amazon S3 is a HIPAA Eligible Service, excluding the use of Amazon S3 Transfer Acceleration). For more information, see Using the Amazon SQS Extended Client Library for Java in the Amazon SQS Developer Guide.

Answer 55

Longer message retention provides greater flexibility to allow for longer intervals between message production and consumption. You can configure the Amazon SQS message retention period to a value from 1 minute to 14 days. The default is 4 days. Once the message retention limit is reached, your messages are automatically deleted.

Answer 56

To configure the message retention period, set the MessageRetentionPeriod attribute using the console or using the Distributiveness method. Use this attribute to specify the number of seconds a message will be retained in Amazon SQS. You can use the MessageRetentionPeriod attribute to set the message retention period from 60 seconds (1 minute) to 1,209,600 seconds (14 days). For more information on working with this message attribute, see the Amazon SQS API Reference.

Answer 57

To configure the maximum message size, use the console or the SetQueueAttributes method to set the MaximumMessageSize attribute. This attribute specifies the limit on bytes that an Amazon SQS message can contain. Set this limit to a value between 1,024 bytes (1 KB), and 262,144 bytes (256 KB). For more information, see Using Amazon SQS Message Attributes in the Amazon SQS Developer Guide. To send messages larger than 256 KB, use the Amazon SQS Extended Client Library for Java. This library lets you send an Amazon SQS message that contains a reference to a message payload in Amazon S3 that can be as large as 2 GB.

Answer 58

Amazon SQS messages can contain up to 256 KB of text data, including XML, JSON and unformatted text. The following Unicode characters are accepted: #x9 | #xA | #xD | [#x20 to #xD7FF] | [#xE000 to #xFFFD] | [#x10000 to #x10FFFF] For more information, see the XML 1.0 Specification.

Answer 59

A single Amazon SQS message queue can contain an unlimited number of messages. However, there is a 120,000 limit for the number of inflight messages for a standard queue and 20,000 for a FIFO queue. Messages are inflight after they have been received from the queue by a consuming component, but have not yet been deleted from the queue.

Answer 60

You can create any number of message queues.

Answer 61

Queue names are limited to 80 characters.

Answer 62

You can use alphanumeric characters, hyphens (-), and underscores (\_).

Answer 63

A message queue's name must be unique within an AWS account and region. You can reuse a message queue's name after you delete the message queue.

Answer 64

You can associate an access policy statement (and specify the permissions granted) with the message queue to be shared. Amazon SQS provides APIs for creating and managing access policy statements: AddPermission RemovePermission SetQueueAttributes GetQueueAttributes For more information, see the Amazon SQS API Reference.

Answer 65

The message queue owner pays for shared message queue access.

Answer 66

The Amazon SQS API uses the AWS account number to identify AWS users.

Answer 67

To share a message queue with an AWS user, provide the full URL from the message queue you want to share. The CreateQueue and ListQueues operations return this URL in their responses.

Answer 68

Yes. You can configure an access policy that allows anonymous users to access a message queue.

Answer 69

The permissions API provides an interface for sharing access to a message queue to developers. However, this API cannot allow conditional access or more advanced use cases.

Answer 70

The SetQueueAttributes operation supports the full access policy language. For example, you can use the policy language to restrict access to a message queue by IP address and time of day. For more information, see Amazon SQS Policy Examples in the Amazon SQS Developer Guide.

Answer 71

For service region availability, see the AWS Global Infrastructure Region Table. Note: FIFO queues are currently available only in the US West (Oregon), US East (Ohio), US East (N. Virginia), and EU (Ireland) regions. This feature will be available in more regions in the coming months.

Answer 72

No. Each Amazon SQS message queue is independent within each region.

Answer 73

Amazon SQS pricing is the same for all regions, except China (Beijing). For more information, see Amazon SQS Pricing.