Integration Flashcards
Step Functions tasks
- Lambda tasks
- Activity tasks: HTTP worker (EC2, mobile device, on premise)
- Service task: connect to supported aws services
- Wait task
Step Functions Standard vs Express
- Standard: exactly-once execution maximum 1 year duration. Priced per state transition. Executions can be listed and described
- Express: at-least-once execution maximum 5 minutes duration. Higher start and transition rate. Priced by number of executions.
Step Functions
Synchronous and Asynchronous Express Workflows
Asynchronous Express Workflows return confirmation that the workflow was started, but don’t wait for the workflow to complete
Synchronous Express Workflows start a workflow, wait until it completes, and then return the result.
SQS
- Used to decouple services
- Max 256KB for message size
- Could be used as a write buffer for DynamoDB
- Supports FIFO
SQS Dead Letter Queue
After the MaximumReceives threshold is exceeded the message goes into a DLQ. The queue type should match the original queue.
It is useful for debugging.
SQS Idempotency
Messages can be processed twice by consumer. Idempotency must be implemented at the consumer level.
Amazon MQ
Managed Message broker for RabbitMQ and ActiveMQ, doesn’t scale as much as SQS.
Use it with traditional protocols when migrating to the cloud, insted of re-engineering the application.
S3 events to multiple queues
Use the fan-out pattern with SNS+SQS to send the same S3 event type and prefix.
Integration between SNS and S3
From SNS send to Kinesis Data Firehose and from there to S3.
https://medium.com/@twcardenas/streaming-data-to-s3-from-sns-using-firehose-1f27a8c23a0c
SNS Message Filtering
Policy to filter messages sent to SNS topics subscriptions. If a subscription doesn’t have a filter policy, it receives every message.
SNS Message Delivery Retries
In case of server side error a delivery policy is applied
* Immediate retry phase (no delay): 3 times for AWS managed endpoints
* Pre backoff phase
* Backoff phase
* Post backoff phase
SNS Custom Delivery Policies
Only HTTP/S supports custom policies.
1. Healthy Retry Policy
2. Sickly Retry Policy
3. Throttle Policy
SNS - Dead Letter Queue
After exhausting the delivery policy, messages are sent to a DLQ.
DLQ are attached to a subscription.
API Gateway HTTP endpoint AWS service integrations
- EventBridge
- SQS
- AppConfig
- Kinesis
- Step Functions
Amazon Pinpoint
Send push notifications, in-app notifications, emails, text messages, voice messages, and messages over custom channels.