All Flashcards

(231 cards)

1
Q

IAM Policy Structure

A

Version: 2012-10-17
ID (optional): identifier for the policy
Statement: one or more individual statements

Statement consists of:

  • Sid: identifier for the statement (optional)
  • Effect: whether the statement allows or denies access
  • Principal: account/user/role to which applied
  • Action: list of resources to which actions are applied
  • Condition: condition for when the policy is in effect (optional)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is the issue for the following errors:

  1. Your application is not accessible and you get a “timed out” when trying to access it.
  2. If your application gives a “connection refused” error
A
  1. Security Group issue

2. Application error, or it’s not launched (security group worked)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q
What are these Ports?
21
22
80
443
3389
A

21 - FTP (File Transfer Protocol) - upload files into a file share

22 - SSH (Secure Shell) - log into Linux instance, but also SFTP (Secure File Transfer Protocol) - upload files using SSH

80 - HTTP access to unsecured website

443 - HTTPS access to a secured website

3389 - RDP (Remote Desktop Protocol) - log into a Windows Instance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How to connect to Linux EC2 using PowerShell?

A

ssh -i PathTo.pemFile ec2-user@PublicIpAddressOfInstance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Pros and Cons of Cluster Placement Group - All instances on the same server rack in same AZ

A

Pros: Great Network - 10Gbps bandwidth between instances

Cons: If the rack fails, all instances fail at the same time

Use Case: Big data job that needs to complete fast or app that needs extremely low latency and high network throughput

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Pros and Cons of Spread Placement Group - All instances are located on different racks/hardware, and across AZs

A

Pros: Can spread across AZs for reduced risk of simultaneous failure.

Cons: Limited to 7 instances per AZ per placement group

Use Case: App that needs to maximize high availability and critical apps where each instance must be isolated from failure from each other

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Pros and Cons of Partition Placement Group - Each partition is a rack and each partition can have multiples in each AZ and spread across multiple AZ.

A

Pros: Up to 7 partitions per AZ and can spread across multiple AZs (within the same region) for up to 100s of instances. Don’t share the same hardware so a failure isn’t catastrophic

Use Cases: Big Data apps

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is an ENI?

A

Elastic Network Interface - acts as virtual network card and is a component of the VPC

They are bound to a specific AZ

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is EC2 Nitro?

A

New underlying platform for EC2 instances.

Will have higher speed EBS, better security, better networking

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Why would you want to change the default vCPU options?

A

Sometimes licensing is charged based on number of cores. So the default of 2 threads per core and 8 cores (which would be 16 vCPU) could cost a lot. So if you want to keep the same amount of RAM, but don’t need all those vCPU, you can disable multithreading (allow just 1 thread per core) and lower the amount of overall cores to lower the cost of the licensing charges

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

EBS MultiAttach

A

Usually an EBS volume can only be attached to ONE ec2 instance at a time. However, with io1/io2 family EBS Volumes, you can attach these to multiple instances within the same AZ.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Which can be mounted in multi AZ?

EFS or EBS

A

EFS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Can Windows instances have an EFS mounted?

A

No. Only for Linux

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

EFS Performance Modes

A

General Purpose - latency sensitive use cases (web servers)

Max I/O - higher latency, throughput, highly parallel (big data, media processing)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

EFS Throughput Modes

A

Bursting - 1TB = 50MiB/s + burst of up to 100MiB/s

Provisioned - set your throughput regardless of storage size (ex: 1GiB/s for 1TB storage)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What layer is TCP? HTTP? HTTPS? Network?

A

Network = Layer 3
TCP = Layer 4
HTTP and HTTPS = Layer 7

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What protocol and on which port does the Gateway Load Balancer use?

A

GENEVE protocol on port 6081

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Sticky Sessions

A

This mean that a client accessing an EC2 instance through a load balancer will be directed to the same EC2 instance every time. This is done via a “cookie”. The cookie will expire eventually, and the client will be directed to whichever instance the load balancer sees fit if it is a duration based cookie. You can also create your own without an expiry date

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Which load balancer always has Cross Zone Load balancing?

A

Application Load Balancer. There are no charges for inter-AZ data transfer
It can be enabled for NLB and CLB. Only NLB will charge you for inter-AZ data transfer.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is SNI (Server Name Indication)?

A

SNI solves the problem of loading multiple SSL certificates onto one web server (to serve multiple websites)
It’s a newer protocol and requires the client to indicate the hostname of the target server in the initial SSL handshake

Only works for ALB & NLB or CloudFront (not CLB)

In other words, you can have an ALB or NLB balance traffic between 2 different websites at once. When a user wants to access one of the websites, it will use SNI to tell the load balancer which site they want, so the load balancer can select the right SSL certificate, and encrypt the traffic to the correct site

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Connection Draining/Deregistration Delay

If using a CLB, it is called Connection Draining. If using an ALB or NLB, it is called Deregistration Delay

A

This is a setting on an EC2 instance where once it becomes unhealthy, it doesn’t shutdown right away. The load balancer will stop routing new traffic to it, but for the traffic that has already been routed to it, the draining time will allow those people time to finish their task before the instance shuts down. Default is 300 seconds. Can go up to 3600 seconds.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Which is becoming legacy and which is new between “Launch Configuration” and “Launch Template”? (used for auto scaling groups)

A

Configuration is legacy, template is newer

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

You are using an Application Load Balancer to distribute traffic to your website hosted on EC2 instances. It turns out that your website only sees traffic coming from private IPv4 addresses which are in fact your Application Load Balancer’s IP addresses. What should you do to get the IP address of clients connected to your website?

A

When using an Application Load Balancer to distribute traffic to your EC2 instances, the IP address you’ll receive requests from will be the ALB’s private IP addresses. To get the client’s IP address, ALB adds an additional header called “X-Forwarded-For” contains the client’s IP address.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Application Load Balancers can route traffic to different Target Groups based on the following, EXCEPT:

Client Location
Hostname
Request URL Path
Source IP Address

A

Client Location. ALBs can route traffic to different Target Groups based on URL Path, Hostname, HTTP Headers, and Query Strings.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
For compliance purposes, you would like to expose a fixed static IP address to your end-users so that they can write firewall rules that will be stable and approved by regulators. What type of Elastic Load Balancer would you choose?
Network Load Balancer has one static IP address per AZ and you can attach an Elastic IP address to it. Application Load Balancers and Classic Load Balancers have a static DNS name.
26
What are the RDS database engines?
``` Postgres MariaDB MySQL Oracle Microsoft SQL Server Aurora ```
27
Why is using RDS better than just launching your own database on an EC2 instance?
RDS is managed by AWS which means it has automated provisioning, OS patching, continuous backups, monitoring dashboard, read replicas, can be multi-AZ for disaster recovery, can set up maintenance windows, scaling ability, and backed by EBS. Downside is that you can't SSH into it since it is managed by AWS, not you.
28
Does RDS storage scale automatically?
Yes. You can set a maximum
29
Can RDS read replicas span across AZs?
Read Replicas can be within an AZ, cross AZ, or even cross region
30
Are RDS read replicas Asynchronous? (meaning that you can read them before they have a chance to match the main database exactly)
Yes, so reads are EVENTUALLY consistent.
31
What is RDS Multi AZ?
A feature used mainly as a disaster recovery in which there is a SYNCHRONOUS read replica created that is completely unused unless the main fails. This read replica will share the DNS Name with the master database so if that master database fails, the read replica will automatically take over as master
32
How much downtime is average for going from a single AZ RDS database to a mult-AZ database?
There is no downtime
33
If you create an RDS Database, and elect to not encrypt it, how can you later encrypt the read replicas made from this database?
You can't.
34
At what step do you encrypt an RDS database?
Must be defined at launch time
35
What are 2 ways to encrypt your RDS Database?
AWS KMS - AES-256 | Transparent Data Encryption (TDE) (only available for Oracle or SQL Server)
36
How do you encrypt an unencrypted RDS database?
Create a snapshot of the unencrypted database, copy that snapshot, and when you create that copy, you'll have the option to enable encryption. Then restore the database from the encrypted snapshot. This creates a new, encrypted database to which you can migrate everything over to. Then delete the unencrypted database
37
Which database engines support access management to the RDS database via IAM authentication?
MySQL and PostgreSQL
38
RDS Shared Responsibility
Your responsibility: - Check the ports/IP/security group inbound rules - In-database user creation and permissions or manage through IAM - Creating a database with or without public access - Ensure parameter groups or DB is configured to only allow SSL connections AWS responsibility: Since RDS is a managed service, you will have: - No SSH access - No manual DB patching - No manual OS patching - No way to audit the underlying instance
39
Aurora FAQs
- Not open sourced - 5x performance over MySQL and over 3x for Postgres - Storage automatically grows from 10GB up to 128TB - Can have 15 read replicas and the process is much faster - Failover is basically instant - Costs ~20% more than RDS, but much more efficient - Makes 6 copies of your data across 3 AZs - One master, but any of the replicas can become master for failover - Supports cross region replication
40
Aurora DB Cluster
There is a master that does read and write. But instead of you connecting to the master directly, you connect to a WRITER ENDPOINT that directs you to the master. (Good incase the master fails, you don't have to find the new master). Same thing for reading the read replicas. You don't connect to them directly, but instead, you connect to a READER ENDPOINT which also acts as a load balancer for all the replicas. Replicas can also auto scale.
41
Aurora Security
Similar to RDS.
42
Aurora Custom Endpoints
Aurora automatically has a Reader Endpoint to guide you to read replicas and load balance. However, if you have a variety of instance sizes making up the read replicas, you may want to use larger instance sizes for more work intensive queries. To do this, you can create custom endpoints that override the default reader endpoint.
43
Elasticache FAQs
- Caches are in-memory databases with really high performance, low latency - RDS is to get managed relational databases as Elasticache is to get managed Redis or memcached - Helps reduce load off of databases for read intensive workloads - Helps make your app stateless - AWS takes care of OS maintenance/patching, optimizations, setup, config, monitoring, failure recovery and backups - Involves heavy app code changes to use - Works kind of like CloudFront but for databases. App needs something from the database, but will check Elasticache first to see if it's been cached.
44
Redis vs Memcached
Redis: - Multi AZ with auto failover - Read replicas to scale reads and have high availability - data durability using AOF persistence - backup and restore features Memcached - Multi-node partitioning of data (sharding) - No high availability (replication) - No persistence - No backup and restore - Multi-threaded architecture
45
ElastiCache Security
- Does not support IAM Authentication - Redis AUTH lets you create a password (token) when you create a Redis user and supports SSL in flight encryption - Memcached supports SASL-based authentication
46
Patterns for ElastiCache
Lazy Loading: all the read data is cached, data can become stale in cache Write Through: Adds or updates data in the cache when written to DB (no stale data) Session Store: store temp session data in a cache (using TTL features)
47
ElastiCache - Redis Use Case
- Gaming leaderboards are computationally complex - Redis Sorted Sets guarantee both uniqueness and element ordering - Each time a new element is added, it's ranked in real time, then added in correct order
48
``` FQDN (Fully Qualified Domain Name): Protocol: Domain Name: Sub Domain: Second Level Domain: Third Level Domain: Root: ```
``` FQDN (Fully Qualified Domain Name): http://api.www.example.com. Protocol: http Domain Name:api.www.example.com. Sub Domain:www.example.com. Second Level Domain:example.com. Third Level Domain:.com. Root: . ```
49
How DNS Works
When you type in your URL (example.com), you are asking your local DNS Server for example.com. If your local DNS server (assigned by your company or ISP) doesn't know it, the local DNS server will go ask the Root DNS Server (managed by ICANN). If the Root DNS server doesn't know it, it will at least tell you where to look and give you the info for the TLD DNS Server since you are looking for a .com website. If the TLD DNS Server doesn't know, it can give you a bit more info and lead you to the SLD DNS Server, which is the Domain Registrar (Route 53, GoDaddy, etc), and they will know the IP Address for example.com. Your local DNS Server will now cache that info
50
Is Route 53 authoritative?
yes
51
Main Route 53 DNS record types?
A AAAA CNAME NS
52
A: AAAA: CNAME: NS:
A: maps a host name to IPv4 AAAA: maps a host name to IPv6 CNAME: maps a host name to another host name - the target is a domain name that must have an A or AAAA record - Can't create a CNAME record for the top node of a DNS namespace (Zone Apex) (For example, you can't create for example.com, but you can for www.example.com NS: Name Servers for the hosted zone - Control how traffic is routed for a domain
53
Route 53 Hosted Zones
A container for records that define how to route traffic to a domain and its subdomains - Public Hosted Zones: contains records that specify how to route traffic on the internet (public domain names) - Private Hosted Zones: contain records that specify how you route traffic within one or more VPCs (private domain names) You pay $0.50 per month per hosted zone
54
CNAME vs Alias
AWS Resources (load balancers, CloudFront, etc) expose an AWS host name such as lb1-1234.us-east-2.elb.amazonaws.com. But if you want to use myapp.mydomain.com you can use: CNAME - Point the ugly hostname to the pretty hostname. But ONLY for non-root domains. So it'll work for myapp.mydomain.com, but not mydomain.com Alias - Specific to Route 53 and allows you to point the hostname to an AWS resource. This does work for root domains as well as non-root domains. The are also free
55
Alias record targets
``` ELB CloudFront API Gateway Elastic Beanstalk S3 websites (not the bucket, but the website) VPC Endpoints Global Accelerator Route 53 Record (same Hosted Zone) ``` CANNOT for EC2 DNS name
56
``` Route 53 Routing Policies: Simple Weighted Failover Latency Based Geolocation Multi-Value Answer Geoproximity (using Route 53 Traffic Flow feature) ```
Simple - Typically route traffic to a single source (ask for a.com, get back 11.2.55.213) but can be multiple sources, and one is picked at random (ask for x.com, get back 1.2.3.4 as well as 5.6.8.7) (no health checks) Weighted - Route traffic to multiple sources, but assign values to say one source is more or less likely than others (dns records must have the same name and type and there can be health checks) Typically used for load balancing across regions or testing a new app by only sending a small percentage of traffic to the testing instance. If one instance is given 0 weight, no traffic will go there, but if ALL instances are given the weight of 0, they will all return traffic equally Failover: - Active/Passive: If an instance fails a health check, route 53 will direct traffic to the back up instance Latency Based - Auto Directs traffic based on how quickly they can access the instance. (health checks available) Geolocation: Based on where the user is located. Create a "default" record first in case there is no matching location. (health checks available) Does not auto direct, you set which locations go to which instance. Multi-Value Answer - Up to 8 records will be available for a user to access. (health checks available) Geoproximity (using Route 53 Traffic Flow feature) - route traffic based on both user location as well as resource location. Use bias values to give weight. (example: there is a resource set up in N. Virginia and another set up in California, and there are 4 users accessing this resource. User locations are Virginia, California, Louisiana, New Mexico. If bias on both resources is set to 0, it will act like latency based routing. But if you give N. Virginia a bias of 50, and California a bias of 0, then that means N. Virginia has a wider reach, and will take the users in Virginia and Louisiana of course, but now will also take the New Mexico user)
57
Route 53 Health Checks
HTTP health checks are only for PUBLIC resources and will check if the resource is working before sending traffic. - Monitor at Endoint: About 15 different health checks will report - Calculate health checks: Combine health checks into one - Private Hosted Zone: Since health checks can't access instances in a private subnet, you would create a CloudWatch Alarm to go off when the instance becomes unhealthy, and the health checker can watch for that alarm instead of trying to watch the instance
58
Route 53 Traffic Flow
This is a UI to more easily create and manage DNS routing records
59
Elastic Beanstalk Cost?
Free itself, only pay for the services provisioned.
60
S3 Keys
Files have a key. If the bucket URL is s3://mybucket, then a jpg named bob in that bucket will have the URL of s3://mybucket/bob.jpg. If you place the bob.jpg in a folder call dude, then the URL for the jpg will be s3://mybucket/dude/bob.jpg. The key for the jpg is everything after the s3://mybucket. So the key for the bob.jpg in the dude folder would be dude/bob.jpg To break down the key, dude/ would be the prefix and bob.jpg would be the object name
61
S3 Encryption Options
SSE-S3 - keys handled and managed by AWS - AES-256 - We upload the object, S3 provides and applies the key - Header will be: "x-amz-server-side-encryption":"AES256" SSE-KMS - AWS KMS to handle and manage keys - Advantages are user control + audit trail - We upload the object, S3 provides and applies the key - Header will be: "x-amz-server-side-encryption":"aws:kms" SSE-C - You manage your own keys - S3 does not store the key - HTTPS must be used - We upload the object and the key, but S3 will still apply the key to encrypt Client Side Encryption - You encrypt the object before uploading to S3 and decrypt it when retrieving it
62
S3 Security
User based: IAM Policies - Which API calls should be allowed for a specific user from IAM console Resource based: Bucket Policies - bucket wide rules from S3 console, allows cross account
63
SDK
Must use when coding against AWS Services such as DynamoDB
64
S3 MFA-Delete
Must enable versioning Only the root user can enable/disable MFA-Delete Can only be enabled using the CLI, not the console Once enabled, you'll need to get an MFA code to permanently delete an object or suspend versioning
65
Amazon Glacier Retrieval Options
Expidited - 1 to 5 minutes Standard - 3 to 5 hours Bulk - 5 to 12 hours (minimum storage duration is 90 days)
66
Amazon Glacier Deep Archive Retrieval options
Standard - 12 Hours Bulk - 48 hours Minimum storage duration is 180 days
67
S3 KMS Limitation
KMS does have a limit of requests per second (varies by region), so if you have this as your default encryption on your S3 bucket, then it's possible to hit that limit
68
S3 multi-part upload
Recommended for files over 100MB | Required for files over 5GB
69
S3 Transfer Acceleration
Increase transfer speed by transferring file to an AWS edge location which will forward the data to the S3 bucket in the target region. This means that we only use the public internet to get it to the edge location, then use the private AWS network to get it from the edge location to the bucket
70
S3 byte range fetches
Can be used to speed up downloads. Downloads in parts instead of all in a single file
71
S3 select and Glacier Select
This allows you to use SQL to make requests from specific parts of a file. So if you only need a few rows and columns from a CSV file, you can request this, and S3 will filter out what you want, and deliver it
72
S3 - Requester Pays
In general, bucket owner pays for all costs. This will allow for the requester to be billed for the request. (bucket owner still pays for the storage, just not the retrieval)
73
Athena
Serverless query service to perform analytics against S3 objects uses SQL to query
74
Unicast vs Anycast IP (Global Accelerator uses Anycast IP)
Unicast IP: one server holds one IP address Anycast IP: All servers hold the same IP address and the client is routed to the nearest one. So a user will want to connect to your ALB. Instead of connecting via the public internet, they will connect to the closest edge location, then travel the rest of the way over the AWS private network
75
Snowball Edge
FOR STORAGE ONLY: 40 vCPUs 80TB Use for data transfers of less than a petabyte FOR EDGE COMPUTING: 52 vCPUs, 208GB RAM Optional GPU 42TB storage
76
Snowcone
FOR STORAGE ONLY: 8TB Use for data transfers up to 24TB ``` FOR EDGE COMPUTING: 2CPUs 4GB memory wired or wireless access USB-C power with optional battery ```
77
Snowmobile
100PB
78
AWS OpsHub
A software you install locally to give a GUI for using the snow devices. (without this, a CLI is needed)
79
Amazon FSx
Allows you to launch 3rd party high-performance files system on AWS such as: FSx for Lustre FSx for Windows File Server
80
FSx for Windows File Server
EFS is a shared POSIX system for Linux systems (so can't use for windows instances). So this is when you'd use FSx for Windows File Server Supports SMB protocol & Windows NTFS Scalable Can be multi AZ backed up to S3
81
FSx for Lustre
A type of parallel distributed file system for large-scale computing The name Lustre is a combo of Linux and cluster Used for things like Machine Learning and High Performance Computing (HPC) Scalable Seamless integration with S3 (read and write)
82
FSx File System Deployment Options
Scratch File System: Temporary and data is not replicated. Very Fast 200MBps per TB). Used for short term processing Persistent File System: Long term storage and data is replicated within same AZ. Replaces failed files within minutes. Use for long term processing
83
How do you expose S3 for on prem access?
Storage Gateway
84
Storage Gateway
bridge between on prem and cloud 3 TYPES File Gateway: - Configured S3 buckets are accessible using the NFS and SMB protocol - Supports S3 standard, IA, One Zone IA - Bucket access using IAM roles for each file gateway - Most recently accessed data is cached in the file gateway - Can be mounted on many servers - Integrated with Active Directory (AD) for user authentication Volume Gateway - - Block Storage using iSCSI protocol backed by S3 - Backed by EBS snapshots which can help restore on prem volumes - There are both Cached volumes for low latency access to most recent data, or Stored Volumes to have the entire dataset on prem with scheduled backups to S3 Tape Gateway - Some companies have backup processes using physical tapes - Virtual Tape Library (VTL) backed by S3 and Glacier
85
Storage Gateway Hardware
Others require a gateway to be installed like software on your own servers. If you don't have room for that, then you can buy a piece of hardware
86
FSx File Gateway
Native access to FSx for Windows File Server | Local cache for frequently accessed data
87
AWS Transfer Family
Fully managed service to transfer to and from S3 and EFS using the FTP protocol Uses either FTP, FTPS or SFTP (FTP is the only one that isn't encrypted)
88
Storage Comparison
S3: Object storage Glacier: Object Archival EFS: Network file system for LINUX FSx for Windows: same as EFS but for windows FSx for Lustre: HPC Linux file system EBS: network storage for one EC2 instance at a time Instance Store: Physical store for EC2. Faster than EBS, but goes away when EC2 shutsdown Storage Gateway: transfers on-prem to and from cloud Snow family: same as storage gateway, but with phyical devices instead of over network Database: indexing and querying
89
SQS
Default Retention: 4 days, maximum of 14 days | Less than 256KB per message
90
SQS Dead Letter Queue
If a message is failed to be processed, it goes back to the queue, and if this keeps happening, you can have the message leave the main queue and enter a "Dead Letter Queue" so you can review it later and debug the issue
91
Kinesis
Data Streams: capture, process, and store data streams Firehose: load data streams into AWS data stores Data analytics: analyze data streams with SQL or Apache Flink Video Streams: capture, process, and store video streams
92
Kinesis Data Streams
Streams are made up of shards Billing is per shard provisioned retention is between 1 and 365 days
93
Kinesis Firehose
Stores data from data producers (like Kinesis Data Streams, CloudWatch, IoT), will batch things together to write in batches to consumers like S3, Redshit, or ElasticSearch Pay for data going through.
94
Kinesis Data Streams vs Kinesis Data Firehose
Data streams: - Streaming service for ingest at scale - Write custom code - Real time (~200ms) - Managed scaling (shard splitting/merging) - Data storage for 1 to 365 days - Supports replay capability Firehose: - Load streaming data into S3, Redshift, ElasticSearch - Fully Managed - Near real time (since it does things in batches) - Auto scaling - No data storage - Doesn't support replay capability
95
Kinesis Data Analytics (SQL Application)
Data Streams and Firehose will be the source of data sent to Data Analytics, then you write SQL code to process this data in real time, and have the results of the SQL query either sent back to data streams to be sent to Lamda or EC2 apps, or to firehose, which can send to S3, Redshift, ElasticSearch Pay for data that goes through it
96
Amazon MQ
Use when migrating a non-AWS queueing system from on-prem to the cloud without re doing everything
97
ECS Task Roles
Allow each task to have a specific role and use different roles for different ECS services you run
98
How do ECS tasks share data?
You can mount EFS volumes to tasks, doesn't matter if they are launched in an EC2 instance, or if they are launched via Fargate. It is also multi-AZ
99
ECS Scaling
Service CPU Usage - Lauch a new task if CPU usage goes above certain percentage or vice versa. If you are using EC2 instances instead of fargate, you also have the option to use ECS Capacity Provider scaling that will launch a new instance if your current instances don't have enough room to launch a new task when the tasks need to scale out SQS Queue - have a queue set up between users and tasks, once the queue gets to be too long, scale. ECS Capacity Provider available here as well
100
ECS Rolling Updates
When updating from v1 to v2, we can control how many tasks can be started and stopped and in which order. So if you have 4 tasks on V1, and need to update to V2. You can set 50% min and 100% max. So of the 4 tasks running, 2 will go offline to update. Once those 2 come back online as V2, the other two V1s will go offline to update. So you never have less than 50% of your 4 tasks running
101
ECR
Used to store, manage and deploy containers.
102
EKS (K = Kubernetes)
Way to launch managed kubernetes clusters. Alternate to ECS. EKS is open source, while ECS isn't. Tasks are called Pods in EKS
103
Serverless services in AWS
``` Lamda DynamoDB Cognito API Gateway Amazon S3 SNS SQS Kinesis Data Firehose Aurora Serverless step functions Fargate ```
104
Lamda
``` Virtual functions limited by time (15 minutes) run on demand scaling is automated up to 10GB or RAM per function Increasing RAM will also improve CPU and network Supports the following languages: - Node.js - Python - Java - C# - Golang - C#/Powershell - Ruby - Custom runtime API (to run any language) ```
105
Containers on Lamda
The container image must implement the Lamda Runtime API. If it is an arbitrary Docker image without the lamda runtime API, then use ECS/Fargate
106
Main Lamda Integrations
``` API Gateway - create a rest API Kinesis - DynamoDB - S3 - CloudFront - CloudWatch Events/Event Bridge - CloudWatch Logs - SNS - SQS - Cognito - ```
107
Lamda use case examples
- You can set up a lamda function with S3 so that when an image is uploaded to S3, the lamda function will trigger, and create a thumbnail version of that image and place it in the same or a different bucket. You can also have it tied to DynamoDB so that you can store the thumbnails metadata (like creation date, dimensions, etc) - CRON job can be set up. (CRON = a scheduling task). You set up a CloudWatch Events/Event Bridge to go every X amount of minute/hours, and this will trigger the Lamda function. Better than using EC2 since the EC2 will have to run constantly, and lamda would only run when triggered
108
Lamda pricing
First 1,000,000 requests are free and then $0.20 per million requests after that. You also pay per duration of compute time. First 400,000 GB-second are free. (this means you have 400,000 free seconds of compute time if the function is 1GB. If it's only 128MB, then you get 3.2 million seconds). Then it's $1 per 600,000 GB-seconds after that
109
Lamda Limits (per region)
Execution: - Memory Allocation: 128MB - 10GB (1MB increments) - Maximum execution time: 900 seconds (15 minutes) - Environment variables (4kb) - Disk capacity in the "function container" (in /tmp): 512MB - Concurrent executions: 1000 (can be increased) Deployment: - Lamda function deployment size (compressed .zip): 50MB - Size of uncompressed deployment (code + dependencies): 250MB - Can use the /tmp directory to load other files at startup - Size of environment variables: 4KB
110
Lamda@edge
Deploy lamda functions at edge locations Can change cloudfront requests and responses. Basically, when a user access CloudFront, (to get access to your website/app, and have the origin return a response) Lamda can modify the sent messages to cloudfront from the user, the sent message from cloudfront to the origin, the response from the origin to cloudfront, and the response from cloudfront back to the user.
111
DynamoDB
Fully managed, highly available (replication across multiple AZs) NoSQL millions of requests per second low latency retrieval Made of Tables - each table has a primary key that is created at the time the table is created - each table can have an infinite number of items (rows) - each item has attributes (can be added over time - can be null) - maximum item size of an item is 400KB Data types supported are: - Scalar - Document - Set
112
DynamoDB - Read/Write Capacity Modes
Control how you manage your table's capacity Provisioned Mode: - Specify the number of reads/writes per second at time of creating - So you need to plan capacity beforehand - Pay for what you provisioned - Possibility to add auto-scaling On-Demand Mode: - Read/writes auto scale - No capacity planning needed - Pay for what you use, but this is more expensive
113
DynamoDB Accelerator (DAX)
In-Memory Cache helps solve read congestion by caching 5min TTL for cache (default)
114
DyanmoDB Streams
``` Ordered stream of item-level modifications (create/update/delete) in a table Stream records can be: - sent to kinesis data streams - read by lamda - read by kinesis client library apps ``` Data retention for up to 24 hours
115
DynamoDB Global Tables
Having the same table replicated in multiple regions. "Active-Active" replication. Can READ and WRITE to the table from any region Must enable DynamoDB streams as a pre-requisite
116
DynamoDB TTL (time to live)
Can set an expiry time for items or rows to automatically delete after a certain amount of time
117
DynamoDB Indexes
GSI - Global Secondary Index LSI - Local Secondary Index Allow to query on attributes other than the primary key
118
DynamoDB Transactions
Can write to more than one table at once (or neither). Example: if your app can process a bank transaction, you'd want to write to the table that is the ledger of transactions, but also write to the table that lists the account balance. must write to both. Can't write to one without the other
119
API Gateway
If you want a client to be able to enovke a lamda function, then there are a few ways like letting them talk directly to the lamda, but that means the client would need IAM permissions. Could place an ALB between the client and lamda, and that would expose the lamda as an HTTP endpoint. OR We could use API Gateway. This will create a rest API that will be public and accessible. And the API Gateway would send along the client requests to the lamda
120
API Gateway Integrations
Lamda Function HTTP AWS Service (expose any AWS API through API Gateway)
121
API Gateway - Endpoint Types
Edge-Optimized (default): - For global clients - requests are routed through edge locations - the API Gateway still lives in only one region Regional: - For clients in the same region - Can manually combine with cloudfront Private: - Can only be accessed from your VPC using an interface VPC endpoint (ENI)
122
API Gateway - Security
IAM Permissions: Can create an IAM policy for a user/role and the API gateway will verify permissions before sending to the backend app. (Uses "Sig v4" to do so). Only for internal use as you can't give IAM permissions to someone outside the company. Lamda Authorizer (formerly Custom authorizers): Uses Lamda to validate the token in header being passed. Option to cache authentication. Cognito User Pool: Cognito fully manages user lifecycle and API gateway verifies identity with cognito. Only authentication, not authorization. So user gets a token from cognito once authenticated, then user sends token to API gateway to authorize based on the token
123
Cognito
Used when we want to give our users an identity so that they can interact with our app. Cognito User Pools: - Sign in functionality for app users - Integrate with API Gateway Cognito Identity Pools (Federated Identity): - Provide AWS credentials to users so they can access AWS resources directly - Integrate with cognito user pools as an identity provider Cognito Sync: - Sync data from device to cognito - May be replaced by AppSync
124
Cognito User Pools (CUP)
Create a serverless database of user for your mobile apps Simple Login: username/email and password combo can enable federated identities (facebook, google etc logins) sends back JSON web token that can be used with API Gateway for authentication
125
Federated Identity Pools
Goal: - Provide direct access to AWS resources from the client side How: - Log in to federated identity provider - or remain anonymous - Get temp AWS credentials that come with IAM permissions ex: provide temp access to write to S3 using Facebook login
126
SAM - Serverless Application Model
Framework for developing and deploying serverless apps | - All config is YAML code
127
Databases Decisions
What are the needs? - read heavy? write heavy? balanced? - throughput needs? consistent or changing? - how much data store? for how long? will it grow? object size? how are they accessed? - Data durability? source of truth for the data? - latency requirements? concurrent users? - data model? how will you query the data? joins? structured? semi-structured? - strong schema? flexibility? noSQL? - license costs?
128
Database Options
- RDBMS (=SQL/OLTP): RDS, Aurora - great for joins - NoSQL database: DyanmoDB (similar to JSON docs), ElastiCache (key/value pairs), Neptune (graphs) - No joins, noSQL - Object Store: S3 (for big objects)/Glacier (for backups/archives) - Data warehouse: (=SQL analytics/Bi): redshift (OLAP), Athena - Search: ElasticSearch (JSON): free text, unstructured searches - Graphs: Neptune: display relationships between data
129
RDS Engines
Managed PostgreSQL/MySQL/Oracle/SQL Server
130
Aurora Engines
PostgreSQL/MySQL
131
ElastiCache Engines
A cache, so Redis/Memcached
132
DyanmoDB Engines
AWS proprietary
133
Redshift Engine
PostgreSQL
134
CloudWatch Dashboard
Global, so they can include graphs from different regions and even different AWS accounts ``` 3 Dashboards (up to 50 metrics) for free $3 per dashboard per month after that ```
135
CloudWatch Logs for EC2
By default, no logs are sent to CloudWatch. Must run a CloudWatch agent on EC2 to do so. Also, need to make sure IAM perms are correct Can also install the agent on an on-prem server as well
136
EC2 Instance Recovery
Instance Status: check the EC2 System status: Check the underlying hardware Recovery: If instance goes into alarm, then a cloudwatch alarm can start a recovery
137
AWS Config
Helps record config changes over time Regional, but can be aggregated across regions and accounts and even stored in S3 to be later queried by Athena
138
CloudWatch vs CloudTrail vs Config
CloudWatch: - Performance monitoring and dashboards - Events and Alerts - Log aggregation & analysis CloudTrail: - Record API calls within your account by everyone - Can define trails for specific resources - Global Service Config: - Record config changes - Evaluate resources against compliance rules - Get timeline of changes and compliance
139
CloudWatch/CloudTrail/Config uses with an ELB
CloudWatch: - Monitor incoming connections metric - Visualize error codes as a % over time - Make a dashboard to get an idea of your load balancer performance Config: - Track security group rules for the ELB - Track config changes for the ELB - Ensure an SSL cert is always assigned to the ELB CloudTrail: - Track who made any changes to the ELB with API calls
140
STS - Security Token Service
Grant limited access to AWS resources for 1 hour (can be refreshed) When you see "cross account access" think STS
141
Using STS to assume a role
- Define an IAM Role within your account or cross account - Define which principals can access this IAM roles - Use STS to retrieve credentials and impersonate the IAM role you have access to - Valid from 15 minutes to 1 hour
142
Identity Federation
Federation lets users outside of AWS to assume temp roles for accessing AWS resources. Using federation, you don't need to create IAM users. (user management is outside of AWS)
143
Identity Federation: SAML 2.0
Integrate with Active directory provide access to console or CLI no need to create an IAM user for each of the employees ex: Employee in an on-prem location needs to access aws console. the employee will browse to the on-prem identity provider that will verify with an identity store. This will authenticate the employee to then get an authorization from STS and assume a role that will grant them access to the console This is the "old way", and the new way is "Single SignOn"
144
Identity Federation: Custom Identity Broker Application
Use only if provider is not SAML compatible. Similar to SAML, but the difference in the example is that instead of the employee requesting authorization after authentication from the identity provider, the identity provider will request authorization, then pass the credentials to the employee
145
Identity Federation: AssumeRoleWithWebIdentity
Not recommended by AWS, use Cognito instead With Cognito: Goal - provide direct access to AWS resources from the client side (mobile, webapp) Example - provide temp access to write to S3 bucket using Facebook login Problem - We don't want to create IAM users for our app users How - Login to federated identity provider (facebook, google, etc) or remain anonymous and get temp access from the federated identity pool. Comes with a pre-defined IAM policy stating their permissions
146
Microsoft Active Directory
Database of objects: User accounts, computers, printers, files shares, security groups Centralize security management objects are organized in trees
147
AWS Directory Service
AWS Managed Microsoft AD (Active Directory) - Create your own AD in AWS, manage users locally, supports MFA - Establish a "trust" connection with your on-prem AD. - Can authenticate on AWS or on-prem ADs AD Connector - Directory gateway (proxy) to redirect to on-prem AD - Users are managed on the on-prem AD - Only authenticate with on-prem AD. Requests through AWS AD are forwarded to on-prem AD Simple AD - AD-compatible managed directory on AWS - Cannot be joined with on-prem AD - Just the AWS AD
148
AWS Organizations
- Global Service - Allows to manage multiple AWS accounts - The main account is the master account (can't change) - Other account are "member" accounts - Member accounts can only be part of one organization - Consolidated Billing across all accounts - single payment method - Pricing benefits from aggregated usage (volume discounts) - API is available to automate AWS account creation
149
Multi Account Strategies
- Keep services from talking to each other - Compliance - Staggered billing Can still enable cloudwatch on all accounts and send logs to a central S3
150
Organizational Units (OU)
A way to separate accounts for security purposes. Ex: Sales OU, Retail OU, Finace OU, HR OU etc.
151
Service Control Policies (SCP)
- Whitelist or Blacklist IAM actions - Applied at OU or account level - Does not apply to master account - SCP is applied to all the users and roles of the account, including root - SCP doesn't allow anything by default
152
AWS Organization - Moving Accounts
To move an account from Org 1 to Org 2, the account must: 1. Be removed from Org 1 2. Get an invite to Org 2 3. Accept invite to Org 2 If you want to move the master account, you must: 1. Remove ALL the member accounts from Org 1 2. Delete Org 1 3. Repeat process above to get to Org 2
153
IAM Conditions
- aws:SourceIP: restrict the client IP FROM which the API calls are being made (This mean that you can do things like deny access to everything unless it comes from a specific IP) - aws:RequestedRegion: restrict the region the API calls are made TO (This means you can allow access to certain services ONLY if the request comes from a certain AWS Region) can also have tag based restrictions as well as MFA based restrictions
154
IAM Permission Boundries
- Supported for users and roles, but NOT groups - Can set the maximum permissions an IAM entity can get Ex: You can create a user and give him the Admin policy. But then you can set a permissions boundary to only allow access to S3. The boundary will overrule the regular admin policy, and he could only do S3 things
155
Resource Access Manager (RAM)
Share AWS resources that you own with other AWS accounts or within your org such as: VPC Subnets: - allows all resources to be launched in the same subnets - must be from the same org - cannot share security groups or default VPC - participants can manage their own resources - participants can't view/modify/delete resources that don't belong to them
156
AWS Single Sign-On (SSO)
Centrally managed Single Sign-On to access multiple accounts and 3rd party apps Integrated with organizations Supports SAML Integration with On-prem AD centralize perm management centralized auditing with cloudtrail
157
KMS (Key management service)
``` CUSTOMER MASTER KEY (CMK) TYPES Symmetric (AES-256 keys): - First offering - Necessary for envelope encryption - You never get access to the key unencrypted (muist call KMS API to use) ``` Asymmetric (RSA&ECC key pairs) - Public key to encrypt, private key to decrypt - Used for encrypt/decrypt, but also sign/verify ops - Public key is downloadable, but you can't access the private key unencrypted - Use case: encryption outside AWS where the user can't call the KMS API
158
KMS FAQs
- Able to audit key usage using CloudTrail - 3 types of CMKs - - AWS Managed Service Default CMK: free - - User Keys created in KMS $1/month - - User Keys imported (must be 256 symmetric) $1/mo - Pay for API call to KMS ($0.03/10,000 calls) - Can only encrypt 4kb of data per call, if more than 4kb, use envelope encryption - To use, make sure both the key policy allows the user and the IAM policy allows the API calls - Keys are bound to a specific region.
159
Copying a KMS encrypted data store (such as EBS) across regions
To do this, you must first take a snapshot of the volume. This snapshot will also be encrypted with KMS. Then you copy that snapshot to the new region. This will retain the KMS encryption, but will change the key. Then recreate the volume using the snapshot which will remain encrypted with the newly created KMS Key
160
KMS Key Policies
- Cannot control access without the policies Default Key Policy: - created if you don't provide a specific policy - Grants complete access to the key to the root user - To give someone access to the keys, create the IAM policies to access the KMS Key Custom KMS Policy: - Define users, roles, who can admin - Useful for cross-account access of keys
161
Copying snapshots across accounts
- Create snapshot encrypted with your own CMK - Attach key policy to authorize cross-account - Share encrypted snapshot - The receiver would need to create a copy of the snapshot, encrypt it with a KMS key in their account and create the volume from that snapshot
162
KMS Automatic Key Rotation
Only for customer managed CMK (not the default AWS managed CMKs) - if enabled, will be every year - previous key is kept active to decrypt old data - new key has same ID but the backend is different
163
KMS Manual Key Rotation
- Can pick rotation periods (90days, 180 days) - New key has different ID since you create it manually - Keep previous key active - better to use aliases* (hides the change of key for the app) - Good solution to rotate CMK that are not eligible for auto-rotation (like asymmetric CMK) *Aliases are basically fronts between the app and the actual key. The app calls on the alias, and the alias will direct the app to the correct key. So you can change keys all the time, but the app won't know, because it is only in contact with the alias
164
SSM Parameter Store
- Secure Storage for config and secrets - Optional seamless encryption using KMS - Serverless, scalable, durable, easy SDK - Versioning - Can be used with cloudwatch events - Integrated with cloudformation
165
Parameter Policies (only available for advanced tier, not free tier)
- Allow to assign TTL to a parameter to force updating or deleting sensitive data such as passwords - Can assign multiple policies at a time
166
AWS Secrets Manager
- Newer service - Meant for storing secrets - Capability to force rotation every X days - Automate generation of secrets on rotation (uses Lamda) - Integration with RDS - encrypted with KMS Anytime you see "secret store" "secret manager" "rds integration", think Secrets Manager
167
CloudHSM
- KMS = AWS manages the software for encryption - CloudHSM = AWS provisions encryption hardware - Dedicated Hardware (HSM = Hardware Security Module) - You manage your own encryption keys entirely (not AWS) - HSM devices is tamper resistant - Supports symmetric and asymmetric keys - no free tier - Must use the CloudHSM Client Software - Redshift integration - Multi AZ - Good option to use with SSE-C encryption
168
Shield (DDoS protection)
Shield Standard: - Free and auto activated Shield Advanced: - $3000/mo - Protect against more sophisticated DDoS attacks - 24/7 access to DRP (DDoS response team) - protect against higher fees during usage spikes due to DDoS
169
Web Application Firewall (WAF)
- Protects your web app from common web exploits (Layer 7) - Layer 7 is HTTP (Layer 4 is TCP) - Can ONLY be deployed on ALB, API Gateway, CloudFront To use, you need to define a Web ACL (Access Control List): - Rules can include IP addresses, http headers, http body, or URL strings - protects from common attacks like SQL injection and cross-site scripting (XSS) - Size constraints, geo-match (block countries) - Rate-based rules (to count occurences of events) for DDoS protection (ex: can say a specific IP can't make more than 5 calls in 60 seconds)
170
Firewall Manager
Manage rules in all accounts of AWS Organization - WAF Rules - Shield advanced - security groups
171
GuardDuty
- Intelligent Threat Discovery to protect AWS Account - Uses Machine Learning algorithms, anomaly detection, 3rd party data - Can protect against crypto currency attacks (has a dedicated finding for it) - Monitors VPC Flow Logs, CloudTrail Logs, DNS Logs
172
Inspector
- Automated Security Assessments for EC2 instances (ONLY for EC2) - Analyze the running OS against known vulnerabilities - Analyze against unintended network accessibility - AWS Inspector agent must be installed on OS in EC2 instance Once installed, it will assess then get you a report. Can send notification to SNS
173
Macie
- Fully managed. - Uses machine learning and pattern matching to discover and protect your sensitive data in AWS - Helps ID and alert you to personally identifiable info (PII)
174
CIDR - IPv4
``` CIDR = Classless Inter-domain Routing Help identify IP ranges - ww.xx.yy.zz/32 = 1 IP - 0.0.0.0/0 = all IPs - 192.168.0.0/26 = 192.168.0.0 through 192.168.0.63 (64 IP addresses) ```
175
CIDR - IPv4
Consists of 2 components: Base IP - Represent an IP contained in the range - Example: 192.168.0.0 ``` Subnet Mask - Defines how many bits can change in the IP - Example /0, /24, /32 - Can take 2 forms: } /8 = 255.0.0.0 } /16 = 255.255.0.0 } /24 = 255.255.255.0 } /32 = 255.255.255.255 ```
176
Subnet Masks
The /number is part of a math equation to determine how many IPs we can have. Example: 192.168.0.0/32 = 1 IP because 32 = 0 in the 2^0 equation and 2^0 = 1 192.168.0.0/31 = 2 IP because 31 = 1 in the 2^1 equation and 2^1 = 2 As the /number decreases by one, the exponent in the equation is doubled. So 192.168.0.0/30 = 2^2, therefore, 4 IPs /29 = 2^3 = 8 IPs So the IP range all the way to /24f is: - 192.168.0.0 through 192.168.0.255 (256 IPs) If you drop below 24, then the next 0 starts to change. So /16 would be 192.168.0.0 through 192.168.255.255 (65,536 IPs) Below 16 starts to change the 168. And all the way down to 0, which is 0.0.0.0 through 255.255.255.255, or all IPs
177
Public vs Private IP
The Internet Assigned Numbers Authority (IANA) established certain blocks of IPv4 addresses for the use of private (LAN) and public (internet) addresses Private IP can only allow certain values: - 10.0.0.0 through 10.255.255.255 (10.0.0.0/8) in big networks - 172.16.0.0 through 172.31.255.255 (172.16.0.0/12) AWS default VPC in that range - 192.168.0.0 - 192.168.255.255 (192.168.0.0/16) Home networks All other IPs will part of the public internet
178
Default VPC
All new AWS accounts have a default VPC New EC2 instances are launched in this default VPC unless otherwise specified Default VPC has internet connectivity and all EC2 instances inside it have public IPv4 addresses We also get a public and a private IPv4 DNS names
179
VPC in AWS - IPv4
- VPC = Virtual Private Cloud - Can have a max of 5 in an AWS region (can have more upon request though) - Max CIDR per VPC is 5 - For each CIDR, Minimum size is /28 (16 IPs) and max is /16 (65,536 IPs) - Since VPC is private, only the Private IPv4 ranges are allowed: - 10.0.0.0 - 10.255.255.255 (10.0.0.0/8) - 172.16.0.0 - 172.31.255.255 (172.16.0.0/12) - 192.168.0.0 - 192.168.255.255 (192.168.0.0/16) Your VPC CIDR should NOT overlap with your other networks (eg corporate network)
180
Subnet (IPv4)
AWS reserves 5 IP addresses in each subnet. The first 4 and the last one. These 5 IPs cannot be used by you Example: If CIDR block is 10.0.0.0/24 then the reserved IPs are: - 10.0.0.0 Network Address - 10.0.0.1 for the VPC router - 10.0.0.2 for mapping to amazon provided DNS - 10.0.0.3 not used currently, but still reserved - 10.0.0.255 Network Broadcast Address, however, AWS does not support broadcast in a VPC, so AWS doesn't allow it's use So remember this if asked a question that designates a required number of usable IP addresses.
181
Internet Gateway
Allows resources in a VPC to connect to the internet. They don't GIVE internet access, just allow it. Route tables must be edited to complete the access to the internet. Only 1 VPC per internet gateway, and vice versa
182
Bastion Hosts
This is an EC2 instance in a public subnet that acts as a passthrough to an instance in a private subnet. So a user would connect to the bastion host EC2 instance, then from there, could SSH into the private EC2 instance Need to make sure security groups are set up properly to allow this connection to be made
183
NAT Instances (outdated, but might still be on the exam)
- NAT = Network Address Translation - Allows EC2 instances in private subnets to connect to the internet - Must be launched in a public subnet - Must disable EC2 setting: "Source/destination check" - Must have elastic IP attached to it - Route tables must be configured to route traffic from private subnets to the NAT instance
184
NAT Gateway
- AWS Managed NAT Instance - Pay per hour of usage and bandwidth - Created in a specific AZ - Uses an Elastic IP - Can't be used by an EC2 within the same subnet - No security group setup required - Redundant in a single AZ, but for fault tolerance, must create multiple NAT Gateways across multiple AZs - Can't be used as a bastion host
185
DNS Resolution in VPC
DNS Resolution (enableDnsSupport) - Decides if DNS resolution from Route 53 Resolver server is supported for the VPC - True (default) means it queries the Amazon Provider DNS Server at 169.254.169.253 or the reserved IP address at the base of the VPC IPv4 network range plus two (.2) DNS Hostnames (enableDnsHostnames) - By default it is true for the default VPC, and false for newly created VPCs - Won't do anything unless enableDnsSupport is true - If True, assigns public hostname to EC2 instance if it has a public IPv4 The reason you want to do this is so that an EC2 in the public subnet can connect with the EC2 instance in the private subnet via a DNS name such as web.mycompany.private. This is called a private hosted zone
186
NACLs & Security Groups
- One NACL per subnet, but multiple subnets can be covered by a single NACL, new subnets are assigned the default NACL - You define NACL Rules (rule numbers range from 1 - 32,766. Higher precedence with a lower number, so if rule 100 says allow, and rule 200 says deny, then allow will override the deny). Use a * which denies a request in case of no rule match AWS recommends adding rules in increments of 100 so you have room to fit rules between other rules if needed
187
Default NACL
This one accepts everything on both inbound and outbound. Don't modify this one, just create new ones
188
Ephemeral Ports
- For any two endpoints to establish a connection, they must use ports - Clients connect to a defined port, (ie Port 80 for HTTP) and expect a response on an ephemeral port - But for the clients to get a response, the client needs a port for the responding server to connect to. This port is created by the client, and is ephemeral - Different OS has different port ranges
189
NACL with Ephemperal Ports
If you have an EC2 instance in a public subnet trying to talk to an RDS database in a private subnet, and both subnets have their own NACL, then the request from the EC2 to the RDS is easy, just set up normal outbound rules on the public subnet and normal inbound rules on the private subnet. But for the RDS to send a response to the EC2, it needs to use the ephemeral port of the EC2. So the outbound rule on the Private subnet NACL needs to include a range of ports, and the public subnet NACL needs to have inbound rules to cover the ephemeral port range of the RDS
190
VPC Reachability Analyzer
Diagnostic tool that troubleshoots network issues between 2 endpoints. doesn't send packets, just builds a model and analyzes the configuration
191
VPC Peering
- Privately connect 2 VPCs using AWS network - Must not have overlapping CIDRs - Both VPCs must have this enabled to talk to each other - Must update route tables in each VPC subnet to ensure the EC2 instances can talk to each other - Can be cross account and cross region
192
VPC Endpoints (AWS Private Link)
- Private endpoints that allow you to connect to services within the cloud without having to go through the public internet Two Types: - Interface Endpoints: provision an ENI (private IP address) as an entry point (must attach a security group). This supports most AWS services - Gateway Endpoints: Provision a gateway and must be used as a target in a route table. This supports S3 and DynamoDB
193
VPC Flowlogs
Capture info about IP traffic going into your interfaces: - VPC level - Subnet level - ENI level Help monitor & troubleshoot connectivity issues Can send logs to S3 and cloudwatch logs
194
Site-to-site VPN
Want to connect AWS VPC and on-prem data cener Needs: - Virtual Private gateway (VGW) which is a VPN concentrator on the AWS side of the connection - Customer Gateway (CGW) which is a software app or physical device on the customer side of the connection The customer gateway can either be public, and use its own public IP address, or private and go through a NAT device and use the public IP address of the NAT device Once set up, you need to enable route propagation for the VPC in the route table in your subnets Lastly, in order for you to ping EC2 instances from on-prem, make sure the security groups around your EC2 instances allow inbound traffic on the ICMP protocol
195
VPN CloudHub
If you have multiple locations, each with an on-prem data center, and you want them all to talk to each other as well as the AWS VPC, you can set up CloudHub that will allow this communication through the VGW and VPN. It does go over the public network of course.
196
Direct Connect (DX)
Provides a dedicated PRIVATE connection from a remote network to your VPC
197
Direct Connect Gateway
If you want to set up a direct connect to one or more VPCs in many different regions (same account), you use this
198
Direct Connect Connection Types
Dedicated - 1Gbps and 10Gbps capacity - Physical ethernet port dedicated to a customer - Request made to AWS first then completed by AWS Direct Connect Partners Hosted - 50Mbps, 500Mbps, to 10Gbps - Connection requests are made via AWS Direct Connect Partners] - Capacity can be added or removed on demand (so more flexible) Often takes longer than a month to set up
199
Direct Connect encryption
The network is private, but the data is not encrypted Use VPN alongside the direct connect for in transit encryption
200
Direct Connect Resiliency
High Resiliency for Critical Workloads - One connection at multiple locations Maximum Resiliency for Critical workloads - Four connections across 2 locations. Achieved by separate connections terminating on separate devices in more than one location
201
Exposing your VPC to other VPC
Option 1: make it public - goes through www public internet - tough to manage access Option 2: VPC peering - must create many peering relations - opens the WHOLE network Option 3: AWS Private Link (VPC Endpoint Services) - Most Secure and Scalable way to expose a service to many VPCs (your own, or other accounts) - Does not require VPC peering, internet gateway, NAT, route tables etc - Requires a network load balancer on the service VPC and ENI on the customer VPC
202
Transit Gateway
For having transitive peering between thousands of VPC, on-prem data centers. Basically a centralized service that all VPC and on prem data centers connect to, and whoever is connected to transit gateway can talk to each other. - Can work cross region - Share cross account using Resource Access Manager - Route tables are used for limiting the connections - Only service that supports IP Multicast
203
VPC Traffic Mirroring
Allows you to capture and inspect network traffic in your VPC Example: You have a public EC2 instance being accessed, so it has inbound and outbound traffic. You can set up mirroring to also send a copy of the inbound traffic to a NLB pointing to other EC2 instances that have some sort of security software installed. Can set up filters if you don't want ALL traffic mirrored
204
IPv6
IPv4 was for 4.3 billion addresses (they'll be exhausted soon) IPv6 can have 3.4x10^38 addresses All are public, no private range Format is x.x.x.x.x.x.x.x (a single x represent a hexadecimal that ranges from 0000 to ffff)
205
IPv6 in VPC
- If enabled, your EC2 instances will get at LEAST a private internal IPv4 and a public IPv6 address. - CANNOT disable IPv4, but can IPv6
206
Egress only Internet Gateways
Only used for IPv6 (similar to a NAT gateway, but for IPv6 instead of IPv4)
207
RPO and RTO
Recovery Point Objective - A point in time you recover to. So if you do a daily backup at 12:00pm, and disaster happens at 2:00pm, then you will lose those 2 hours worth of work Recovery Time Objective - So the disaster happened at 2:00pm, and it took 1 hour to get back up and running, that one hour is the RTO
208
Disaster Recovery Strategies
- Backup and Restore - Pilot Light - Warm Standby - Hot Site/Multi Site Approach (in order from slowest to fastest RTO)
209
Backup and Restore (High RPO)
Actual backups and just use them to recreate things. Pretty cheap
210
Pilot Light
- A small version of the app is always running in the cloud. - Usually just the critical components of the app (like a pilot light is to a water heater) - Very similar to backup and restore for everything else, but since it has the critical systems already running, it's faster
211
Warm Standby
- Full system is up and running, but at minimum size - When main systems fail, the warm standby will scale to production load and Route 53 will failover to this warm standby
212
Multi Site/Hot Site Approach
- Lowest RTO, but very expensive. | - The backup is a fully functional duplicate of everything
213
Database Migration Service (DMS)
- Quickly and securely migrate databases to AWS - The source database remains available during the migration - Supports both homogeneous and heterogeneous migrations - Continuous data replication - You must create an EC2 instance to perform the replication tasks
214
AWS Schema Conversion Tool (SCT)
- Used when the source database and the destination database are using different engines - You don't need SCT if migrating the same engine
215
DMS Continuous Replication
Example: You have an Oracle DB on-prem and want to migrate to a MySQL DB on RDS. You set up a server on-prem with SCT installed, and that converts the oracle schema to mySQL and puts that into the RDS DB. You then have an EC2 instance with the DMS software installed do the actual data migration
216
DataSync
- Used to move large amount of data from on-prem to AWS. - Can sync to S3 (including glacier), EFS, FSx for Windows - Moves via the NFS or SMB protocols - Can be scheduled to replicate hourly, daily, weekly - Leverage the DataSync agent to connect the systems - Can setup bandwidth limit
217
AWS Backup
- Fully managed backup service - Supports FSx, EFS, DynamoDB, EC2, EBS, RDS (all engines), Aurora, Storage Gateway (Volume Gateway) - Supports cross region and cross account backups - On-demand or scheduled - Can create a backup based on tags
218
S3 Events
Can only trigger SNS, SQS, or Lamda
219
High Performance Computing (HPC) | Data Management & Transfer
- AWS Direct Connect: Mopve GB/s of data to the cloud over a private secure network - Snowball & Snowmobile: Move PB of data to the cloud - AWS DataSync: move large amount of data between on-prem and S3, EFS, FSx for Windows
220
High Performance Computing (HPC) | Compute & Networkiing
- EC2 instances: CPU or GPU optimized. Spot instances/spot fleets for cost savings and auto scaling - EC2 placement group type "Cluster" for good network performance - EC2 enhanced networking (SR-IOV): higher bandwidth, higher packet per second (PPS), lower latency } Option 1: Elastic Network Adapter (ENA), up to 100 Gbps } Option 2: Intel 82599 VF, up to 10 Gbps (legacy) - Elastic Fabric Adapter (EFA): Improved ENA for HPC, ONLY for Linux. Great for inter-node communication, tightly coupled workloads as it leverages Message Passing Interface (MPI) standard which bypasses the Linux OS for lower latency and better reliability
221
High Performance Computing (HPC) | Storage
Instance attached storage: - EBS - Instance Store Network Storage: - S3 - EFS - FSx for Lustre (HPC optimized, backed by S3)
222
High Performance Computing (HPC) | Automation and Orchestration
AWS Batch - supports multi-node parallel jobs, which enables you to run single jobs that span multiple EC2 instances - Easy to schedule AWS Parallel Cluster - cluster management tool - automates creation of VPC, Subnet, cluster type, instance type
223
CICD
Continuous Integration. A way for developers to continuously write code, push it to the test server, and each time there is a passing version on the test server, it is pushed to the live server
224
CloudFormation
- Infrastructure as code - Create a code that automates the provisioning of everything you want, exactly as you want - You upload a template to S3, then reference it in cloudformation. - Templates are YAML files - Stack Sets allow you to create across multiple regions or accounts
225
Step Functions
- Build serverless visual workflow to orchestrate your Lamda functions (this lamda triggers this lamda which triggers this other lamda) - STATE MACHINE
226
SWF
- Simple Workflow Service - Almost the same as Step Functions, but for EC2 instead of Lamda, and not serverless - Old
227
EMR (Elastic Map Reduce)
- Helps create Hadoop clusters (big data) to analyze and process vast amount of data - can be made of hundreds of EC2 instances - takes care of all provisioning for you
228
OpsWorks
- Chef and Puppet help you perform server config automatically, or repetitive actions - They work great with EC2 and on-prem VM - OpsWorks is just managed Chef and Puppet
229
AWS Workspaces
- Managed, secure, Cloud Desktop - On demand (pay per usage) - Secure, encrypted, network isolation - Integrated with Microsoft Active Directory - Replaces VDI (Virtual Desktop Infrastructure)
230
AppSync
- Syncs data across mobile and web apps in real time | - GraphQL
231
Cost Explorer
Visualize, understand and manage your AWS costs and usage over time Choose a savings plan Forecast up to 12 months in the future