Database Specialty - CloudFormation and Automation Flashcards

1
Q

Infrastructure as Code

A
  • Currently, we have been doing a lot of manual work
  • All this manual work will be very tough to reproduce:
    • In another region
    • in another AWS account
    • Within the same region if everything was deleted
  • Wouldn’t it be great, if all our infrastructure was… code?
  • That code would be deployed and create / update / delete our infrastructure
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is CloudFormation

A
  • CloudFormation is a declarative way of outlining your AWS Infrastructure, for
    any resources (most of them are supported).
  • For example, within a CloudFormation template, you say:
    • I want a security group
    • I want two EC2 machines using this security group
    • I want two Elastic IPs for these EC2 machines
    • I want an S3 bucket
    • I want a load balancer (ELB) in front of these machines
  • Then CloudFormation creates those for you, in the right order, with the exact
    configuration that you specify
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Benefits of AWS CloudFormation (1/2)

A
  • Infrastructure as code
    • No resources are manually created, which is excellent for control
    • The code can be version controlled for example using git
    • Changes to the infrastructure are reviewed through code
  • Cost
    • Each resources within the stack is tagged with an identifier so you can easily see how
      much a stack costs you
    • You can estimate the costs of your resources using the CloudFormation template
    • Savings strategy: In Dev, you could automation deletion of templates at 5 PM and
      recreated at 8 AM, safely
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Benefits of AWS CloudFormation (2/2)

A
  • Productivity
    • Ability to destroy and re-create an infrastructure on the cloud on the fly
    • Automated generation of Diagram for your templates!
    • Declarative programming (no need to figure out ordering and orchestration)
  • Separation of concern: create many stacks for many apps, and many layers. Ex:
    • VPC stacks
    • Network stacks
    • App stacks
  • Don’t re-invent the wheel
    • Leverage existing templates on the web!
    • Leverage the documentation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How CloudFormation Works

A
  • Templates have to be uploaded in S3 and then referenced in CloudFormation
  • To update a template, we can’t edit previous ones. We have to reupload a new version
    of the template to AWS
  • Stacks are identified by a name
  • Deleting a stack deletes every single artifact that was created by
    CloudFormation.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Deploying CloudFormation templates

A
  • Manual way:
    • Editing templates in the CloudFormation Designer
    • Using the console to input parameters, etc
  • Automated way:
    • Editing templates in a YAML file
    • Using the AWS CLI (Command Line Interface) to deploy the templates
    • Recommended way when you fully want to automate your flow
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

CloudFormation Building Blocks

A

Templates components (one course section for each):
1. Resources: your AWS resources declared in the template (MANDATORY)
2. Parameters: the dynamic inputs for your template
3. Mappings: the static variables for your template
4. Outputs: References to what has been created
5. Conditionals: List of conditions to perform resource creation
6. Metadata

Templates helpers:
1. References
2. Functions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

YAML Crash Course

A
  • YAML and JSON are the languages you can
    use for CloudFormation.
  • JSON is horrible for CF
  • YAML is great in so many ways
  • Let’s learn a bit about it!
  • Key value Pairs
  • Nested objects
  • Support Arrays
  • Multi line strings
  • Can include comments
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are resources?

A
  • Resources are the core of your CloudFormation template (MANDATORY)
  • They represent the different AWS Components that will be created and
    configured
  • Resources are declared and can reference each other
  • AWS figures out creation, updates and deletes of resources for us
  • There are over 224 types of resources (!)
  • Resource types identifiers are of the form:
    AWS::aws-product-name::data-type-name
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

How do I find resources documentation?

A
  • I can’t teach you all of the 224 resources, but I can teach you how to
    learn how to use them.
  • All the resources can be found here:
    http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aw
    s-template-resource-type-ref.html
  • Then, we just read the docs J
  • Example here (for an EC2 instance):
    http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aw
    s-properties-ec2-instance.html
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Analysis of CloudFormation Template

A
  • Going back to the example of the introductory section, let’s learn why it
    was written this way.
  • Relevant documentation can be found here:
  • http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/awsproperties-ec2-instance.html
  • http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/awsproperties-ec2-security-group.html
  • http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/awsproperties-ec2-eip.htm
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are parameters?

A
  • Parameters are a way to provide inputs to your AWS CloudFormation
    template
  • They’re important to know about if:
    • You want to reuse your templates across the company
    • Some inputs can not be determined ahead of time
  • Parameters are extremely powerful, controlled, and can prevent errors
    from happening in your templates thanks to types.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

When should you use a parameter?

A
  • Ask yourself this:
  • Is this CloudFormation resource configuration likely to change in the future?
  • If so, make it a parameter.
  • You won’t have to re-upload a template to change its content
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

How to Reference a Parameter

A
  • The Fn::Ref function can be leveraged to reference parameters
  • Parameters can be used anywhere in a template.
  • The shorthand for this in YAML is !Ref
  • The function can also reference other elements within the template
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Concept: Pseudo Parameters

A
  • AWS offers us pseudo parameters in any CloudFormation template.
  • These can be used at any time and are enabled by default
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What are mappings?

A
  • Mappings are fixed variables within your CloudFormation Template.
  • They’re very handy to differentiate between different environments
    (dev vs prod), regions (AWS regions), AMI types, etc
  • All the values are hardcoded within the template
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

When would you use mappings vs parameters ?

A
  • Mappings are great when you know in advance all the values that can be
    taken and that they can be deduced from variables such as
    • Region
    • Availability Zone
    • AWS Account
    • Environment (dev vs prod)
    • Etc…
  • They allow safer control over the template.
  • Use parameters when the values are really user specific
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Fn::FindInMap
Accessing Mapping Values

A
  • We use Fn::FindInMap to return a named value from a specific key
  • !FindInMap [ MapName, TopLevelKey, SecondLevelKey ]
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What are outputs?

A
  • The Outputs section declares optional outputs values that we can import into
    other stacks (if you export them first)!
  • You can also view the outputs in the AWS Console or in using the AWS CLI
  • They’re very useful for example if you define a network CloudFormation, and
    output the variables such as VPC ID and your Subnet IDs
  • It’s the best way to perform some collaboration cross stack, as you let expert
    handle their own part of the stack
  • You can’t delete a CloudFormation Stack if its outputs are being referenced
    by another CloudFormation stack
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Outputs Example

A
  • Creating a SSH Security Group as part of one template
  • We create an output that references that security group
21
Q

Cross Stack Reference

A
  • We then create a second template that leverages that security group
  • For this, we use the Fn::ImportValue function
  • You can’t delete the underlying stack until all the references are deleted
    too.
22
Q

What are conditions used for?

A
  • Conditions are used to control the creation of resources or outputs
    based on a condition.
  • Conditions can be whatever you want them to be, but common ones
    are:
    • Environment (dev / test / prod)
    • AWS Region
    • Any parameter value
  • Each condition can reference another condition, parameter value or
    mapping
23
Q

How to define a condition?

A
  • The logical ID is for you to choose. It’s how you name condition
  • The intrinsic function (logical) can be any of the following:
  • Fn::And
  • Fn::Equals
  • Fn::If
  • Fn::Not
  • Fn::Or
24
Q

Using a Condition

A
  • Conditions can be applied to resources / outputs / etc…

Resources:
MountPoint:
Type: “AWS: :EC2: :VolumeAttachment”
Condition: CreateProdResources

25
CloudFormation Must Know Intrinsic Functions
* Ref * Fn::GetAtt * Fn::FindInMap * Fn::ImportValue * Fn::Join * Fn::Sub * Condition Functions (Fn::If, Fn::Not, Fn::Equals, etc…)
26
Fn::Ref
* The Fn::Ref function can be leveraged to reference * Parameters => returns the value of the parameter * Resources => returns the physical ID of the underlying resource (ex: EC2 ID) * The shorthand for this in YAML is !Ref DbSubnet1: Type: AWS: :EC2: :Subnet Properties: VpcId: !Ref MyVPC
27
Fn::GetAtt
* Attributes are attached to any resources you create * To know the attributes of your resources, the best place to look at is the documentation. * For example: the AZ of an EC2 machine!
28
Fn::FindInMap Accessing Mapping Values
* We use Fn::FindInMap to return a named value from a specific key * !FindInMap [ MapName, TopLevelKey, SecondLevelKey ]
29
Fn::ImportValue
* Import values that are exported in other templates * For this, we use the Fn::ImportValue function
30
Fn::Join
* Join values with a delimiter * * This creates “a:b:c”
31
Function Fn::Sub
* Fn::Sub, or !Sub as a shorthand, is used to substitute variables from a text. It’s a very handy function that will allow you to fully customize your templates. * For example, you can combine Fn::Sub with References or AWS Pseudo variables!
32
Condition Functions
* The logical ID is for you to choose. It’s how you name condition * The intrinsic function (logical) can be any of the following: * Fn::And * Fn::Equals * Fn::If * Fn::Not * Fn::Or Conditions: CreateProdResources: !Equals [ !Ref EnvType, prod]
33
CloudFormation Rollbacks
* Stack Creation Fails: * Default: everything rolls back (gets deleted). We can look at the log * Option to disable rollback and troubleshoot what happened * Stack Update Fails: * The stack automatically rolls back to the previous known working state * Ability to see in the log what happened and error messages
34
ChangeSets
* When you update a stack, you need to know what changes before it happens for greater confidence * ChangeSets won’t say if the update will be successful
35
Nested stacks
* Nested stacks are stacks as part of other stacks * They allow you to isolate repeated patterns / common components in separate stacks and call them from other stacks * Example: * Load Balancer configuration that is re-used * Security Group that is re-used * Nested stacks are considered best practice * To update a nested stack, always update the parent (root stack)
36
CloudFormation – Cross Stacks
* Helpful when stacks have different lifecycles * Use Outputs Export and Fn::ImportValue * When you need to pass export values to many stacks (VPC Id, etc…)
37
CloudFormation – Nested Stacks
* Helpful when components must be re-used * Ex: re-use how to properly configure an Application Load Balancer * The nested stack only is important to the higher level stack (it’s not shared)
38
CloudFormation - StackSets
* Create, update, or delete stacks across multiple accounts and regions with a single operation * Administrator account to create StackSets * Trusted accounts to create, update, delete stack instances from StackSets * When you update a stack set, all associated stack instances are updated throughout all accounts and regions.
39
CloudFormation for DB Specialty Exam
* AWS::RDS::DBInstance https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws- properties-rds-database-instance.html * Important Parameters: * DeleteAutomatedBackups: remove automated backups immediately after the DB instance is deleted. Default value: remove all automated backups * DeletionProtection: enable deletion protection on the DB in RDS. The database can’t be deleted while it has DeletionProtection on. * MasterUsername / MasterPassword: must be provided as text values (we’ll see how to handle that securely later on) * EnableIAMDatabaseAuthentication: can be enabled, does not replace Master username & password * DBClusterIdentifier: (optional) if the DB belongs to an Aurora DB cluster
40
RDS and CloudFormation – DB Updates
* If Update Requires: Replacement: * New DB is created * Links are updated * Old DB is deleted * If a DB instance is deleted or replaced during an update, AWS CloudFormation deletes all automated snapshots. * However, it retains manual DB snapshots * During an update that requires replacement, you can apply a stack policy to prevent DB instances from being replaced
41
RDS and CloudFormation – database update
* Take a snapshot before updating an update. If you don't, you lose the data when AWS CloudFormation replaces your DB instance. To preserve your data: * Deactivate any applications that are using the DB instance so that there's no activity on the DB instance. * Create a snapshot of the DB instance * If you want to restore your instance using a DB snapshot, modify the updated template with your DB instance changes and add the DBSnapshotIdentifier property with the ID of the DB snapshot that you want to use. * After you restore a DB instance with a DBSnapshotIdentifier property, you must specify the same DBSnapshotIdentifier property for any future updates to the DB instance. * Update the stack. * DBSnapshotIdentifier is also useful for mass creating dev/test databases with some data from prod
42
CloudFormation protections
* CloudFormation Stack Termination Protection: can’t delete the stack without removing it (but you can still update the stack) * CloudFormation Stack Policy: prevent updates to CloudFormation templates. * CloudFormation DeletionPolicy: what happens if a resource gets deleted * Delete: delete the resource (default behaviour for most resources) * Retain: do not delete the resources * Snapshot: (only to resources that support snapshot) snapshot then delete * The default CloudFormation behavior for RDS depends on DBClusterIdentifier : * If DBClusterIdentifier not specified (= non Aurora), CloudFormation will snapshot * If DBClusterIdentifier is specified (=Aurora), CloudFormation deletes the DB instance
43
Aurora with CloudFormation
* First create an AWS::RDS::DBCluster * Then add AWS::RDS::DBInstance * Default DeletionPolicy of RDS::DBCluster is Snapshot * Means if individual DBInstance are deleted, no snapshot is made (default) * If the DBCluster is deleted, a snapshot will be made * RDS::DBInstance will inherit configurations
44
CloudFormation Parameters
* Parameters are referenced using !Ref * They can have 4 different sources: * (static) Given using the CLI / Management console * (static) Given using a flat file from local filesystem or S3 * (dynamic) From the SSM Parameter Store * (dynamic) From AWS Secrets Manager * The last two are the most secure way * Exam trap: you cannot pass in KMS encrypted parameters through * Use SSM Parameter Store or Secrets Manager instead!
45
CloudFormation Parameters - SSM
* Retrieve a String (plaintext) or SecureString (KMS encrypted) from SSM Parameter Store at runtime! * Amazing to store environment specific variables * Hierarchical way of storing parameters * SSM CF parameters: will be resolved every time you run the template, does not work for SecureString. * Dynamic parameter pattern: '{{resolve:ssm:parameter-name:version}}’, you must specify the exact version, works for SecureString.
46
CF Parameters – Secrets Manager
* Secrets rotation + integration with RDS = * Dynamic parameter pattern: '{{resolve:ssm:parameter-name:version}}’, you must specify the exact version * Neither Secrets Manager nor CloudFormation logs or persists any resolved secret value * Updating a secret in Secrets Manager does not automatically update the secret in CloudFormation * To manage updating the secret in your template, consider using version-id to specify the version of your secret. * {{resolve:secretsmanager:secret-id:secret- string:json-key:version-stage:version-id}
47
Secrets Manager + RDS
1. Create a AWS::SecretsManager::Secret * Specify GenerateSecretString to randomly generate a secret 2. Create an AWS::RDS::DBInstance that references that secret * Using dynamic references 3. Create a AWS::SecretsManager::SecretTargetAttachment * Link to Secrets Manager secret and the RDS database together * Will add RDS properties to Secret Manager JSON
48
Secrets Manager Rotation Schedule
4. Create a AWS::SecretsManager::RotationSchedule * It will create a Lambda function * You must do this after creating the secret attachment
49
Other CloudFormation concepts
* CloudFormation Drift: * See if the resource configurations have changed from the template over time * Stack Set: * Apply the same CloudFormation template into many accounts / regions * Change Set: * Evaluate the changes that will be applied before applying them (safety)