Storage | Amazon Elastic File System (EFS) Flashcards by Parri Pandian

What is Amazon Elastic File System?

General

Amazon Elastic File System (EFS) | Storage

Amazon EFS is a fully-managed service that makes it easy to set up and scale file storage in the Amazon cloud. With a few clicks in the AWS Management Console, you can create file systems that are accessible to Amazon EC2 instances via a file system interface (using standard operating system file I/O APIs) and that support full file system access semantics (such as strong consistency and file locking).

Amazon EFS file systems can automatically scale from gigabytes to petabytes of data without needing to provision storage. Tens, hundreds, or even thousands of Amazon EC2 instances can access an Amazon EFS file system at the same time, and Amazon EFS provides consistent performance to each Amazon EC2 instance. Amazon EFS is designed to be highly durable and highly available. With Amazon EFS, there is no minimum fee or setup costs, and you pay only for the storage you use.

How well did you know this?

Not at all

Perfectly

What use cases is Amazon EFS intended for?

General

Amazon Elastic File System (EFS) | Storage

Amazon EFS is designed to provide performance for a broad spectrum of workloads and applications, including Big Data and analytics, media processing workflows, content management, web serving, and home directories.

How well did you know this?

Not at all

Perfectly

When should I use Amazon EFS vs. Amazon Simple Storage Service (S3) vs. Amazon Elastic Block Store (EBS)?

General

Amazon Elastic File System (EFS) | Storage

Amazon Web Services (AWS) offers cloud storage services to support a wide range of storage workloads.

Amazon EFS is a file storage service for use with Amazon EC2. Amazon EFS provides a file system interface, file system access semantics (such as strong consistency and file locking), and concurrently-accessible storage for up to thousands of Amazon EC2 instances.

Amazon EBS is a block level storage service for use with Amazon EC2. Amazon EBS can deliver performance for workloads that require the lowest-latency access to data from a single EC2 instance.

Amazon S3 is an object storage service. Amazon S3 makes data available through an Internet API that can be accessed anywhere.

How well did you know this?

Not at all

Perfectly

What regions is Amazon EFS currently available in?

General

Amazon Elastic File System (EFS) | Storage

Please refer to Regional Products and Services for details of Amazon EFS service availability by region.

How well did you know this?

Not at all

Perfectly

How do I get started using Amazon EFS?

General

Amazon Elastic File System (EFS) | Storage

To use Amazon EFS, you must have an Amazon Web Services account. If you do not already have an AWS account, you can create one by clicking the “Try the Free Tier” button on the Amazon EFS detail page.

Once you have created an AWS account, please refer to the Amazon EFS Getting Started guide to begin using Amazon EFS. You can create a file system via the AWS Management Console, the AWS Command Line Interface (AWS CLI), and Amazon EFS API (and various language-specific SDKs).

How well did you know this?

Not at all

Perfectly

How do I access a file system from an Amazon EC2 instance?

General

Amazon Elastic File System (EFS) | Storage

To access your file system, you mount the file system on an Amazon EC2 Linux-based instance using the standard Linux mount command and the file system’s DNS name. Once you’ve mounted, you can work with the files and directories in your file system just like you would with a local file system.

Amazon EFS uses the NFSv4.1 protocol. For a step-by-step example of how to access a file system from an Amazon EC2 instance, please see the Amazon EFS Getting Started guide.

How well did you know this?

Not at all

Perfectly

What Amazon EC2 instance types and AMIs work with Amazon EFS?

General

Amazon Elastic File System (EFS) | Storage

Amazon EFS is compatible with all Linux-based AMIs for Amazon EC2. You can mix and match the instance types connected to a single file system. For a step-by-step example of how to access a file system from an Amazon EC2 instance, please see the Amazon EFS Getting Started guide.

How well did you know this?

Not at all

Perfectly

How do I manage a file system?

General

Amazon Elastic File System (EFS) | Storage

Amazon EFS is a fully-managed service, so all of the file storage infrastructure is managed for you. When you use Amazon EFS, you avoid the complexity of deploying and maintaining complex file system infrastructure. An Amazon EFS file system grows and shrinks automatically as you add and remove files, so you do not need to manage storage procurement or provisioning.

You can administer a file system via the AWS Management Console, the AWS command-line interface (CLI), or the Amazon EFS API (and various language-specific SDKs). The Console, API, and SDK provide the ability to create and delete file systems, configure how file systems are accessed, create and edit file system tags, and display detailed information about file systems.

How well did you know this?

Not at all

Perfectly

How do I load data into a file system?

Data Protection and Availability

Amazon Elastic File System (EFS) | Storage

There are a number of methods for loading existing file system data into Amazon EFS, whether your existing file system data is located in AWS or in your on-premises servers.

Amazon EFS file systems can be mounted on an Amazon EC2 instance, so any data that is accessible to an Amazon EC2 instance can also be read and written to Amazon EFS. To load file data that is not currently stored in AWS, you can use EFS File Sync to copy data directly to Amazon EFS.

For on-premises file systems, EFS File Sync provides a fast and simple way to securely sync existing file systems into Amazon EFS. EFS File Sync works over any network connection, including with AWS Direct Connect. AWS Direct Connect provides a high bandwidth and lower latency dedicated network connection, over which you can mount your EFS filesystems. You can also use standard Linux copy tools to move data files to Amazon EFS.

For more information about accessing a file system from an on-premises server, please see the On-premises Access section of this FAQ.

For more information about moving data to the Amazon cloud, please see the Cloud Data Migration page.

How well did you know this?

Not at all

Perfectly

How is Amazon EFS designed to provide high durability and availability?

Data Protection and Availability

Amazon Elastic File System (EFS) | Storage

Every file system object (i.e. directory, file, and link) is redundantly stored across multiple Availability Zones. In addition, a file system can be accessed concurrently from all Availability Zones in the region where it is located, which means that you can architect your application to failover from one AZ to other AZs in the region in order to ensure the highest level of application availability. Mount targets themselves are designed to be highly available.

How well did you know this?

Not at all

Perfectly

How do I back up a file system?

Data Protection and Availability

Amazon Elastic File System (EFS) | Storage

Amazon EFS is designed to be highly durable. Using the EFS-to-EFS Backup solution, you can schedule automatic incremental backups of your Amazon EFS file system. For more information, please see the Amazon EFS Walkthrough: Backup Solutions for Amazon EFS File Systems.

How well did you know this?

Not at all

Perfectly

How do I access my file system from outside my VPC?

Scale and Performance

Amazon Elastic File System (EFS) | Storage

Amazon EC2 instances within your VPC can access your file system directly, and Amazon EC2 Classic instances outside your VPC can mount a file system via ClassicLink. On-premises servers can mount your file systems via an AWS Direct Connect connection to your VPC.

How well did you know this?

Not at all

Perfectly

How much data can I store?

Scale and Performance

Amazon Elastic File System (EFS) | Storage

Amazon EFS file systems can store petabytes of data. Amazon EFS file systems are elastic, and automatically grow and shrink as you add and remove files. You do not provision file system size or specify a size up front and you pay only for the storage you use.

How well did you know this?

Not at all

Perfectly

How many Amazon EC2 instances can connect to a file system?

Scale and Performance

Amazon Elastic File System (EFS) | Storage

Amazon EFS supports one to thousands of Amazon EC2 instances connecting to a file system concurrently.

How well did you know this?

Not at all

Perfectly

How many file systems can I create?

Scale and Performance

Amazon Elastic File System (EFS) | Storage

By default, you can create up to 10 file systems per AWS account per region. You can request to increase your file system limit by visiting AWS Service Limits.

How well did you know this?

Not at all

Perfectly

How does Amazon EFS performance compare to that of other storage solutions?

Scale and Performance

Amazon Elastic File System (EFS) | Storage

Amazon EFS file systems are distributed across an unconstrained number of storage servers, enabling file systems to grow elastically to petabyte-scale and allowing massively parallel access from Amazon EC2 instances to your data. Amazon EFS’s distributed design avoids the bottlenecks and constraints inherent to traditional file servers.

This distributed data storage design means that multi-threaded applications and applications that concurrently access data from multiple Amazon EC2 instances can drive substantial levels of aggregate throughput and IOPS. Big Data and analytics workloads, media processing workflows, content management and web serving are examples of these applications.

The table below compares high-level performance and storage characteristics for Amazon’s file and block cloud storage offerings.

Amazon EFS

Per-operation latency: Low, consistent

Throughput scale: Multiple GBs per second

Amazon EBS

PIOPS

Per-operation latency: Low, consistent

Throughput scale: Single GB per second

Amazon EFS’s distributed nature enables high levels of availability, durability, and scalability. This distributed architecture results in a small latency overhead for each file operation. Due to this per-operation latency, overall throughput generally increases as the average I/O size increases, since the overhead is amortized over a larger amount of data. Amazon EFS’s support for highly parallelized workloads (i.e. with consistent operations from multiple threads and multiple EC2 instances) enables high levels of aggregate throughput and IOPS.

How well did you know this?

Not at all

Perfectly

What’s the difference between “General Purpose” and “Max I/O” performance modes? Which one should I choose?

Scale and Performance

Amazon Elastic File System (EFS) | Storage

“General Purpose” performance mode is appropriate for most file systems, and is the mode selected by default when you create a file system. “Max I/O” performance mode is optimized for applications where tens, hundreds, or thousands of EC2 instances are accessing the file system — it scales to higher levels of aggregate throughput and operations per second with a tradeoff of slightly higher latencies for file operations. For more information, please see the documentation on File System Performance.

How much throughput can a file system support?

Access Control

Amazon Elastic File System (EFS) | Storage

The throughput available to a file system scales as a file system grows. Because file-based workloads are typically spiky – requiring high levels of throughput for periods of time and lower levels of throughput the rest of the time – Amazon EFS is designed to burst to allow high throughput levels for periods of time. All file systems deliver a consistent baseline performance of 50 MB/s per TB of storage, all file systems (regardless of size) can burst to 100 MB/s, and file systems larger than 1TB can burst to 100 MB/s per TB of storage. As you add data to your file system, the maximum throughput available to the file system scales linearly and automatically with your storage.

File system throughput is shared across all Amazon EC2 instances connected to a file system. For example, a 1TB file system that can burst to 100MB/s of throughput can drive 100MB/s from a single Amazon EC2 instance, or 10 Amazon EC2 instances can collectively drive 100MB/s. For more information, please see the documentation on File System Performance.

How do I control which Amazon EC2 instances can access my file system?

Access Control

Amazon Elastic File System (EFS) | Storage

When you create a file system, you create endpoints in your VPC called “mount targets.” When mounting from an EC2 instance, your file system’s DNS name, which you provide in your mount command, resolves to a mount target’s IP address. Only resources that can access a mount target can access your file system. You can control the network traffic to and from your file system mount targets using VPC security groups.

How do I control who can access my file system?

Encryption

Amazon Elastic File System (EFS) | Storage

You can control who can administer your file system using AWS Identity and Access Management (IAM). You can control access to files and directories with POSIX-compliant user and group-level permissions.

What is Amazon EFS Encryption?

Encryption

Amazon Elastic File System (EFS) | Storage

Amazon EFS seamlessly offers encryption of EFS file systems. Data is transparently encrypted while being written, and transparently decrypted while being read, so you don’t have to modify your applications. Encryption keys are managed by the AWS Key Management Service (KMS), eliminating the need to build and maintain a secure key management infrastructure.

What is the AWS Key Management Service (KMS)?

Encryption

Amazon Elastic File System (EFS) | Storage

AWS KMS is a managed service that makes it easy for you to create and control the encryption keys used to encrypt your data. AWS Key Management Service is integrated with AWS services including Amazon EFS, Amazon EBS, and Amazon S3, to make it simple to encrypt your data with encryption keys that you manage. AWS Key Management Service is also integrated with AWS CloudTrail to provide you with logs of all key usage to help meet your regulatory and compliance needs. To learn more about KMS, visit the AWS Key Management Service product page.

How do I enable encryption for my Amazon EFS file system?

Encryption

Amazon Elastic File System (EFS) | Storage

You can enable encryption for your EFS file system in the EFS console, or by using the AWS CLI or SDKs. When creating a new file system in the EFS console, click “Create File System” and click the checkbox to enable encryption. For more details, see the user documentation on Encryption.

Does encryption impact Amazon EFS performance?

On-premises Access

Amazon Elastic File System (EFS) | Storage

Encrypting your data has a minimal effect on I/O latency and throughput.

How do I access an EFS file system from servers in my on-premises datacenter? On-premises Access Amazon Elastic File System (EFS) | Storage

To access EFS file systems from on-premises, you must have an AWS Direct Connect connection between your on-premises datacenter and your Amazon VPC. AWS Direct Connect establishes a private network connection between your on-premises datacenter and AWS. If you do not already have an AWS Direct Connect connection, you can create one by following the instructions in the AWS Direct Connect user guide. You mount an EFS file system on your on-premises Linux server using the standard Linux mount command for mounting a file system via the NFSv4.1 protocol. For more information about accessing EFS file systems from on-premises servers via AWS Direct Connect, please see the documentation.

What can I do by enabling access to my EFS file systems from my on-premises servers? On-premises Access Amazon Elastic File System (EFS) | Storage

You can mount your Amazon EFS file systems on your on-premises servers, and move file data to and from Amazon EFS using standard Linux tools and scripts. The ability to move file data to and from Amazon EFS file systems enables three use cases. First, you can migrate data from on-premises datacenters to permanently reside in Amazon EFS file systems. Second, you can support cloud bursting workloads to offload your application processing to the cloud. You can move data from your on-premises servers into your EFS file systems, analyze it on a cluster of EC2 instances in your Amazon VPC, and store the results permanently in your EFS file systems or move the results back to your on-premises servers. Third, you can periodically copy your on-premises file data to EFS to support backup and disaster recovery scenarios.

Can I use an AWS VPN to access an EFS file system from on-premises? On-premises Access Amazon Elastic File System (EFS) | Storage

No, Amazon EFS does not support access over AWS VPN.

Can I access my Amazon EFS file system concurrently from my on-premises datacenter servers as well as Amazon EC2 instances? On-premises Access Amazon Elastic File System (EFS) | Storage

Yes, you can access your Amazon EFS file system concurrently from servers in your on-premises datacenter as well as Amazon EC2 instances in your Amazon VPC. Amazon EFS provides the same file system access semantics, such as strong data consistency and file locking, across all EC2 instances and on-premises servers accessing a file system.

What is the recommended best practice when moving file data to and from on-premises servers via AWS Direct Connect? On-premises Access Amazon Elastic File System (EFS) | Storage

Because of the propagation delay tied to data traveling over long distances, the network latency of a Direct Connect connection between your on-premises datacenter and your Amazon VPC can be tens of milliseconds. If your file operations are serialized, the latency of the Direct Connect connection directly impacts your read and write throughput; in essence, the volume of data you can read or write during a period of time is bounded by the amount of time it takes for each read and write operation to complete. To maximize your throughput, parallelize your file operations so that multiple reads and writes are processed by EFS concurrently. Standard tools like GNU parallel enable you to parallelize the copying of file data. For more information, see the online documentation.

How do I copy existing data from on-premises file storage to Amazon EFS? EFS File Sync Amazon Elastic File System (EFS) | Storage

There are a number of methods to copy existing on-premises data into Amazon EFS. EFS File Sync provides a fast and simple way to securely sync existing file systems into Amazon EFS, and works over any network, including AWS Direct Connect AWS Direct Connect provides a high bandwidth and lower latency dedicated network connection over which you can mount your Amazon EFS filesystems. Once mounted, you can use EFS File Sync to copy data into Amazon EFS up to 5x faster than standard Linux copy tools. For more information on EFS File Sync, please see the EFS File Sync section of this FAQ.

What is EFS File Sync? EFS File Sync Amazon Elastic File System (EFS) | Storage

EFS File Sync provides a fast and simple way to securely move data from existing on-premises or in-cloud file systems into Amazon EFS file systems. EFS File Sync copies files and directories into Amazon EFS at speeds up to 5x faster than standard Linux copy tools, with simple setup and management in the AWS Console. When syncing on-premises or in-cloud file systems into Amazon EFS, EFS File Sync delivers fast parallel data transfer, encrypts data in transit, and ensures the integrity of the copied data. Visit our documentation site for EFS File Sync to learn about the technical details and get started.

How do I get started using EFS File Sync? EFS File Sync Amazon Elastic File System (EFS) | Storage

To use EFS File Sync, visit the Amazon EFS Console to download and deploy File Sync agent into your IT environment. Configure the source and destination file systems, start the sync task, and monitor progress through the console or using AWS CloudWatch. When your source file system is on-premises the agent is deployed in your data center, and when the source file system is in AWS the agent is deployed as an Amazon EC2 instance. Standard rates apply for this EC2 instance. For more information on getting started, read the documentation section.

How is my data protected using EFS File Sync? EFS File Sync Amazon Elastic File System (EFS) | Storage

All data transferred by EFS File Sync from the source file systems to the destination in Amazon EFS is encrypted via SSL/TLS.

Does EFS File Sync support copying data between AWS Regions? EFS File Sync Amazon Elastic File System (EFS) | Storage

Yes. You can deploy an EFS File Sync agent in one AWS Region, and sync to an EFS file system in another AWS Region. Your source file system can be either an EFS file system, or a file system shared from an EC2 instance. For more information on setting up EFS File Sync, read the documentation section.

In what regions is EFS File Sync available? Compatibility Amazon Elastic File System (EFS) | Storage

EFS File Sync is available in all regions where Amazon EFS is available.

What interoperability and compatibility is there between existing AWS services and Amazon EFS? Compatibility Amazon Elastic File System (EFS) | Storage

Amazon EFS is integrated with a number of other AWS services, including Amazon CloudWatch, AWS CloudFormation, AWS CloudTrail, AWS IAM, and AWS Tagging services. Amazon CloudWatch allows you to monitor file system activity using metrics. AWS CloudFormation allows you to create and manage file systems using templates. AWS CloudTrail allows you to record all Amazon EFS API calls in log files. AWS Identity and Access Management (IAM) allows you to control who can administer your file system. AWS Tagging services allows you to label your file systems with metadata that you define.

What type of locking does Amazon EFS support? Compatibility Amazon Elastic File System (EFS) | Storage

Locking in Amazon EFS follows the NFSv4.1 protocol for advisory locking, and enables your applications to use both whole file and byte range locks.

Are file system names global (like Amazon S3 bucket names)? Pricing and Billing Amazon Elastic File System (EFS) | Storage

Every file system has an automatically generated ID number that is globally unique. You can tag your file system with a name, and these names do not need to be unique.

How much does Amazon EFS cost? Pricing and Billing Amazon Elastic File System (EFS) | Storage

With Amazon EFS, you pay only for the amount of file system storage you use per month in GB. There is no minimum fee and no set-up costs. There are no additional costs for bandwidth or requests. For Amazon EFS pricing information, please visit the pricing section on the Amazon EFS Pricing page.

How much does Amazon EFS File Sync cost? Pricing and Billing Amazon Elastic File System (EFS) | Storage

With Amazon EFS File Sync, you pay per-GB for data copied to Amazon EFS. For EFS File Sync pricing information, please visit the Amazon EFS File Sync Pricing page.