AWS EC2 Instances and EBS

1.     Introduction

a.     An Amazon EC2 (Elastic Compute Cloud) instance is equivalent to a VM (virtual machine) running in the AWS environment.

b.     EBS (Elastic Block Store) is the virtual block storage that EC2 instances use as virtual disks.

2.     EC2

a.     Creating an instance = Launching. And there are various predefined “sizes” of instances that can be launched.

b.     You are charged per hour while the instance is running and not charged for the instance when it is stopped.

c.      Two major concepts for launching an instance:

                                                              i.      The predefined instance size = Instance Type.

                                                            ii.      The software loaded on the instance = AMI (Amazon Machine Image).

d.     Instance Types.

                                                              i.      Defines the virtual hardware.

                                                            ii.      Predefined sizes vary based on these dimensions.

1.     vCPUs

2.     Memory

3.     Storage (size and type)

4.     Network Performance

                                                          iii.      Each type is grouped in families noting that in each family the ratio of vCPUs to memory scales (instances with larger counts vCPUs have more memory).

                                                         iv.      Families:

1.     C4 = Compute Optimized

2.     r3 = Memory Optimized

3.     i2 = Storage Optimized

4.     g2 = GPU-based instances (for graphics)

5.     Also some instance types (within a family) supports “Enhanced Networking” which provides higher network throughput, lower latency, and less “jitter”.

e.     AMI

                                                              i.      Defines every aspect of the software state at instance launch and is based on either Linux or Windows.

                                                            ii.      Four sources of AMIs:

1.     Published by AWS.

2.     The AWS Marketplace.

3.     Generated from Existing Instances.

4.     Uploaded Virtual Servers, e.g., OVA.

f.       Possible Instance Access Methods:

                                                              i.      Public DNS – auto generated at launch.

                                                            ii.      Public IP – persists while the instance is running and cannot be moved to another instance.

                                                          iii.      Elastic IP – persists even after the instance is stopped and CAN be moved to another instance on demand.

                                                         iv.      Private IP.

                                                           v.      ENI = Elastic Network Interfaces.

g.     Initial Access = Public key/Private Key

h.     Security Groups

                                                              i.      Virtual firewall applied at the instance level.

                                                            ii.      Classic SGs control outgoing traffic only.

                                                          iii.      VPC SGs control outgoing AND incoming traffic.

                                                         iv.      Defaults to deny all and specifies only what is allowed in/out.

                                                           v.      Attributes of an SG are Port, Protocol, Source/Destination.

                                                         vi.      Instances can have multiple SG, and results in the union (aggregation) of all rules apply.

                                                       vii.      Stateful firewall.

i.        Instance Lifecycle

                                                              i.      Launching

1.     Bootstrapping = userdata

2.     VM Import/Export

                                                            ii.      Instance Metadata (SGs, ID, type, AMI ...) can be probed from instance using the following address (note that this is the root to a tree):

                                                          iii.      Managing Instances:

1.     Tags = Key/Value.

2.     Monitoring using Cloudwatch.

                                                         iv.      Modifying an Instance

1.     Resize by stopping, change instance type, and restarting.

2.     Security groups (for VPC instances only) can be added/removed dynamically (must always have one SG).

3.     Termination Protection.

j.       Options

                                                              i.      Pricing Options

1.     On-Demand Instances = published price per hour running.

2.     Reserved Instances (costs are 50% - 70% cheaper than On-Demand for the chosen term) = term commitment for 1 or 3 years (can be sold on the Marketplace) and applies to an AZ.

a.     Payment for RIs:

                                                                                                                                      i.      All Upfront

                                                                                                                                    ii.      Partial Upfront

                                                                                                                                  iii.      No Upfront

b.     Modifying an RI

                                                                                                                                      i.      Can switch AZs in the same region.

                                                                                                                                    ii.      Change between VPC and Classic.

                                                                                                                                  iii.      Can change instance type within the same family (Linux Only), e.g., An m4.2xlarge can be changed to two m4.xlarge.

3.     Spot Instances = bidding for an instance type.

a.     Once launched they run until one of the following occurs:

                                                                                                                                      i.      They are terminated.

                                                                                                                                    ii.      The spot price goes above the big price.

                                                                                                                                  iii.      There is not enough unused capacity to meet the demand.

b.     Two minute warning before termination.

4.     How to take advantage of the pricing options to maximize cost-efficiencies.

                                                            ii.      Tenancy Options (each below options increases in costs).

1.     Shared Tenancy (default) = instance is running on shared hardware.

2.     Dedicated Instances = instance is running on hardware dedicated to a single customer.

3.     Dedicated Host = allows customer to specify which host runs an instance.

4.     Tenancy is determined at instance launch and cannot be changed.

                                                          iii.      Placement Groups = a logical grouping of instances within a single AZ enabling them to participate in a 10 Gbps network.

                                                         iv.      Instance Stores = Ephemeral Storage .

1.     Additional, Block level storage (essentially local SSD or HDD).

2.     Low durability, if the drive fails, the data is lost.

3.     Size and type are predefined and based on the instance type (included in the price).

4.     Data on Ephemeral Storage does not persist after stop/start or termination.

3.     Amazon Elastic Block Store (EBS)

a.     EBS Basics

                                                              i.      Replicated within AZ to protect from component failure.

                                                            ii.      Can only be attached to a single instance at one time.

                                                          iii.      Billed by the amount provisioned, not the data stored.

b.     Types

                                                              i.      Magnetic Volumes

1.     lowest cost per gigabyte.

2.     Range from 1 GB to 1 TB.

3.     Average performance of 100 IOPS with burst capabilities.

4.     Suitable for infrequent access, sequential access.

                                                            ii.      General Purpose SSD

1.     Strong performance at a moderate price point.

2.     Range from 1 GB to 16 TB.

3.     Baseline performance of 3 IOPS per provisioned gigabyte capping at 10,000 IOPS.

4.     Volumes under 1 TB can burst up to 3,000 IOPS for periods of time.

5.     Suitable for boot volumes, small to medium size DBs, dev and test environments.

                                                          iii.      Provisioned IOPS SSD

1.     Designed to meet I/O intensive workloads.

2.     Most expensive.

3.     Range from 4 GB to 16 TB.

4.     Specify both size and desired IOPS from 30X the size of the volume to 20,000 IOPS.

5.     Delivers within 10% of the desired performance 99.9% of the time.

6.     Pricing based on combination of size and IOPS reservation.

                                                         iv.      Note that there are also Throughput-Optimized HDD and Cold HDD volumes.

c.      EBS-Optimized Instances
Uses an optimized configuration stack and provides additional, dedicated capacity for best disk I/O.

4.     Protecting Data      Snapshots

a.     Taking Snapshots

                                                              i.      They are incremental point-in-time backups.

                                                            ii.      Can be taken via the console, cli, and API (or via a schedule?)

                                                          iii.      Snapshot data stored in S3, but not in a bucket you have access to.

                                                         iv.      The time of snapshots request is the point-in-time, but may be pending until blocks copied to S3.

                                                           v.      Snapshots are “free” to take. You pay for the space consumed.

                                                         vi.      Snapshots are constrained to the region, but can be copied to other regions.

                                                       vii.      No capability to take consistent point-in-time snapshots of multiple volumes.

b.     Creating Volumes from Snapshots

                                                              i.      Volume created immediately, but is lazily loaded.

                                                            ii.      Best practice to initialize volume after creation from snapshot.

                                                          iii.      Can be used to increase the size of an EBS volume (but new dynamic EBS feature makes this unnecessary)

                                                         iv.      Can recover a volume by attaching created volume from snapshot to a running instance

c.      Encryption Options

                                                              i.      EBS volumes can be encrypted and use AES-256 encryption.

                                                            ii.      Encryption occurs on the servers hosting the EC2 instances.

                                                          iii.      Expect the same IOPS performance as non-encrypted volumes with some latency.