Oracle on AWS white papers

Some great new Oracle on AWS white papers:

      Best Practices for running Oracle Database on Amazon Web Services
http://d0.awsstatic.com/whitepapers/best-practices-for-running-oracle-database-on-aws.pdf
Advanced Architectures for Oracle Database on Amazon EC2
http://d3ulk6ur3a3ha.cloudfront.net/enterprise-marketing/Oracle/AWSAdvancedArchitecturesforOracleDBonEC2.pdf
Strategies for Migrating Oracle Database to AWS
http://d0.awsstatic.com/whitepapers/strategies-for-migrating-oracle-database-to-aws.pdf
Determining the IOPS Needs for Oracle Database on AWS
http://d0.awsstatic.com/whitepapers/determining-iops-needs-for-oracle-database-on-aws.pdf
Choosing the Operating System for Oracle Workloads on Amazon EC2
http://d0.awsstatic.com/whitepapers/choosing-os-for-oracle-workloads-on-ec2.pdf

AWS EBS Performance

Here is a very good presentation on Amazon EBS performance from reInvent in 2013:
http://www.slideshare.net/AmazonWebServices/ebs-webinarfinal

AWS PIOPS SSD backed

The question of what the technology backing EBS Provisioned IOPS volumes comes up often.  PIOPS volumes are backed by Solid-State Drives (SSDs), Provisioned IOPS volumes support up to 30 IOPS per GB which enables you to provision 4000 IOPS on a volume as small as 134 GB. 

More details here:
https://aws.amazon.com/ebs/details/

Oracle Database AMIs

An easy way to Bring Your Own License (BYOL) to AWS and start running the Oracle Database is to utilize the AWS Marketplace AMIs.  There are six AMIs for Oracle 12c and 11g.  Standard Edition One, Standard Edition and Enterprise Edition versions are available for both 12c and 11g. All these Marketplace AMIs, use PIOPS with a provisioning of 500 IOPS.  The OS system is Red Hat Enterprise Linux 6.5. The AWS Marketplace AMIs can be found here:

https://aws.amazon.com/marketplace/search/results/ref=sp_navgno_search_box?page=1&searchTerms=oraclempbyol

Oracle Database on SSD

New options for running the Oracle Database on EC2 using SSD storage for your database. The I2 instances feature the latest generation of Intel Ivy Bridge processors - each virtual CPU (vCPU) is a hardware hyperthread from an Intel Xeon E5-2670 v2 (Ivy Bridge) processor. I2 instances are available in four sizes as listed in the table below. Instance Type vCPUs ECU Rating Memory (GiB) Instance Storage SSD (GB) Note           i2.xlarge 4 14 30.5 1 x 800 800 GB of SSD i2.2xlarge 8 27 61 2 x 800 1.6 TB of SSD i2.4xlarge 16 53 122 4 x 800 3.2 TB of SSD. hi.4xlarge with 2 TB  of SSD was previously largest SSD instance type i2.8xlarge 32 104 244 8 x 800 6.4 TB of SSD The i2.8xlarge instance size is capable of providing over 365,000 4 kilobyte (KB) random read IOPS and over 315,000 4 KB random write IOPS.   Great for Oracle databases that have high IO requirements.  This document has more on configuring Oracle Databases on EC2 and using SSD based instances : http://cloudconclave.blogspot.com/2013/11/aws-database-reference-implementation.html

Striping volumes for an Oracle Database

Often times you will want to stripe your Oracle database volumes for better performance.  Here are the steps for creating the EBS volumes and creating the Linux logical volume:

1. aws ec2 create-volume --availability-zone us-west-2b --size 100

2. aws ec2 create-volume --availability-zone us-west-2b --size 100

3. aws ec2 describe-volumes --volume-ids vol-7c2e9955 vol-2e2e9907

4. aws ec2 attach-volume --volume-id vol-7c2e9955 --instance-id i-f0ef69c6 --device /dev/sdc

5. aws ec2 attach-volume --volume-id vol-2e2e9907 --instance-id i-f0ef69c6 --device /dev/sdd

6. aws ec2 start-instances --instance-id i-NewDBServer

7. ssh -i YourFile.pem ec2-user@10.50.x.x

Once you are logged into the EC2 instance,  stripe and then recognize your additional volumes with this series of commands:

1. sudo mkdir /mnt/oraclestripe

2. sudo vgcreate vg /dev/sdc /dev/sdd

3. lvcreate -L 200 -n lv -i 2 vg

(Notes:

1. Using default stripesize 64.00 KiB

2. The parameter -i in the lvcreate command specifies the number of physical volumes to stripe across. If you wanted to stripe across more than 2 drives, you would have to specify that number.

)

4. sudo mkfs -t ext3 /dev/vg/lv

5. sudo mount /dev/vg/lv /mnt/oraclestripe

6. Next, update /etc/fstab to ensure these mounted drives are automatically mounted on reboot.

A. sudo vi /etc/fstab and add these lines:
/dev/sdb /mnt/piops1 ext3 defaults 0 2
/dev/vg/lv /mnt/oraclestripe ext3 defaults 0 2

7. df -h (to view the new volumes mounted to the logical volumes created)

Creating PIOPS volume for Oracle Database

When running your Oracle Database on AWS EC2, you may find that you need additional IOPS for your database and which to add an EBS volume to your Oracle instances for some of your highly accessed tables that require high IO.  Here are the steps to create a new PIOPS volume and attached it your EC2 instance.

1. aws ec2 stop-instances --instance-id i-NewDBServer
2. aws ec2 create-volume --availability-zone az-asdf-1a --size 200 --volume-type io1 --iops 2000

3. aws ec2 attach-volume --volume-id vol-FastDrive --instance-id i-NewDBServer - -device /dev/xvdb

Get the OS in order for your EC2 instance to recognize your volume by running the following commands:

1. sudo mkdir /mnt/piops1

2. sudo mkfs.ext3 /dev/xvdb

3. sudo mount /dev/xvdb /mnt/piops1

4. df -h (to see the newly mounted piops volume)

You would then create an Oracle tablespace on this new volume and move your tables to this new tablespace.

EC2 EBS optimized instances throughput

EC2 EBS-Optimized Instances are listed here:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html

It is important to understand the maximum throughput in MB/s and maximum 16 KB IOPS when architecting Oracle databases on AWS.  

Network throughput for an Oracle database : Smaller number of large PIOPS volumes or large number of smaller PIOPS volumes

Individual 4000 PIOP volumes at 16K  I/O's can theoretically transfer 64MB/s.  Smaller PIOP volumes (e.g 1000) can burst up to 40MB/s.   In this case, especially for bursty workloads, it may be more cost effective to use smaller PIOPS volumes (but more of them) in a raid0 configuration, than to use fewer higher rated PIOPS volumes.

Storage tiering on AWS

Here is the replay of the session I just presented with an AWS partner (App Associates) and customer (Riso):http://cloudconclave.blogspot.com/2013/11/aws-storage-tiering-for-enterprise.html

I opened the session by 'testing' the audiences understanding of AWS storage tiers:
1. Assuming a 16K block size, what storage option provides average throughput of 1 to 2 MBPS ?
2. What storage option has single threaded through put of around 17 MBPS ?
3. Assuming a 16K block size, what EBS volume provides an average of 16-20 MBPS through put ?
4. Once again assuming 16K block size and also assuming 4 1K PIOPS volumes, for what EC2 instances will you start to see network saturation ?
5. What storage option produces approximately 100-145 MBPS read and write through put?
6. For which storage option is it possible to transfer approximately 3 TB a day a day over a WAN ?




















Answers:
1. Standard IOPS
2. S3
3. 1K PIOPS
4. Any instance with .5 Gpbs network connection. For example, m2.2xlarge
5. Hi1.4xlarge (high IO) ephemeral storage
6. AWS Storage gateway

Oracle Database size and network throughput : IOPS and network throughput

When configuring an Oracle Database on AWS EC2, you need to consider both storage (EBS) IOPS and network throughput.   With PIOPS, you can achieve up to 4K IOPS per EBS volumes.  However, the EC2 instances (assuming EBS optimized or 10 Gbps cluster compute) can be a potential bottle neck.  For example, the m2.2xlarge instance type has a maximum throughput of 0.5Gb.  Which means this instance type is limited to approximately 3750 * 16 KiB IOPS. Therefore, one 4K PIOPS volume would start to saturate the network.   Take into account the IOPS and instance network throughput when designing your Oracle Database on AWS EC2.

HPC for high IO

Enabled by Cluster family of EC2 instances, which includes:

• Cluster Compute instances: cc1.4xlarge,cc2.8xlarge
• Cluster GPU instances : cg1.4xlarge
• New c3.large, c3.xlarge, c3.2xlarge, c3.4xlarge and c3.8xlarge

Attributes of these instances:

• 10 Gbps Ethernet
• Latency and bandwidth about 10X better than “EC2 Classic” 
• Uses jumbo frames between cluster instances
• New EC2 API: Placement Groups. A placement group is a logical cluster 

Because HPC instance types have 10 Gbps networks, using 20 volumes attached to an HPC instance you can achieve in the neighborhood of 100 IOPS using 4K PIOPS volumes.

http://www.awsps.com.s3.amazonaws.com/training/computefest/AWS-HPC-BigData-Computefest.pdf

AWS Storage Tiering for Enterprise Workloads

This a replay of the AWS reInvent session that covers storage tiering options for running Oracle Enterprise workloads such as Oracle Databases on AWS:

http://www.youtube.com/watch?v=i5Il80bhQCI

Slides:
http://www.slideshare.net/tomlaszewski/aws-storage-tiering-for-enterprise-workloads

AWS EBS PIOPS : block size and IOPS

Having spent more time in the database world than in the web development world, I am accustomed to measuring (database) performance/through put in terms of IOPS or TPS.  The web/video/image world like to use MB/sec.  Why I am saying this? Because it relates to the conversation about getting a certain level of PIOPS (based upon a 16 KB block) on AWS EBS and how this effects MB/sec.  MB/sec, I am beginning to understand, and maybe move to the 'dark side', is the ultimate measure of disk 'performance'.  

Example: A 2000 Provisioned IOPS volume can handle:

•2000 16KB read/write per second, or 1000 32KB read/write per second, or 500 64KB read/write per second 

•You will get consistent 32 MB/sec throughput (with 16KB or higher IOs)

•Perform an index creation action and sends I/O of 32K, IOPS becomes 1000, you still get 32MB/sec throughput

•On best effort, you may get up to 40 MB/sec throughput 

So, you may be better off using a 64 KB block size but your PIOPS will show up as lower but your MB/sec could be better.

AWS storage performance

A common question is "How would the client compare the performance of storage on Amazon vs. local data center? "  Well here are some numbers for AWS storage:

a.   Instance storage Disk : Equivalent to local disk for a server

b.   Instance storage SSD

                                                             i.     ~120,000 random read IOPS (4 KB blocks)

                                                            ii.     ~10,000-85,000 random write IOPS (4 KB blocks)

c.    EBS Standard (100 IOPS)

                                                             i.     Reads typically <20ms writes typically <10ms

                                                            ii.     Best effort to 10’s of MB/sec

d.   EBS PIOPS (4K IOPS),

                                                             i.     On best effort, you may get up to 40 MB/sec throughput for 2K PIOPS disk

e.   S3 : loading :http://cloudconclave.blogspot.com/2013/05/aws-s3-load-throughput.html

f.     Glacier : Long time archiving. Takes 3-5 hours to retrieve.

g.   Storage Gateway : On premise to AWS S3 ‘replication’.  So, will depend on the pipe (internet, AWS DirectConnect) from on premise to AWS.

AWS Storage options, cost, and SLA

A common question when moving from on premise to AWS is what are the storage options
 and how much they will cost.  A great place to calculate the cost is the Simple Monthly Calculator: http://calculator.s3.amazonaws.com/calc5.html. 

Before you can calculate the cost you need to be aware of the options, cost of each option, when to use it, and SLAs:

a.   EBS: http://aws.amazon.com/pricing/ec2/ : Data Transfer, EBS standard and EBS PIOPS. Sample pricing:

                                                             i.     Data Transfer, EBS Standard, EBS PIOPS : http://aws.amazon.com/pricing/ebs/

                                                            ii.     Price is for allocated storage

                                                          iii.     SLA : annual failure rate (AFR) of between 0.1% – 0.5%, where failure refers to a complete loss of the volume

                                                          iv.     Common use cases: RDBMS (PIOPS), Application Server files (IOPS)

b.   S3 : http://aws.amazon.com/pricing/s3/

                                                             i.     First TB per month of storage: .095

                                                            ii.     Data transfer : Up to 10 TB .120 per GB + request prices (.005 per 1K requests)

                                                          iii.     Price is for used storage

                                                          iv.     SLA: 99.999999999% durability, 99.99% availability

                                                           v.     Common use cases: backups, multi media content, web logs, EMR jobs

c.    Glacier: http://aws.amazon.com/glacier/pricing/

                                                             i.     First TB per month of storage: .01

                                                            ii.     Price is for used storage

                                                          iii.     SLA: 99.999999999% durability

                                                          iv.     Common use cases: archiving, long term storage

d.   Instance storage : Included in price of instance

                                                             i.     Common use cases: web server files (instance disk storage), data warehouses (Redshift runs on instance storage), RDBMS (instance SSD..with redundancy), temporary files

AWS EBS PIOPS increased to 4K

This announcement will provide additional throughput for high transaction volume Oracle databases running on EC2:

• Jeff Barr's Blog post: http://aws.typepad.com/aws/2013/05/provision-up-to-4k-iops-per-ebs-volume.html
• What's new announcement: https://aws.amazon.com/about-aws/whats-new/2013/05/07/announcing-4000-iops-per-piops-volume-and-marketplace-support/
• EBS detail page: http://aws.amazon.com/ebs/
• AWS Marketplace Provisioned IOPS detail page: https://aws.amazon.com/marketplace/piops
• EBS Provisioned IOPS Documentation: http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AmazonEBS.html

SSD for Oracle Databases

Here are a couple of articles/web site with the pros and cons of running your Oracle database on SSD:

http://www.pythian.com/blog/de-confusing-ssd-for-oracle-databases/
http://www.slideshare.net/gwenshap/ssd-collab13

If your Oracle database is less than 2 TB you could run it on the  hi1.4xlarge instance type.  This instance type has  2 SSD-based volumes each with 1024 GB of instance storage.  More on AWS instance types here: http://aws.amazon.com/ec2/instance-types/

You could also just use Oracle RDS on AWS.  With PIOPS for RDS, you can get up to 25K IOPS on your Oracle database. 
Instagram uses SSD : http://m.cio.com/article/716829/SSDs_Boost_Instagram_39_s_Speed_on_Amazon_EC2

EC2 EBS Optimized Instances

Because Oracle databases are typically terabytes in size and required 1000's of IOPS, you will typically use EBS PIOPS volumes with EBS-optimized EC2 instances.   This leads the often asked question of "What does EBS optimized EC2 instances really mean?"  The short answer is that storage pipe to the EBS (PIOPS) volume is dedicated and does not compete with network traffic like it does in a non EBS-optimized instance.  A more detailed answer can be found here:
http://perspectives.mvdirona.com/2012/08/01/EBSProvisionedIOPSOptimizedInstanceTypes.aspx

On a related note, additional instance types now offered EBS Optimized functionality: http://aws.amazon.com/about-aws/whats-new/2013/03/19/announcing-ebs-optimized-support-for-additional-instance-types/

Oracle Database PIOPS and striping

A common question on PIOPS and striping:

Question: Striping with PIOPS - Is it necessary to stripe data when using PIOPS?

Response: It depends on how many PIOPS you are looking to achieve.  Since max PIOPS is per volume is 2000 PIOPS, if you need more then 2000 PIOPS then you would need to stripe.  Less than 2000 PIOPS then striping is not required.