Oracle Database on Amazon Web Services Book

The first book that covers running Oracle on Amazon Web Services:
http://www.amazon.com/Oracle-Databases-Amazon-Web-Services/dp/012800956X

Written by these companies:
http://www.appsassociates.com/awslabs/
http://aws.amazon.com/rds/partners/dbvisit/
http://www.e-dba.com/oracle_labs_aws.html

Network interoperability lab

I found out about a place where networking interoperability is the main focus.  It happens to be where I live in New Hampshire.
www.iol.unh.edu

OpenSwan on AWS

A common use case for using a third party VPN solution such as OpenSwan is to connect two regions VPCs through the use of an IPSec VPN server.  

First, set up a VPC in both regions with, here is what I did:

Region 1 (US-West-2) - VPC 10.0.0.0/16 with private subnet 10.0.0.0/24

Region 2 (Australia)- VPC 172.0.0.0/16 with private subnet 172.0.0.0/24

==================================================================================================================

Configure the VPN server software for the EC2 instances - Region 1

==================================================================================================================

Step 1
------
sudo yum install openswan

Step 2
------
nano /etc/ipsec.conf

Step 3
------
sudo vi /etc/ipsec.d/vpc1-to-vpc2.conf

Step 4
------
conn vpc1-to-vpc2
 type=tunnel
 authby=secret
 left=%defaultroute
 leftid=<EIP1>
 leftnexthop=%defaultroute
 leftsubnet=<VPC1 CIDR>
 right=<EIP2>
 rightsubnet=<VPC2 CIDR>
 pfs=yes
 auto=start

Step 5
------
sudo vi /etc/ipsec.d/vpc1-to-vpc2.secrets

Step 6
------
<EIP1> <EIP2>: PSK "<TYPE A KEY HERE>"

==================================================================================================================

Configure the VPN server software for the EC2 instances - Region 2

==================================================================================================================

Step 7
------
sudo vi /etc/ipsec.d/vpc2-to-vpc1.conf

Step 8
------
conn vpc2-to-vpc1
 type=tunnel
 authby=secret
 left=%defaultroute
 leftid=<EIP2>
 leftnexthop=%defaultroute
 leftsubnet=<VPC2 CIDR>
 right=<EIP1>
 rightsubnet=<VPC1 CIDR>
 pfs=yes
 auto=start

Note the CIDR needs to include the block range. For example: 10.0.0.0/16

Step 9
------
sudo vi /etc/ipsec.d/vpc2-to-vpc1.secrets

Step 10
-------
<EIP2> <EIP1>: PSK "<TYPE THE SAME KEY FROM STEP 6 HERE>"

==================================================================================================================

Configuration in each region

==================================================================================================================

Step 11
-------
a-
sudo service ipsec start

b-
sudo chkconfig ipsec on

c-
sudo vi /etc/sysctl.conf

net.ipv4.ip_forward = 1

d-
sudo service network restart


==================================================================================================================

Test your connections

==================================================================================================================

Step 1 - Region 1
------
ping 172.0.0.50

Step 2 - Region 2

ping 10.0.0.50

AWS Elastic Beanstalk and NAT instance

An Elastic BeanStalk launched in a VPC with a private subnet requires a NAT. Each instance needs to be able to talk to the Internet in order to answer the waitcondition. Connectivity can be provide through a NAT instance but there does have to be access to the internet.

The following show a VPC configuration with a private VPC. It is the connectivity to the Elastic Beanstalk end point that is needed. As you can see it is outside the VPC.

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo-vpc-basic.html

You can find more detailed instructions for creating and configuring a NAT instance here:

 http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html

Instance types and network throughput

Network throughout is a very good metrics to know when running an Oracle Database on AWS or for any time you are moving data from different AWS services other AWS services. Maximum throughput for different Amazon EC2 instance types can be found here:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html

This 2013 AWS reInvent session also has details http://www.slideshare.net/AmazonWebServices/stg302-28617072 (Slide 19)

The m1.xlarge instance maximum network through put is 128 MBPS / 1K Mbps. The CC2.8xlarge, CR1.8xlarge, hi1.4xlarge, and cg1.xlarge (http://www.slideshare.net/AmazonWebServices/stg302-28617072) can provide up to 800 MPBS.

On premise mapped to AWS

A question that often comes up when companies are migrating Oracle workloads to AWS is: "How do my on premise IT architecture components map to AWS?".  Here are some of the most common components mapping from on premise to AWS:

Storage tiering on AWS

Here is the replay of the session I just presented with an AWS partner (App Associates) and customer (Riso):http://cloudconclave.blogspot.com/2013/11/aws-storage-tiering-for-enterprise.html

I opened the session by 'testing' the audiences understanding of AWS storage tiers:
1. Assuming a 16K block size, what storage option provides average throughput of 1 to 2 MBPS ?
2. What storage option has single threaded through put of around 17 MBPS ?
3. Assuming a 16K block size, what EBS volume provides an average of 16-20 MBPS through put ?
4. Once again assuming 16K block size and also assuming 4 1K PIOPS volumes, for what EC2 instances will you start to see network saturation ?
5. What storage option produces approximately 100-145 MBPS read and write through put?
6. For which storage option is it possible to transfer approximately 3 TB a day a day over a WAN ?




















Answers:
1. Standard IOPS
2. S3
3. 1K PIOPS
4. Any instance with .5 Gpbs network connection. For example, m2.2xlarge
5. Hi1.4xlarge (high IO) ephemeral storage
6. AWS Storage gateway

Oracle Database size and network throughput : IOPS and network throughput

When configuring an Oracle Database on AWS EC2, you need to consider both storage (EBS) IOPS and network throughput.   With PIOPS, you can achieve up to 4K IOPS per EBS volumes.  However, the EC2 instances (assuming EBS optimized or 10 Gbps cluster compute) can be a potential bottle neck.  For example, the m2.2xlarge instance type has a maximum throughput of 0.5Gb.  Which means this instance type is limited to approximately 3750 * 16 KiB IOPS. Therefore, one 4K PIOPS volume would start to saturate the network.   Take into account the IOPS and instance network throughput when designing your Oracle Database on AWS EC2.

Fixed MAC address for EC2 instances

Some software licenses require a fixed MAC address to exist.  Running on AWS EC2 the instance will get a new MAC address on reboot or shutdown or restart.  I way to ensure the new EC2 instance will get the same MAC address is to use an Elastic Network Interfaces (ENI).  ENI is like an additional network card that have fixed IP address (internal and external) and fixed MAC address.

IDS : Host-based and network-based

Intrusion Detection Systems and Intrusion Protection Systems on AWS come up when discussion security when moving to AWS.  Here is short introduction to Oracle technologist that may not deal with IDS and IPS in their current on premise environment.

A host-based intrusion detection system (HIDS) is an intrusion detection system that monitors and analyzes the internals of a computing system, and in some cases the network packets on its network interfaces (just like an NIDS).  A host-based IDS monitors all or parts of the dynamic behavior and the state of a computer system. HIDS was first designed for the mainframe.  HIDS uses sensors (agents) located on each host.    These host-based agents, which are sometimes referred to as sensors (or agents), would typically be installed on a machine that is deemed to be susceptible to possible attacks. The term “host” refers to an individual computer/virtual host. This means that separate sensor would be needed for every machine/virtual host. Sensors/agents work by collecting data about events taking place on the system being monitored. This data is recorded by operating system in audit trails. Therefore, HIDS is very log intensive.

Network-based intrusion detection systems offer a different approach. NIDS collects information from the network itself rather than from each separate host. They operate essentially based on a “wiretapping concept" (network taps).  Information is collected from the network traffic stream, as data travels on the network.  The intrusion detection system checks for attacks or irregular behavior by inspecting the contents and header information of all the packets moving across the network. The network sensors come equipped with “attack signatures” that are rules on what will constitute an attack, and most network-based systems allow advanced users to define their own signatures.  this method is also known as packet sniffing, and allows the sensor to identify hostile traffic.

  

Traffic costs between EC2 instances, AZs, and VPCs

Often times when running on AWS you will have multiple AZs, accounts and VPCs.  This is a simple summary of EC2 data transfer costs:

1. Within the same AZ  : Free: It is within the same AZ and over private IP.
2. Between two AZs :   Cost. Will incur inter-az charges for both inbound and outbound

3. Between Accounts in different VPCs :  Cost. Inter-az charges are applicable for data transfer charges between instances in the different AZ's (same region) and in different accounts.
4. Between one EC2 classic instance and VPC EC2 instance in a same AZ : Cost
5. Instances to S3 in the same region: Free
6. Instance to S3 in different region: Cost. cross region charges

AWS Networking 101 for Oracle DBAs, Developers and Architects

Oracle DBAs understand TCP/IP and ports as this is how they connect to and manage an Oracle database.  However, there is no need to understand other networking constructs such as routing tables, network translation, VPN tunnels, or even a network mask.  This blog post will cover networking terminology, AWS networking services and features, and specifics around DNS.

Below are some general network terms and constructions you need to understand when you move to AWS:

1. CIDRs (Classless Inter-Domain Routing) : CIDR is also known as supernetting as it effectively allows multiple subnets to be grouped together for network routing.  CIDR specifies an IP address range using a combination of an IP address and its associated network mask. An example is, 192.168.1.0/24.  This means that the first three octnets (192, 168,  and 1) are fixed and the last octnet is available to use.  Therefore, there are 256 IP addresses available to use 192.168.1.0 - 192.168.1.255.  CIDRs are used in AWS VPC and security groups. 
2. VPN (Virtual Private Network) : Extends a private network across a public network.  This allows AWS to be an extension of your corporate network.  It also provides security, encryption, and management across your Internet-based connection to AWS.
3. Ipsec : Is a protocol suite for securing IP communications.  When you establish a VPN connection to AWS VPC, you create an IPSec tunnel for secure communication over the Internet. More here : http://cloudconclave.blogspot.com/2013/03/getting-started-with-aws-vpc.html
4. Layer 2 and Layer 3 networks : The Internet Protocol (IP) address is a layer 3 address.  Layer 3 networks do routing at the IP level.  Layer 2 networks operate at the data link layer of the network.  Therefore, they use the Media Access Control (MAC) address to determine where to direct the message.  AWS is a layer 2 network.  The fact AWS is a layer 2 network could impact some of the 3RD party solutions that can run on AWS.
5. Multicast and unicast : Multicast is a true broadcast. The multicast source relies on multicast-enabled routers to forward the packets to all client subnets that have clients listening.Unicast is a one-to one connection between the client and the server. Unicast uses IP delivery methods such as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP), which are session-based protocols.  AWS only supports unicast.  Some software products (such as Oracle RAC) use multicast so they can not be run on AWS infrastructure. 
6. VLAN : A single layer-2 network may be partitioned to create multiple distinct broadcast domains.  When using AWS Direct Connect, you can provision virtual interface (VLAN) connections to the AWS cloud, Amazon VPC, or both.  You can not extend you data center VLAN into the AWS cloud when using AWS Direct Connect.
7. NAT : Network Address Translation (NAT) is the process of modifying IP address information in IPv4 headers while in transit across a traffic routing device.  NAT AWS EC2 instances are used to translate IP addresses in an AWS VPC when instances are in a private subnet and need to communicate with the outside world.
8. SDN : Software-defined networking (SDN) is an approach to computer networking which abstracts the distributed systems, the control plane and the data plane. SDN is similar to what virtual machines have done for compute virtualization. SND is network virtualization.  
9. iptables : The Linux iptables are essentially the way an AWS NAT instance does the IP (actually does port routing so AWS NAT is actually a PAT - Port Address Translation).

10. Overlay networks : An overlay network is a computer network which is built on the top of another network.  For example, since the AWS network is a layer 2 network that does not support multi-cast, you cloud place a overlay network on top of the base AWS network that supports multi-cast.  Blog post on overlay and SDN : http://cloudconclave.blogspot.com/2013/06/overlay-networks-on-aws.html
11. BGP : Border Gateway BC Protocol (BGP) is the protocol which is used to make core routing decisions on the Internet; it involves a table of IP networks or "prefixes" which designate network reachability among autonomous systems (AS).  BGP does dynamic routing and AWS refers to a BGP device as the Customer Gateway when using a VPN connection to AWS VPC.
 12. ASA : Cisco ASA is a static routing device.  The Cisco ASA device is referred to as the Customer Gateway when using a VPN connection to AWS VPC.

These are AWS specific services and components:

1. VPC : Amazon Virtual Private Cloud (Amazon VPC) lets you provision a logically isolated section of the Amazon Web Services (AWS) Cloud where you can launch AWS resources in a virtual network that you define. You have complete control over your virtual networking environment, including selection of your own IP address range, creation of subnets, and configuration of route tables and network gateways.
2. Internet Gateway :  The Internet Gateway allows EC2 instances in a VPC communicate with the Internet.  When you launch an AWS VPC with a public subnet it comes with an Internet gateway, and instances launched into a public subnet have a public IP address and communicate with the internet using the Internet Gateway. 
Instances that you launch into a private subnet do not receive a public IP address, and can't communicate with the Internet. You can enable Internet access for instances that you launch into a private subnet by using a NAT instance.
3. Customer Gateway : A customer gateway is a physical device or software application on your side of the VPN connection.  The Customer Gateway is used to create an secure IPsec VPN tunnel to AWS VPC.
4. Virtual Private Gateway A virtual private gateway is the VPN concentrator on the Amazon side of the VPN connection.  The VPG is a service provided by AWS.
5. ENI : An elastic network interface (ENI) is a virtual network interface that you can attach to an instance in a VPC. ENIs allow an EC2 instance to have more than one IP address.  This includes a primary private IP address, one or more secondary private addresses, or an Elastic IP address. You can create a network interface, attach it to an instance, detach it from an instance, and attach it to another instance. The attributes of a network interface follow the network interface as it is attached or detached from an instance and reattached to another instance. When you move a network interface from one instance to another, network traffic is redirected to the new instance.  This is feature is useful for creating a management network, dual homed instances, or security appliances in your VPC.
6. ElasticIP : An Elastic IP address (EIP) is a static public IP address that can be assigned to an EC2 instance or an ENI.  A more appropriate name for an EIP may be a Public IP address. With an EIP, you can mask the failure of an instance by rapidly remapping the address to another instance. Your EIP is associated with your AWS account, not a particular instance, and it remains associated with your account until you choose to explicitly release it.

There's one pool of EIPs for use with the EC2-Classic platform and another for use with your VPC. You can't associate an EIP that you allocated for use with a VPC with an instance in EC2-Classic, and vice-versa.
7. Public and Private Subnet : A subnet is a range of IP addresses in your VPC. You can launch AWS resources into a subnet that you select. Use a public subnet for resources that must be connected to the Internet, and a private subnet for resources that won't be connected to the Internet. instances in the public subnet can receive inbound traffic directly from the Internet, whereas the instances in the private subnet can't. The instances in the public subnet can send outbound traffic directly to the Internet, whereas the instances in the private subnet can't.More on public and private subnets can be found here: http://cloudconclave.blogspot.com/2013/05/aws-vpc-public-and-private-subnets.html

8. NAT Instances : Instances that you launch into a private subnet in a virtual private cloud (VPC) can't communicate with the Internet. You can optionally use a network address translation (NAT) instance in a public subnet in your VPC to enable instances in the private subnet to initiate outbound traffic to the Internet, but prevent the instances from receiving inbound traffic initiated by someone on the Internet.
 9. Route 53 : Amazon Route 53 is a Domain Name System (DNS) web service.  More on Route 53 can be found here: http://cloudconclave.blogspot.com/2013/05/routing-53-as-your-dns-service.html.  Route 53 resolves an IP address to a domain name.
10. Direct Connect : Direct Connect makes it easy to establish a dedicated network connection from your premises to AWS. Using AWS Direct Connect, you can establish private connectivity between AWS and your datacenter, office, or colocation environment, which in many cases can reduce your network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections.  Direct Connect has speeds of 1 Gbps or 10 Gbps.   When companies are extending their Oracle solutions into the cloud, they often times chose to use Direct Connect as Internet speeds are not fast enough.  More on Direct Connect http://cloudconclave.blogspot.com/2013/06/aws-direct-connect-active-active-with.html and http://cloudconclave.blogspot.com/2013/06/aws-vpn-connection-as-direct-connect.html.  Direct Connect also refers to a facility that is next to an AWS data center that can be used to host third party hardware and software solutions such as Oracle RAC. More on this here: http://cloudconclave.blogspot.com/2013/06/oracle-rac-on-aws.html
11. CloudFront : CloudFront is an edge location content delivery service.  It is mostly used to deliver static content such as web sites, documents, videos, pictures etc.  However, it can also be used for dynamic content.

Specific to Route 53 (the AWS DNS Hosting Service):http://cloudconclave.blogspot.com/2013/05/routing-53-as-your-dns-service.html
1. DNS hosting service : A DNS hosting service is a service that runs Domain Name System servers.  

2. A records : An A record (Address Record) points a domain or subdomain to an IP address.

3. Zone apex record : I sometimes called the root domain or naked domain.  The apex record would be domainname.com without a www or any another prefix.

4. Cname : A CNAME (Canonical Name) points one domain or subdomain to another domain name, allowing you to update one A Record each time you make a change, regardless of how many Host Records need to resolve to that IP address.
5. Alias records : Route 53 offers ‘Alias’ records (a Route 53-specific virtual record). Alias records are used to map resource record sets in your hosted zone to Elastic Load Balancing load balancers, CloudFront distributions, or S3 buckets that are configured as websites. Alias records work like a CNAME record in that you can map one DNS name (example.com) to another ‘target’ DNS name (elb1234.elb.amazonaws.com). They differ from a CNAME record in that they are not visible to resolvers. Resolvers only see the A record and the resulting IP address of the target record.

Security also plays a key role when configuring a network on AWS. More on security can be found here: http://cloudconclave.blogspot.com/2013/07/aws-security-101-for-oracle-dbas.html

Overlay networks on AWS

Software Defined Networks (SDN) are all the rage and for good reason.  A couple of very good reasons when running on AWS are:
1. Supporting multi-cast on AWS which does not natively support multi-cast
2. Dealing with network address overlapping when using AWS VPC

There are a number of alternatives.  https://code.google.com/p/openpgm/
This is one that is certified to run on AWS:
http://www.cohesiveft.com/products/vns3

Here is a very comprehensive post on multicast on AWS:

 https://www.buckhill.co.uk/blog/how-to-enable-broadcast-and-multicast-on-amazon-aws-ec2/2#.VZP_MbzI-lO

Distributed File System : Network, Distributed or Clustered ?

I often times hear these three distinct DFSs used to mean one in the same.  This presentation does a nice job of describe how they are different and how they are the same:

http://lvee.org/uploads/image_upload/file/273/savchenko-distributed-fs.pdf


Network File System: A single server (or at least an appearance) and multiple network clients.
Examples: NFS, CIFS

Clustered File System:Servers sharing the same local storage (usually SAN at block level)shared storage architecture.
Examples: GFS2, OCFS2

Distributed file system : “Shared nothing” model, independent servers. intelligent server architecture.
Examples: pNFS, AFS

Determining Domain Name System (DNS) name server

When setting up a CDN (like AWS CloudFront), you will want to determine if your web site is really being served from CloudFront you can use the following command:
-dig www.pearson.aws.migrating2cloud.com

AWS RDS database encryption

There a couple of ways to encrypt you data at rest when using AWS RDS:

1. Oracle RDS TDE support : http://aws.amazon.com/about-aws/whats-new/2013/04/18/amazon-rds-oracle-encryption
2. Oracle RDS with Ciphercloud : http://www.ciphercloud.com/database-encryption-and-tokenization.aspx.  AWS specific here (and mentions HIPPA) : http://www.ciphercloud.com/aws-encryption-ciphercloud.aspx

Oracle TDE also supports encryption at the network tier. 

Extending your home or small business network to AWS VPC

Extending you home, small business, data center, co lo, or enterprise offices to AWS running VPC, you will probably want to run an IPSEC tunnel.  AWS lists the certified VPN devices here:  http://aws.amazon.com/vpc/faqs/#C9.  However, if you are doing this from a home or small office connection, you may not want to use an expensive dedicated device.  In this case the open source pfSense makes sense: www.pfsense.org. It can used to create an inexpensive IPSEC tunnel to AWS VPC using any hardware you have 'laying around'. 

Network speeds and Moore's Law

The speed of the Ethernet has increased from 1 Gbps in 1997 to 100 Gbps in 2010; this increase is slightly slower than the Moore law for traffic [251] which would require 1 Tbps Ethernet by 2013.

Application on premise and database on AWS

It is possible to run an application on premise and run your Oracle database on AWS on EC2 or RDS.  However, in in this scenario, the network latency is typically to high and network bandwidth to limited to support most database systems.  This is where AWS Direct Connect comes in. AWS Direct Connect provides speeds of 1 GBPS to 10GBPS where as a typical internet connection speed of 100MBPS (100Base-T connection) would be significantly slower.  Keep in mind that an internal dedicated network from application server box to database server which would be 10 GBPS to 100 GBPS. If the application server and database server on on the same box there is no network latency. There are customers that run Oracle applications on EC2 and Oracle RAC databases in a co location facility using Direct Connect.

AWS import export : When to use import export

This web page has some guidelines on when to use AWS import export and when to use a network connection:
http://aws.amazon.com/importexport/

Go down to this section: "When to Use Import/Export"

I often get asked this question when someone wants to move large Oracle database to the cloud.