Loading data from on premise Oracle Database to Oracle RDS using RMAN

An RMAN backupset can be loaded directly into RDS. However, you can still use RMAN as a mechanism to move an Oracle database from on premise to RDS.

First, move the data RMAN dump using Tsunami, Aspera, Attunity Cloudbeam, or stand protocols like FTP(s) or HTTP(s) which will be slower. More on moving data from on premise to AWS here: http://cloudconclave.blogspot.com/2013/04/on-premise-application-replication-and.html
OR
You can use Oracle Secure Backup to dump the database directly to S3.

Then load the data into using an Oracle Database instance running on EC2 using RMAN (if RMAN dump on EBS volumes) or Oracle Secure Backup if in S3.

You then load the data into RDS from the EC2 instance using one of the these options:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Procedural.Importing.html

You may ask...why did you go through all this trouble of setting an Oracle database on EC2 when I can simply export/dump the data from the on premise Oracle DB and move to EBS or S3 and then load into RDS.  It is because an RMAN dump is much faster on the on premise side and RMAN produces a more compressed and portable database format to move across the internet to AWS.

You can also use this process to load the data using data pump and Oracle Managed Files:
http://cloudconclave.blogspot.com/2013/10/process-to-load-oracle-data-from-on.html

Native replication for replicating data into and out of RDS has announced in Sept 2012 and details can be found here:
http://aws.typepad.com/aws/2013/09/migrate-mysql-data-to-amazon-rds-and-back.html
This is not yet available for Oracle RDS.

Bulk loading data to AWS from on premise

I spoke about bulk loading data to AWS in this blog post:

http://cloudconclave.blogspot.com/2013/04/on-premise-application-replication-and.html

A couple other options I did not mention are:

1. Apera - Asperasoft Company has developed a proprietary file transfer protocol based on UDP, which has shown to introduce very high-speed file transfer experience over the Internet.

2. http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/UsingEMR_s3distcp.html   Apache DistCp is an open-source tool you can use to copy large amounts of data. DistCp uses MapReduce to copy in a distributed manner  You can also use S3DistCp to copy data between Amazon S3 buckets or from HDFS to Amazon S3.  Since it is based upon MapReduce, it is most applicable when moving HDFS and other map reduce files as you will obviously have MapReduce installed in your environment. 

On premise application replication and snap shots to AWS for DR

When devising an disaster recovery approach that used AWS for DR of an on premise environment, there are literally hundreds of options for keeping the databases in sync. Each relational database vendor will have five of their own options and their partner ecosystems will offer many more.  For the application, it is different situation.  Their are not many features built into business application  and application servers.  Depending on what DR scenario you are using http://cloudconclave.blogspot.com/2012/11/aws-four-dr-scenarios.html you will be using either replication or snapshots to keep your AWS application in sync with your on premise environment.  
For warm standby and active-active you will need replication:

• RSync : Rsync is cheap (free) and easy.  Rsync can be chatty, does not have compression built in, and you have to code parallelism. 
• BitTorrent : BitTorrent is most often associated with moving large media files over the Internet in a fast matter. BitTorrent now has file syncing capabilities so can be used to sync application server files.  
• Amazon Storage Gateway " attach these volumes as iSCSI devices to your on-premises application servers.  These files will be stored in S3 where they can be instantiate as EBS volumes.

Riverbed Whitewater - http://www.riverbed.com/products-solutions/products/cloud-storage-whitewater/ https://aws.amazon.com/marketplace/pp/B007O0FXW4/ref=srh_res_product_title?ie=UTF8&sr=0-2&qid=1366938137611#product-detail

Attunity CloudBeam - http://www.attunitycloudbeam.com/solutions/disaster-recovery  https://aws.amazon.com/marketplace/pp/B00B5PB8IM/ref=srh_res_product_title?ie=UTF8&sr=0-3&qid=1366938015695

For backup and restore and pilot light you can use snapshots. 

• FTP or SFTP : FTP and SFTP are the two most common methods to transfer files over the internet.
• HTTP or HTTPs : HTTP and HTTPS can be used but they are slower than S/FTP.  However, if you don't want to require or can not have an FTP client in use than HTTP(s) can be used.  Some companies may not want to open up another port to the outside world have an FTP server running.
• AWS import/export : If files don't need to be synced that often, FedEx, UPS, or Post Office can be the least expensive method with the least amount of hassle (limited coding, no restarting needed)
• Attunity CloudBeam to move files into S3 and than instantiate as EBS volumes.

Oracle EBusiness Suite does offer rapid clone http://docs.oracle.com/cd/E18727_01/doc.121/e12841/T120505T120517.htm but this is not for moving production system data files but for setting test, dev, or moving a system to another machine. Weblogic also offers a cloning feature :http://docs.oracle.com/cd/E12839_01/core.1111/e10105/clone.htm

Connecting to AWS EC2 using SSH and SFTP

As I mentioned in a previous blog, you can only use SFTP (can not use FTP) to connect to an EC2 instance.  The other thing you need to have correct is an entry in the SSH config file for your  private key (of the key pair value for the EC2 instance).   In the Unix/Mac world, this file is called config and it is located in the ~/.ssh directory.  The following entry needs to be added to this config file:

identityFile ~/tomlaszKeys/PemFileName.pem

Adding this entry will also allow you to connect via SSH without having to pass in the private key file location:

ssh ec2-user@ec2-23-23-237-99.compute-1.amazonaws.com

Instead of:

ssh ec2-user@ec2-23-23-237-99.compute-1.amazonaws.com

 -i ~/PemfileName.pem

AWS EC2 amazon sftp only for web site upload pem pkk ftp ssh

The most difficult part of creating my AWS EC2 web site (other then the 'coding' part ... more on this tomorrow) was getting the files loaded.
1. SFTP can only be used. FTP will not work! Discussion is here: SFTP with EC2 (or can use scp)
2. PEM file to a .PPK file - I used Filezilla as described here: Converting PEM to PPK file. Seems as though sftp does not use pem file like SSH does.
3. login in to EC2 instance using SSH and change user to root ( sudo su ) and change the directory with location of you web pages (/var/www/html) to '755' (chmod
-R 755 /var/www/html). This is because you will be using your ec2-user name to upload using SFTP.
4. login into SFTP using ec2-user and your instance name: sftp ec2-user@
Note: These file go into the directory /var/www/html.