Amazon data warehouse migration to Amazon Redshift

Informative video on moving an Oracle RAC and Hadoop based web log data mining solution to Amazon Redshift:

https://www.youtube.com/watch?v=P4KPPvEb_QI

Redshift example

This a great place to get started with Redshift: http://docs.aws.amazon.com/redshift/latest/gsg/getting-started.html.  There are a couple of pieces of information in the step by step instructions I wanted to elaborate on:
1. Creating a cluster subnet group is done in the Redshift area of the console as seen below. This is not evident in the instructions:

This of course assumes a VPC with two public subnets and a route to an IGW has been created.

2. The copy command is issued from SQL Workbench.copy venue from 's3://awssampledb/tickit/venue_pipe.txt' CREDENTIALS 'aws_access_key_id=<your access key>;aws_secret_access_key=<you secret key>' delimiter '|';

3. Make sure to set auto commit otherwise you need to commit each statements or block of statements and the example does not have commit commands.

Redshift - New node and data distribution

What happens when a new node is added to a Redshift cluster?:

A 2-node cluster will distribute data evenly between two nodes based on a hash of the DISTKEY. If a 3rd node is added, the data needs to be rebalanced amongst the 3 nodes. It’s not just a matter of sending all new data to the 3rd node because that would require lookups to figure out where data is stored. Rather, the node needs to be rebalanced by redistributing the data between the 3 nodes. Redshift takes care of this automatically. Just add the nodes and the data moves.

Redshift redistributes the data as follows:

• A ‘new’ set of nodes is created (in the above example, 3 nodes would be created)
• Redshift moves the data from the 2-node cluster to the 3-node cluster, rebalancing the data during the copy
• Users are then flicked across from the ‘old’ 2-node cluster to the ‘new’ 3-node cluster

This is an example of scalable cloud infrastructure — rather than having to ‘upgrade’ an existing system, it is much more efficient to provision a new system, copy data and then decommission the old system. This is a new way of looking at infrastructure that is quite different to the old way of thinking in terms of physical boxes.

AWS 'Exalytics' on AWS

Exalytics is YAOES (Yet Another Oracle Engineered System). 
https://blogs.oracle.com/emeapartnerbiepm/entry/lastest_bic2g_and_exalytics_demonstration

This web page mentions: The Business Intelligence Challenge to Go (BIC2g) is a demonstration, training, and POC business intelligence platform for Oracle Partners. It is a comprehensive platform, containing the full EPM, BI, and BI Apps (including full ETL with DW) product suites, with extensive (40+) demos. It is available in the Amazon Cloud (not as a download).

This demonstration is actually the test drive for OBIEE (which is the core software stack on the Exalytics machine) on the AWS web site here :

http://aws.amazon.com/solutions/global-solution-providers/oracle/labs/#Oracle_Fusion_Middleware