MySQL Recover from EBS snapshots for logical volume

In this blog post http://cloudconclave.blogspot.com/2013/12/mysql-ebs-snapshots-for-backing-up.html, we backed up a MySQL database that stores its data across an LVM using multiple EBS volumes.  Now we will do a restore.

sudo /etc/init.d/mysqld stop 

sudo umount /dev/md0 

sudo mdadm --stop /dev/md0 

sudo mdadm --zero-superblock /dev/sdf 

sudo mdadm --zero-superblock /dev/sdg 

aws ec2 detach-volume --volume-id vol-1 aws ec2 detach-volume --volume-id vol-2

aws ec2 create-volume --snapshot-id snap-1 --availability-zone AZ -- volume-type standard

aws ec2 create-volume --snapshot-id snap-2 --availability-zone AZ -- volume-type standard

aws ec2 attach-volume --volume-id vol-New1 --instance-id INSTANCE -- device /dev/sdf

aws ec2 attach-volume --volume-id vol-New2 --instance-id INSTANCE -- device /dev/sdg

Create the stripped volumes:

yes | sudo mdadm \ --create /dev/md0 \ --level 0 \ --metadata=1.1 \ --raid-devices 2 \

/dev/xvdf /dev/xvdg

sudo mount -a sudo /etc/init.d/mysqld start

Creating PIOPS volume for Oracle Database

When running your Oracle Database on AWS EC2, you may find that you need additional IOPS for your database and which to add an EBS volume to your Oracle instances for some of your highly accessed tables that require high IO.  Here are the steps to create a new PIOPS volume and attached it your EC2 instance.

1. aws ec2 stop-instances --instance-id i-NewDBServer
2. aws ec2 create-volume --availability-zone az-asdf-1a --size 200 --volume-type io1 --iops 2000

3. aws ec2 attach-volume --volume-id vol-FastDrive --instance-id i-NewDBServer - -device /dev/xvdb

Get the OS in order for your EC2 instance to recognize your volume by running the following commands:

1. sudo mkdir /mnt/piops1

2. sudo mkfs.ext3 /dev/xvdb

3. sudo mount /dev/xvdb /mnt/piops1

4. df -h (to see the newly mounted piops volume)

You would then create an Oracle tablespace on this new volume and move your tables to this new tablespace.

EC2 instance public key invalid fix

I happened to do the same thing that is listed at this web page:  http://dltj.org/article/ec2-fix-authorized_keys/

The steps are on the web page. Here are the command:
1. aws ec2 create-volume --snapshot-id snap-YourVolumesSnapshot --size 10 -- availability-zone az-asdf-1a
Note: Assumes you have a current snapshot: aws ec2 create-snapshot --volume-id vol-beforevolumewasbad

 2. aws ec2 stop-instances --instance-id i-YourInstanceId

3. aws ec2 detach-volume --volume-id vol-badvolume

4. aws ec2 attach-volume --volume-id vol-goodvolume --instance-id i- YourInstanceId --device /dev/sda1

5. aws ec2 start-instances --instance-id i-YourInstanceId

Exanding size of your current EBS root volume

In some cases you may need to increase the size your EC2 instance root volume. Here are the steps to do this:

1. Create a volume from an existing EBS snapshot :
aws ec2 create-volume --snapshot-id snap-YourVolumesSnapshot --size 10 -- availability-zone az-asdf-1a 

2. Stop the instance:
was ec2 stop-instances --instance-id i-YourInstanceId

3. Detach the current volume:
was ec2 detach-volume --volume-id vol-TheSmallDefaultVolume

4. Attach the new volume:
was ec2 attach-volume --volume-id vol-TheLargerVolume --instance-id i- YourInstanceId --device /dev/sda1

5. Log into your bastion host:
ssh -i YourFile.pem ec2-user@10.__.__.__

7. Start the instance:
aws ec2 start-instances --instance-id i-YourInstanceId

8. Check the file system size: (It will still be the smaller size)
df –h

9. Resize the file system to the size of the volume
sudo resize2fs /dev/sda1

10. Check the file system size again:
df –h

EBS volume encryption

Here are two companies that offer EBS volume encryption:

1. SafeNet ProtectV

     A. Support for boot volume encryption

     B. Good integration with Hardware Security Modules (HSM)

     C. Somewhat limited OS support  

2. Trend Micro SecureCloud

      A. Good support for RAID arrays

      B. Supports IAM roles for EC2 Instances, HSM support 
      C. Support for root volume encryption

           D. Supports AWS Storage Gateway as well as EBS volumes

AWS EBS cross region migration

AWS offers the capability to migrate an EBS volume from one region to another.  The feature applies to any application or database data:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-copy-snapshot.html

This can be used to move Oracle database EBS snapshot from one region to another for HA and DR.

AWS Provisioned I/Os

Provisioned IOPS volumes are designed to deliver within 10% of the provisioned IOPS performance 99.9% of the time. Therefore, PIOPS are very good choice when running RDBMS systems like Oracle on EC2 or RDS.

AWS EBS PIOPS volume creating tablespace

Creating you EBS volume and attaching it to your EC2 instance through the AWS console, CLI or APIs is just the first step in using this volume in your Oracle database.  If on Linux, you then need to format, create a mount point, and then mount your volume as follows:

1.mkfs -t ext3 /dev/sdg

2. mkdir /mnt/oracle1000

3. mount /dev/sdg /mnt/oracle1000

(Remember to give permissions - chmod - to oracle user on your mount point)

 More information on mounting EBS volumes can be found here:

http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

To check if the device has been mounted properly, you can issue these commands:
-  df -l
-  mount

To insure that your EBS volumes get mounted at start up time, you can add the device mapping to the /etc/fstab file

Then the Oracle tablespace needs to be created.  This is done using the following command:

This is for creating a tablespace larger then the Oracle standard limits:
create bigfile tablespace datapiops1k
datafile '/mnt/oracle1000/bigfiledata.dbf'
size 600G;