OpenSwan on AWS

A common use case for using a third party VPN solution such as OpenSwan is to connect two regions VPCs through the use of an IPSec VPN server.  

First, set up a VPC in both regions with, here is what I did:

Region 1 (US-West-2) - VPC 10.0.0.0/16 with private subnet 10.0.0.0/24

Region 2 (Australia)- VPC 172.0.0.0/16 with private subnet 172.0.0.0/24

==================================================================================================================

Configure the VPN server software for the EC2 instances - Region 1

==================================================================================================================

Step 1
------
sudo yum install openswan

Step 2
------
nano /etc/ipsec.conf

Step 3
------
sudo vi /etc/ipsec.d/vpc1-to-vpc2.conf

Step 4
------
conn vpc1-to-vpc2
 type=tunnel
 authby=secret
 left=%defaultroute
 leftid=<EIP1>
 leftnexthop=%defaultroute
 leftsubnet=<VPC1 CIDR>
 right=<EIP2>
 rightsubnet=<VPC2 CIDR>
 pfs=yes
 auto=start

Step 5
------
sudo vi /etc/ipsec.d/vpc1-to-vpc2.secrets

Step 6
------
<EIP1> <EIP2>: PSK "<TYPE A KEY HERE>"

==================================================================================================================

Configure the VPN server software for the EC2 instances - Region 2

==================================================================================================================

Step 7
------
sudo vi /etc/ipsec.d/vpc2-to-vpc1.conf

Step 8
------
conn vpc2-to-vpc1
 type=tunnel
 authby=secret
 left=%defaultroute
 leftid=<EIP2>
 leftnexthop=%defaultroute
 leftsubnet=<VPC2 CIDR>
 right=<EIP1>
 rightsubnet=<VPC1 CIDR>
 pfs=yes
 auto=start

Note the CIDR needs to include the block range. For example: 10.0.0.0/16

Step 9
------
sudo vi /etc/ipsec.d/vpc2-to-vpc1.secrets

Step 10
-------
<EIP2> <EIP1>: PSK "<TYPE THE SAME KEY FROM STEP 6 HERE>"

==================================================================================================================

Configuration in each region

==================================================================================================================

Step 11
-------
a-
sudo service ipsec start

b-
sudo chkconfig ipsec on

c-
sudo vi /etc/sysctl.conf

net.ipv4.ip_forward = 1

d-
sudo service network restart


==================================================================================================================

Test your connections

==================================================================================================================

Step 1 - Region 1
------
ping 172.0.0.50

Step 2 - Region 2

ping 10.0.0.50

AWS Elastic Beanstalk and NAT instance

An Elastic BeanStalk launched in a VPC with a private subnet requires a NAT. Each instance needs to be able to talk to the Internet in order to answer the waitcondition. Connectivity can be provide through a NAT instance but there does have to be access to the internet.

The following show a VPC configuration with a private VPC. It is the connectivity to the Elastic Beanstalk end point that is needed. As you can see it is outside the VPC.

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo-vpc-basic.html

You can find more detailed instructions for creating and configuring a NAT instance here:

 http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html

EMR in a private subnet

When running EC2 instance or other AWS services in a private subnet, you need a NAT to access S3. 

You can not use a NAT when using EMR:

http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-plan-vpc-subnet.html

Because access to and from the AWS cloud is a requirement of the cluster, you must connect an Internet gateway to the VPC subnet hosting the cluster. If your application has components you do not want connected to the Internet gateway you can launch those components in other subnets you create within your VPC. In addition, because of the need to access the AWS cloud, you cannot use Network Address Translation (NAT) when you are running Amazon EMR on a VPC.

EC2 instances in public subnet calling S3 in same region

Traffic to and from S3 and  EC2 (in a public subnet) doesn’t go over the public Internet  in the same region.The traffic goes to the “AWS edge” (Internet Gateway) in that region.  It is the “public Internet” in the sense that you need an Internet Gateway, and S3’s endpoints are Internet-facing. However, the traffic does not move beyond the AWS-controlled networks if you stay within the same region.  The EC2 Instances (or other AWS services)  traffic simply travels via the Internet Gateway to S3. Obviously, for EC2 instance in private subnets they traffic would need to go through a NAT instance.

VPC benefits over EC2 classic

Here are some of the reasons why you would use VPC instead of EC2 classic for your Oracle instances:

• Predictable internal IP ranges: You define the IP address range of your VPC as opposed to your IP address being part of the AWS region IP address range.
• Subnets : Logically group Amazon EC2 instances into a private or public subnets and assign them private IP addresses.
• Traffic Routing : Control the outbound/egress traffic from your Amazon EC2 instances (in addition to controlling the ingress traffic to them; EC2 Classic security groups are ingress only) and provide selective internet access to instances.
• Network ACLs : Additional layer of security to your Amazon EC2 instances in the form of network Access Control Lists (ACLs). These allow for deny rules instead of just allow rules that security groups have.
• VPN Connectivity : Connect your VPC to corporate data center and on-premise infrastructure with a VPN connection, so that you can use Amazon VPC as an extension of your existing data center network,
• DHCP options: DHCP option sets let you specify the domain name, DNS servers, NTP servers, etc. that new nodes will use when they’re launched within the VPC. This makes implementing custom DNS much easier. In EC2 you have to spin up a new node, modify DNS configuration, then restart networking services in order to gain the same effect. 
• Multiple IP's per MAC address: The Elastic Network Interfaces (ENI) is a virtual network interface that can include the following attributes :
  • a primary private IP address
  • one or more secondary private IP addresses
  • one Elastic IP address per private IP address
  • a MAC address
  • one or more security groups
  • a source/destination check flag
  • a description
• Multiple ENIs per instance: Attach multiple Elastic Network Interfaces (ENI) to each instance for multiple MAC addresses.
• Moving ENIs (MAC and IP addresses) between instances :  ENI's attributes follow the ENI as it is attached or detached from an instance and reattached to another instance.

Redshift example

This a great place to get started with Redshift: http://docs.aws.amazon.com/redshift/latest/gsg/getting-started.html.  There are a couple of pieces of information in the step by step instructions I wanted to elaborate on:
1. Creating a cluster subnet group is done in the Redshift area of the console as seen below. This is not evident in the instructions:

This of course assumes a VPC with two public subnets and a route to an IGW has been created.

2. The copy command is issued from SQL Workbench.copy venue from 's3://awssampledb/tickit/venue_pipe.txt' CREDENTIALS 'aws_access_key_id=<your access key>;aws_secret_access_key=<you secret key>' delimiter '|';

3. Make sure to set auto commit otherwise you need to commit each statements or block of statements and the example does not have commit commands.

OpenVPN Server on AWS EC2

OpenVPN is a popular method to use to create an encrypted IPSec tunnel or SSL tunnel from client machines to AWS.  However, there is not much documentation or specifics on the web to walk through the set up OpenVPN on AWS and the client tools and configuration necessary.  Here are some step by step instructions for creating a encrypted SSL tunnel with caveats included:

1. Create the OpenVPN instance on AWS: Spin-up an Amazon Linux server (m1.small is fine) in a public subnet in the VPC you want to connect to. The VPC has to be a 10.0.0.0/16 network or you'll have to adjust these instructions a bit.  Put it in a separate security group with TCP 443 inbound from everywhere (for VPN connections) and TCP 22 inbound only from IPs you trust (for SSH admin)

      Note:

·   Need to create VPC with a public subnet .  I created a VPC with a public and private subnet as the whole idea behind this exercise is to have instances locked down from access and to the outside world by placing them in private subnets.

·   Need to create a security group.  Created a new security group OpenVPNConfig. For TCP port 443 (port of OpenVPN server),  needs to have a custom TCP rule for address in 10.0.0.0/16

·   Give 22 (SSH) to 0.0.0.0/0 for now just to get be able to work with instance to configure properly.  Once the OpenVPN server is running and tested, you will connect via VPN only so you should remove this rule.

·   Create a new key pair if desired.

2.Give the OpenVPN instance an EIP. You can do this by associating an ENI to the instance.

3.Login, and yum install openvpn

      A. sudo yum -y install openvpn

4.Do this: http://www.openlogic.com/wazi/bid/188052/From-Zero-to-OpenVPN-in-30-Minutes, with one caveat: when you do the build-dh command it'll generate a dh1024.pem file – that's the one you need, not 01.pem.

NOTE:

1.    Instruction for location say here /usr/share/doc/openvpn/examples/easy-rsa/2.0 but actually here: /usr/share/openvpn/easy-rsa/2.0

2.    Need to execute as root : sudo su

3.    Command used: cp -r /usr/share/openvpn/easy-rsa/2.0

/etc/openvpn/

4. I actually got 01.pem and 02.pem and dh1024.pem

Server Configuration – This is actually the same as the web page starting at section called Server Configuration.

5.Adjust the /etc/openvpn/openvpn.conf file to be something like this.  Note this uses TCP443 instead of UDP so it'll get through the AWS firewall.

port 443

proto tcp-server

dev tun

ca /etc/openvpn/keys/ca.crt

cert /etc/openvpn/keys/test-system.crt

key /etc/openvpn/keys/test-system.key

dh /etc/openvpn/keys/dh1024.pem

cipher BF-CBC

server 10.8.0.0 255.255.255.0

push "route 10.0.0.0 255.255.0.0"

comp-lzo

verb 6

ifconfig-pool-persist /etc/openvpn/ipp.txt

keepalive 10 120

status openvpn-status.log

My file:

port 443

proto tcp-server

dev tun

ca /etc/openvpn/2.0/keys/ca.crt

cert /etc/openvpn/2.0/keys/openvpn-system.crt

key /etc/openvpn/2.0/keys/openvpn-system.key

dh /etc/openvpn/2.0/keys/dh1024.pem

cipher BF-CBC

server 10.8.0.0 255.255.255.0

push "route 10.0.0.0 255.255.0.0"

comp-lzo

verb 6

ifconfig-pool-persist /etc/openvpn/ipp.txt

keepalive 10 120

status openvpn-status.log

6. Make it auto-start: sudo chkconfig openvpn on && sudo service openvpn start

Note:

1. The startup failed the first time I tried to start because of an error in my config file.  I had to run without chkconfig and with –config <config file location and name> to find out what error was.

7. Copy the client1.pem, client.crt and ca.crt from the server (or whatever you generated with build-key etc.) from the instructions you followed above… to your Mac.

            A. My files were: ca.crt, openvpn-system.crt, tom.crt, tom.key, 01.pem (seems to be associated with openvpn-system.crt), 02.pem (seems to be associated with tom.crt)

B. And Diffie-Hellman pem files: dh1024.pem

C. Copy using scp: scp -i /Users/tomlasz/Documents/Documents/EC2KeyPairs/OpenVPN.pem ec2-user@<elastic ip address>:/etc/openvpn/2.0/keys/tom.crt .

D. I needed to do a chmod 777 on the keys directory to get scp to work.

E. I needed to do a chmod 644 on the tom.key file to get scp to work on that file.

8. Setup IPTables on the OpenVPN server so that it'll do NAT out to the VPC for clients connecting to the VPN.  Here are all the commands you need assuming you used the instructions above.  As root, (sudo –s) run these on the server:

iptables -I FORWARD -i tun0 -o eth0 -s 10.8.0.0/24 -d 10.0.0.0/16 -m conntrack --ctstate NEW -j ACCEPT

iptables -I FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

iptables -t nat -I POSTROUTING -o eth0 -s 10.8.0.0/24 -j MASQUERADE

iptables -t nat -I POSTROUTING -o eth0 -s 10.0.0.0/16 -j MASQUERADE

9. Make these rules auto-start by adding those lines to a file like /etc/iptables.conf and then adding this line to /etc/rc.local

iptables-restore < /etc/iptables.conf

10. You're done with the server. 

11. On your Mac (client machine), install Tunnelbrick and use this client config, changing the location of the keys to the files you copied from the server and change the public IP to match the EIP of your OpenVPN server.

client

dev tun

proto tcp-client

# enter the server's hostname

# or IP address here, and port number

remote <elastic ip> 443

#resolv-retry infinite

nobind

persist-key

persist-tun

# Use the full filepaths to your

# certificates and keys

ca /Users/myan/.openvpn/ca.crt

cert /Users/myan/.openvpn/client1.crt

key /Users/myan/.openvpn/client1.key

ns-cert-type server

comp-lzo

#verb 6

12. Follow Tunnelbrick instructions using the OpenVPN client config above, and you're good.  Tunnelbrick can have issues on Mac.  If you do have issues, try Viscosity. 

13. Connect.  Now you can connect to all the 10.x.y.z private addresses in your VPC, provided that their security group allows inbound connections from the security group that you created for the OpenVPN server.

Note: Changed SSH on security group of my OpenVPN instance to 10.0.0.0/16 from open to world (0.0.0.0/0) now that I know it works.

14. Once it's working, roll-up that OpenVPN server into an AMI and the you can launch it into any VPC with a 10.0.0.0/16 network and connect to its EIP from Tunnelbrick, giving you access to all EC2 instances in the VPC through their private addresses.  No jump box, no EIPs – easy.  (Provided your security groups let in connections from the VPN server, which I do by default in all VPCs now.)

Auto Scaling Script

Auto Scaling with VPC 

This example uses the CLI version 1.0.61.2 (API 2011-01-01).  There is now a newer version of the CLI.

Overall notes:

1. Create an image (aka: gold image) that has a health check on it that consists of a simple static HTML page such as a ping.html or an index.html. 

2. The ELB (need an ELB before creating Auto Scaling group) hat has the instances running from your gold image AMI created in step 1. Could potentially (nothing is preventing you from doing) create a auto scale launch configuration (step 6 in process below) with an AMI that is different, with a different instance type as instance type is another parameter of your launch configuration, than the instance that will be part of your auto scaling group.  Both are possible but most will have the same AMI (gold image, so you know you have a health check) and the same instance type.

3. Before you create the ELB you should have instances running of the  gold image to make sure the ELB is working properly.  To provide true HA, make sure to put the instances in two AZs. (for example: us-west-2a, us-west-2b)

4. When creating elb that is using subnets, limitation on ELB is that only one subnet per AZ is allowed.  

5. When creating autoscaling group (as-create-auto-scaling-group)

 need to make sure to specific the VPC subnets and AZs.  The VPC subnets need to be in the AZs specified in the AZ parameter for command.

Steps:

Prework:

1. Identify the VPC : For example, VPC: vpc-9b120cf2

2. Add subnets : In this case, using two public subnets. I had to create the second one (subnet) which was private by default and I had to add an igw to make it public.

3. Subnets -

            A. Subnet: subnet-9f120cf6

            CIDR: 10.0.0.0/24   VPC: vpc-9b120cf2   Availability Zone: us-west-2a

            B. Subnet: subnet-8c130de5

            CIDR: 10.0.5.0/24   VPC: vpc-9b120cf2   Availability Zone: us-west-2b

4. Luanch two instances from AMI that has Apache installed with ping.html (or some other file) as a health check. One instance in subnet A and the other in subnet B. 

5. Create an ELB with the two instances across subnets.  ELB is healthy with two instances running in two AZs in two different public subnets. Limitation on ELB is that one subnet per AZ. 

6. Auto Scaling:

A. as-create-launch-config vpcautoscaling-as-lc --image-id ami-dcd344ec --instance-type t1.micro --key AutoScalingKey

B. as-create-auto-scaling-group vpcautoscaling-as-grp --launch-configuration vpcautoscaling-as-lc  --min-size 4 --max-size 12 --load-balancers VpcautoscalingAutoScalingELB --vpc-zone-identifier subnet-9f120cf6,subnet-8c130de5 --availability-zones us-west-2a,us-west-2b

C. as-describe-auto-scaling-groups --headers

output of command:

INSTANCE  INSTANCE-ID  AVAILABILITY-ZONE  STATE      STATUS   LAUNCH-CONFIG

INSTANCE  i-3af6e708   us-west-2b         InService  Healthy  vpcautoscaling-as-lc

INSTANCE  i-3cf6e70e   us-west-2a         InService  Healthy  vpcautoscaling-as-lc

INSTANCE  i-3ef6e70c   us-west-2a         InService  Healthy  vpcautoscaling-as-lc

INSTANCE  i-38f6e70a   us-west-2b         InService  Healthy  vpcautoscaling-as-lc

D. as-put-scaling-policy vpcautoscaling-scale-out-policy --auto-scaling-group vpcautoscaling-as-grp --adjustment=30 --type PercentChangeInCapacity

ARN: arn:aws:autoscaling:us-west-2:649163059618:scalingPolicy:69270ce4-3350-48f4-9d6f-71bc64225554:autoScalingGroupName/vpcautoscaling-as-grp:policyName/vpcautoscaling-scale-out-policy

E.  as-put-scaling-policy vpcautoscaling-scale-in-policy --auto-scaling-group vpcautoscaling-as-grp --adjustment=1 --type PercentChangeInCapacity

ARM - arn:aws:autoscaling:us-west-2:649163059618:scalingPolicy:d9a6780b-3319-4b00-98f1-e98dcfc68d82:autoScalingGroupName/vpcautoscaling-as-grp:policyName/vpcautoscaling-scale-in-policy

F. mon-put-metric-alarm --alarm-name AddCapacity --metric-name CPUUtilization --namespace "AWS/EC2" --statistic "Average" --evaluation-periods 6 --period 120 --threshold 80 --comparison-operator GreaterThanOrEqualToThreshold --dimensions "AutoScalingGroupName=vpcautoscaling-as-grp"  --alarm-actions arn:aws:autoscaling:us-west-2:649163059618:scalingPolicy:69270ce4-3350-48f4-9d6f-71bc64225554:autoScalingGroupName/vpcautoscaling-as-grp:policyName/vpcautoscaling-scale-out-policy

G. mon-put-metric-alarm --alarm-name RemoveCapacity --metric-name CPUUtilization --namespace "AWS/EC2" --statistic "Average" --evaluation-periods 2 --period 120 --threshold 40 --comparison-operator LessThanOrEqualToThreshold --dimensions "AutoScalingGroupName=vpcautoscaling-as-grp"  --alarm-actions arn:aws:autoscaling:us-west-2:649163059618:scalingPolicy:d9a6780b-3319-4b00-98f1-e98dcfc68d82:autoScalingGroupName/vpcautoscaling-as-grp:policyName/vpcautoscaling-scale-in-policy

H. as-describe-policies --auto-scaling-group vpcautoscaling-as-grp —headers