AWS EC2 instance user name

When logging into an EC2 instance using SSH, you may receive an error. You check to make sure you have the right instance name, IP address, PEM key etc. but it still fails. You may be using the incorrect user name. The EC2 instance user names by OS are listed here:

http://alestic.com/2014/01/ec2-ssh-username

For example, ubuntu servers are “ubuntu@” amazon is “ec2-user@” and other ones such as Debian are root@

EC2 instance public key invalid fix

I happened to do the same thing that is listed at this web page:  http://dltj.org/article/ec2-fix-authorized_keys/

The steps are on the web page. Here are the command:
1. aws ec2 create-volume --snapshot-id snap-YourVolumesSnapshot --size 10 -- availability-zone az-asdf-1a
Note: Assumes you have a current snapshot: aws ec2 create-snapshot --volume-id vol-beforevolumewasbad

 2. aws ec2 stop-instances --instance-id i-YourInstanceId

3. aws ec2 detach-volume --volume-id vol-badvolume

4. aws ec2 attach-volume --volume-id vol-goodvolume --instance-id i- YourInstanceId --device /dev/sda1

5. aws ec2 start-instances --instance-id i-YourInstanceId

Security on AWS

Good overview of top security tips for running on AWS:
http://stratumsecurity.com/2012/12/03/practical-tactical-cloud-security-ec2/

OpenVPN Server on AWS EC2

OpenVPN is a popular method to use to create an encrypted IPSec tunnel or SSL tunnel from client machines to AWS.  However, there is not much documentation or specifics on the web to walk through the set up OpenVPN on AWS and the client tools and configuration necessary.  Here are some step by step instructions for creating a encrypted SSL tunnel with caveats included:

1. Create the OpenVPN instance on AWS: Spin-up an Amazon Linux server (m1.small is fine) in a public subnet in the VPC you want to connect to. The VPC has to be a 10.0.0.0/16 network or you'll have to adjust these instructions a bit.  Put it in a separate security group with TCP 443 inbound from everywhere (for VPN connections) and TCP 22 inbound only from IPs you trust (for SSH admin)

      Note:

·   Need to create VPC with a public subnet .  I created a VPC with a public and private subnet as the whole idea behind this exercise is to have instances locked down from access and to the outside world by placing them in private subnets.

·   Need to create a security group.  Created a new security group OpenVPNConfig. For TCP port 443 (port of OpenVPN server),  needs to have a custom TCP rule for address in 10.0.0.0/16

·   Give 22 (SSH) to 0.0.0.0/0 for now just to get be able to work with instance to configure properly.  Once the OpenVPN server is running and tested, you will connect via VPN only so you should remove this rule.

·   Create a new key pair if desired.

2.Give the OpenVPN instance an EIP. You can do this by associating an ENI to the instance.

3.Login, and yum install openvpn

      A. sudo yum -y install openvpn

4.Do this: http://www.openlogic.com/wazi/bid/188052/From-Zero-to-OpenVPN-in-30-Minutes, with one caveat: when you do the build-dh command it'll generate a dh1024.pem file – that's the one you need, not 01.pem.

NOTE:

1.    Instruction for location say here /usr/share/doc/openvpn/examples/easy-rsa/2.0 but actually here: /usr/share/openvpn/easy-rsa/2.0

2.    Need to execute as root : sudo su

3.    Command used: cp -r /usr/share/openvpn/easy-rsa/2.0

/etc/openvpn/

4. I actually got 01.pem and 02.pem and dh1024.pem

Server Configuration – This is actually the same as the web page starting at section called Server Configuration.

5.Adjust the /etc/openvpn/openvpn.conf file to be something like this.  Note this uses TCP443 instead of UDP so it'll get through the AWS firewall.

port 443

proto tcp-server

dev tun

ca /etc/openvpn/keys/ca.crt

cert /etc/openvpn/keys/test-system.crt

key /etc/openvpn/keys/test-system.key

dh /etc/openvpn/keys/dh1024.pem

cipher BF-CBC

server 10.8.0.0 255.255.255.0

push "route 10.0.0.0 255.255.0.0"

comp-lzo

verb 6

ifconfig-pool-persist /etc/openvpn/ipp.txt

keepalive 10 120

status openvpn-status.log

My file:

port 443

proto tcp-server

dev tun

ca /etc/openvpn/2.0/keys/ca.crt

cert /etc/openvpn/2.0/keys/openvpn-system.crt

key /etc/openvpn/2.0/keys/openvpn-system.key

dh /etc/openvpn/2.0/keys/dh1024.pem

cipher BF-CBC

server 10.8.0.0 255.255.255.0

push "route 10.0.0.0 255.255.0.0"

comp-lzo

verb 6

ifconfig-pool-persist /etc/openvpn/ipp.txt

keepalive 10 120

status openvpn-status.log

6. Make it auto-start: sudo chkconfig openvpn on && sudo service openvpn start

Note:

1. The startup failed the first time I tried to start because of an error in my config file.  I had to run without chkconfig and with –config <config file location and name> to find out what error was.

7. Copy the client1.pem, client.crt and ca.crt from the server (or whatever you generated with build-key etc.) from the instructions you followed above… to your Mac.

            A. My files were: ca.crt, openvpn-system.crt, tom.crt, tom.key, 01.pem (seems to be associated with openvpn-system.crt), 02.pem (seems to be associated with tom.crt)

B. And Diffie-Hellman pem files: dh1024.pem

C. Copy using scp: scp -i /Users/tomlasz/Documents/Documents/EC2KeyPairs/OpenVPN.pem ec2-user@<elastic ip address>:/etc/openvpn/2.0/keys/tom.crt .

D. I needed to do a chmod 777 on the keys directory to get scp to work.

E. I needed to do a chmod 644 on the tom.key file to get scp to work on that file.

8. Setup IPTables on the OpenVPN server so that it'll do NAT out to the VPC for clients connecting to the VPN.  Here are all the commands you need assuming you used the instructions above.  As root, (sudo –s) run these on the server:

iptables -I FORWARD -i tun0 -o eth0 -s 10.8.0.0/24 -d 10.0.0.0/16 -m conntrack --ctstate NEW -j ACCEPT

iptables -I FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

iptables -t nat -I POSTROUTING -o eth0 -s 10.8.0.0/24 -j MASQUERADE

iptables -t nat -I POSTROUTING -o eth0 -s 10.0.0.0/16 -j MASQUERADE

9. Make these rules auto-start by adding those lines to a file like /etc/iptables.conf and then adding this line to /etc/rc.local

iptables-restore < /etc/iptables.conf

10. You're done with the server. 

11. On your Mac (client machine), install Tunnelbrick and use this client config, changing the location of the keys to the files you copied from the server and change the public IP to match the EIP of your OpenVPN server.

client

dev tun

proto tcp-client

# enter the server's hostname

# or IP address here, and port number

remote <elastic ip> 443

#resolv-retry infinite

nobind

persist-key

persist-tun

# Use the full filepaths to your

# certificates and keys

ca /Users/myan/.openvpn/ca.crt

cert /Users/myan/.openvpn/client1.crt

key /Users/myan/.openvpn/client1.key

ns-cert-type server

comp-lzo

#verb 6

12. Follow Tunnelbrick instructions using the OpenVPN client config above, and you're good.  Tunnelbrick can have issues on Mac.  If you do have issues, try Viscosity. 

13. Connect.  Now you can connect to all the 10.x.y.z private addresses in your VPC, provided that their security group allows inbound connections from the security group that you created for the OpenVPN server.

Note: Changed SSH on security group of my OpenVPN instance to 10.0.0.0/16 from open to world (0.0.0.0/0) now that I know it works.

14. Once it's working, roll-up that OpenVPN server into an AMI and the you can launch it into any VPC with a 10.0.0.0/16 network and connect to its EIP from Tunnelbrick, giving you access to all EC2 instances in the VPC through their private addresses.  No jump box, no EIPs – easy.  (Provided your security groups let in connections from the VPN server, which I do by default in all VPCs now.)

EC2 instances adding SSH users

Each EC2 instance has a ec2-user user which can be used access (via SSH) the EC2 Linux instance.  You can add addition SSH users by following these instructions:  http://www.e-zest.net/blog/how-to-add-ssh-users-in-amazon-linux/

Bulk loading data to AWS from on premise

I spoke about bulk loading data to AWS in this blog post:

http://cloudconclave.blogspot.com/2013/04/on-premise-application-replication-and.html

A couple other options I did not mention are:

1. Apera - Asperasoft Company has developed a proprietary file transfer protocol based on UDP, which has shown to introduce very high-speed file transfer experience over the Internet.

2. http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/UsingEMR_s3distcp.html   Apache DistCp is an open-source tool you can use to copy large amounts of data. DistCp uses MapReduce to copy in a distributed manner  You can also use S3DistCp to copy data between Amazon S3 buckets or from HDFS to Amazon S3.  Since it is based upon MapReduce, it is most applicable when moving HDFS and other map reduce files as you will obviously have MapReduce installed in your environment. 

Mindterm SSH on Mac

I moved from a Windows laptop to a Mac and just experienced this issue:

https://forums.aws.amazon.com/thread.jspa?threadID=117962

I did as suggested and used the following for my private key path:

../Downloads/EssentialsLab.pem

Connecting to AWS EC2 using SSH and SFTP

As I mentioned in a previous blog, you can only use SFTP (can not use FTP) to connect to an EC2 instance.  The other thing you need to have correct is an entry in the SSH config file for your  private key (of the key pair value for the EC2 instance).   In the Unix/Mac world, this file is called config and it is located in the ~/.ssh directory.  The following entry needs to be added to this config file:

identityFile ~/tomlaszKeys/PemFileName.pem

Adding this entry will also allow you to connect via SSH without having to pass in the private key file location:

ssh ec2-user@ec2-23-23-237-99.compute-1.amazonaws.com

Instead of:

ssh ec2-user@ec2-23-23-237-99.compute-1.amazonaws.com

 -i ~/PemfileName.pem

AWS EC2 amazon sftp only for web site upload pem pkk ftp ssh

The most difficult part of creating my AWS EC2 web site (other then the 'coding' part ... more on this tomorrow) was getting the files loaded.
1. SFTP can only be used. FTP will not work! Discussion is here: SFTP with EC2 (or can use scp)
2. PEM file to a .PPK file - I used Filezilla as described here: Converting PEM to PPK file. Seems as though sftp does not use pem file like SSH does.
3. login in to EC2 instance using SSH and change user to root ( sudo su ) and change the directory with location of you web pages (/var/www/html) to '755' (chmod
-R 755 /var/www/html). This is because you will be using your ec2-user name to upload using SFTP.
4. login into SFTP using ec2-user and your instance name: sftp ec2-user@
Note: These file go into the directory /var/www/html.