IAM : AWS services not supported and tagging support

I often get the question of which services do not support IAM?  Some services and features do not support IAM and must use the AWS root account credentials:

• Vulnerability Scanning Coordination Form
• AWS Workspaces
• Visibility of all Data Pipelines
• Specific account management pages, such as account profile, AWS account security credentials, payment methods and management of consolidated billing 
• AWS DevPay 
• CloudFront keypair creation

A full list of supported services is here:
http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_SpecificProducts.html

This page also provides a great list of IAM resource level permissions support, tagging, and STS.

EMR in a private subnet

When running EC2 instance or other AWS services in a private subnet, you need a NAT to access S3. 

You can not use a NAT when using EMR:

http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-plan-vpc-subnet.html

Because access to and from the AWS cloud is a requirement of the cluster, you must connect an Internet gateway to the VPC subnet hosting the cluster. If your application has components you do not want connected to the Internet gateway you can launch those components in other subnets you create within your VPC. In addition, because of the need to access the AWS cloud, you cannot use Network Address Translation (NAT) when you are running Amazon EMR on a VPC.

AWS S3 temporary access to S3 bucket files

Sharing files using S3 can be easily be done in the S3 buckets permission by adding 'everyone' to the permissions on the buckets and files. However, you probably don't want any person in the world accessing your files. Here are a couple of quick and easy methods.

1. Temporary URL : Use a tool like S3Fox and you can get a pre-signed URL with a time limit.  This limitation with this approach is that if you have a large number of files you need to generate and send a URL for each file.

2. IAM User : The policies you need to use can be found here: http://mikeferrier.com/2011/10/27/granting-access-to-a-single-s3-bucket-using-amazon-iam/

You also need to provide the URL for IAM user sign-in URL for your account, the IAM user, and the IAM user password. Remember, you need to add a password to the user after you create it.

It is then easy to delete this user or change the password to make sure that if this URL, account and password gets into the wrong hands your files are not downloaded.

AWS IAM user creation and access

IAM user set up can be found here:
http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_SettingUpUser.html

Give them 'Power User' if they need to create instances and fully utilize AWS services.

They will use a different URL (not aws.amazon.com) then you use to access your account.  More here:
http://docs.aws.amazon.com/IAM/latest/UserGuide/WhatUsersNeedToKnow.html

AWS Security 101 for Oracle DBAs, Developers and Architects

Oracle DBAs understand securing data in transit and at rest, but they don't have to deal with file level encryption, security of the databases, firewalls, denial of service attacks, SQL injection attacks, and other OS level security.

General infrastructure security concepts:
1. Some networking concepts such as VPC, VPN, and IPSec also apply to the security realm.  More on these concepts can be found here: http://cloudconclave.blogspot.com/2013/07/aws-network-101-for-oracle-dbas.html
1. SSL : The Secure Sockets Layer (SSL) is a commonly-used protocol for managing the security of a message transmission on the Internet. SSL uses the public-and-private key encryption system from RSA, which also includes the use of a digital certificate.  
2. ACLs : Access Control Lists (ACLs) specifies which users or system processes are granted access to objects, as well as what operations are allowed on given objects.

3. MFA : Multifactor authentication (MFA) is a security system in which more than one form of authentication is implemented to verify the legitimacy of a transaction. The goal of MFA is to create a layered defense and make it more difficult for an unauthorized person to access a computer system or network.  An MFA device can be a Gemalto token (http://onlinenoram.gemalto.com/) or even an iPhone.  http://cloudconclave.blogspot.com/2013/06/mfa-made-easy.html
4. Bastion Host :A bastion host is a special purpose computer on a network specifically designed and configured to withstand attacks. Information on bastion hosts on AWS with Oracle on these two posts: http://cloudconclave.blogspot.com/2013/05/aws-bastion-host-as-single-point-of.html http://cloudconclave.blogspot.com/2013/05/dba-and-developer-access-to-oracle.html
5. iptables : iptables are the tables provided by the Linux kernel firewall.  These firewall rules make it possible for administrators to control what hosts can connect to the system, and limit risk exposure by limiting the hosts that can connect to a system.  Information on iptables for security on AWS here: http://cloudconclave.blogspot.com/2013/06/aws-security-with-iptables.html

6. IDS : An intrusion detection system (IDS) is a device or software application that monitors network or system activities for malicious activities or policy violations and produces reports to a management station. Some systems may attempt to stop an intrusion attempt but this is neither required nor expected of a monitoring system.
7. IPS : Intrusion prevention systems (IPS), also known as intrusion detection and prevention systems (IDPS), are network security appliances that monitor network and/or system activities for malicious activity. The main functions of intrusion prevention systems are to identify malicious activity, log information about this activity, attempt to block/stop it, and report it.   Intrusion prevention systems are considered extensions of intrusion detection systems because they both monitor network traffic and/or system activities for malicious activity.

8. DoS :  A denial-of-service attack (DoS attack) or distributed denial-of-service attack (DDoS attack) is an attempt to make a machine or network resource unavailable to its intended users.  IPS, iptables, AWS security groups, NACLs, and bastion hosts are all ways to prevent DoS attacks.
9. Penetration testing : A penetration test, occasionally pentest, is a method of evaluating computer and network security by simulating an attack on a computer system or network from external and internal threats.

AWS specifics. You must be familiar with all of these concepts in order to perform basic actions on AWS and EC2:

1. Access key and secret key : The access key is used to access AWS using the CLI and TEST API.  The REST and Query APIs use your access keys as the credential.You might be using a third-party product such as S3Fox or ElasticWolf that requires your access keys (because the product itself makes AWS requests for you). Although access keys are primarily used for REST or Query APIs, Amazon S3 and Amazon Mechanical Turk also use access keys with their SOAP APIs. Your Access Key ID identifies you as the party responsible for service requests. You include it in each request, so it's not a secret.The secret key provide anyone that possesses them incredible power to perform delete, terminate, start etc actions on your AWS resources (EC2, ELB, S3 etc) so be very careful with them. Don't e-mail it to anyone, include it any AWS requests, or post it on the AWS Discussion Forums. No authorized person from AWS will ever ask for your Secret Access Key.
2. x509 : X.509 certificates are based on the idea of public key cryptography. It is used for \making requests to AWS product SOAP APIs (except for Amazon S3 and Amazon Mechanical Turk, which use access keys for their SOAP APIs).  SOAP services are being defocused so x509 will not be used as much moving forward.
3. Key pair file (SSH pem file) : You use an Amazon EC2 key pair (aka: PEM file) each time you launch an EC2 Linux/UNIX or Windows instance. The key pair ensures that only you have access to the instance.Each EC2 key pair includes a key pair name, a private key, and a public key.  PEM is a file format that may consist of a certificate (aka. public key), a private key or indeed both concatenated together. Don't pay so much attention to the file extension; it means Privacy Enhanced Mail, a use it didn't see much use for but the file format stuck around. more on using PEM with EC2 here http://cloudconclave.blogspot.com/2012/09/connecting-to-aws-ec2-using-ssh-and-sftp.html
4. Security Groups : A security group acts as a firewall that controls the traffic allowed to reach one or more instances. When you launch an instance, you assign it one or more security groups. You add rules to each security group that control traffic for the instance. You can modify the rules for a security group at any time; the new rules are automatically applied to all instances to which the security group is assigned.

These AWS security concepts are not necessary but one you get beyond the 'playing around phase' of working with AWS these security components are key to working with AWS:
1. ARNs : Amazon Resource Names (ARNs) uniquely identify AWS resources. We require an ARN when you need to specify a resource unambiguously across all of AWS, such as in IAM policies, Amazon Relational Database Service (Amazon RDS) tags, and API calls.  Here is an example ARN: 
<!-- Amazon RDS tag -->
arn:aws:rds:eu-west-1:001234567890:db:mysql-db
ARNs are used extensively with IAM to place security/access policies on AWS services.
2. IAM : AWS Identity and Access Management (IAM) enables you to securely control access to AWS services and resources for your users. Using IAM you can create and manage AWS users and groups and use permissions to allow and deny their permissions to AWS resources. More details here: http://cloudconclave.blogspot.com/2012/10/aws-iam-service.html, http://cloudconclave.blogspot.com/2013/05/aws-getting-started-with-groups-and.html
3. NACLs : Network ACLs operate at the subnet level and evaluate traffic entering and exiting a subnet. Network ACLs can be used to set both Allow and Deny rules. Network ACLs do not filter traffic between instances in the same subnet. In addition, network ACLs perform stateless filtering while security groups perform stateful filtering.

 4. S3 SSE :  http://cloudconclave.blogspot.com/2013/07/s3-sse-without-request-header.html Server-side encryption is about data encryption at rest, that is, Amazon S3 encrypts your data as it writes it to disks in its data centers and decrypts it for you when you access it. As long as you authenticate your request and you have access permissions, there is no difference in the way you access encrypted or unencrypted objects. Amazon S3 manages encryption and decryption for you. For example, if you share your objects using a pre-signed URL, the pre-signed URL works the same way for both encrypted and unencrypted objects.

5. Data Encryption : AWS does not provide encryption of EBS (Elastic Block Storage) . More details on a couple of vendors that provide solutions here: http://cloudconclave.blogspot.com/2013/04/ebs-volume-encryption.html

EC2 instances adding SSH users

Each EC2 instance has a ec2-user user which can be used access (via SSH) the EC2 Linux instance.  You can add addition SSH users by following these instructions:  http://www.e-zest.net/blog/how-to-add-ssh-users-in-amazon-linux/

DBA and developer access to Oracle hosted on AWS

Here are three common methods used to limit access to the AWS environment for DBAs and developers:

1. Bastion host : A bastion can be used as a jump box' / proxy server. Developers and DBAs would be given access using SSH and than use other credentials to log into the web, application, and database servers. More on bastion host security can be found here: http://cloudconclave.blogspot.com/2013/05/aws-bastion-host-as-single-point-of.html.  There is the cost of the EC2 instance that is the bastion host and data transfer out costs.
2. VPN with customer gateway and virtual private gateway.  In the case, you create a VPN tunnel.  The costs here are the VPN hardware on your side (customer gateway), cost of virtual private gateway (VPG), and costs of VPN connections and data transfer out of AWS. More on VPN costs here (this assumes this option): http://cloudconclave.blogspot.com/2013/05/vpn-costs-for-connections-and-data.html
3. OpenVPN : You do not incur the cost of hardware on your side and the VPG on the AWS side.  You still have the cost of the data transfer out.  You would also incur the cost of the EC2 instances that is running an open source VPN software stack (in this case OpenVPN).

I am sure there are other methods as well.

Could also use these constructs to provide secure integration from your on premise or third party applications (SFTP for flat file integration, VPN for web services).

AWS getting started with groups and users

A common question is when first setting up and AWS environment is how to prevent developers, OS administrators, DBAs, architectures and all the different roles you may have in your organization from having the correct privileges.  You use IAM group and users.  You would create a developer group.  Ignore roles to start with as these are for AWS services to access other services  (example: EC2 accessing S3) and cross account access.  Then add policies to the group (use policy generator or select a template). Then add each developer as individual users and adding them to the developer group.

Good resource for all of your questions….You can explicitly manage roles and policy

http://docs.aws.amazon.com/IAM/latest/UserGuide/IAMBestPractices.html

http://docs.aws.amazon.com/IAM/latest/UserGuide/cross-acct-access-walkthrough-creategroup.html

Nice blog entry here:
https://forums.aws.amazon.com/message.jspa?messageID=197920

CloudFormation access denied error message

In the event you get this message when a CloudFormation script:

AccessDenied. User doesn't have permission to call ec2:RunInstances, this could mean that the AMI is not available or you don't have access to it.  The error is not intuitive.  Check to make sure the AMI exists or if you have access to it.

Oracle Enterprise Manager with AWS Database Instance

Oracle Enterprise Manager 11g Database Control support was just announced this May. It seems like it may be a long time in coming but Oracle Database support on the AWS cloud was first released just a few years ago in September 2008. Here is a great blog entry on using OEM with Oracle on AWS:

AWS OEM for Oracle DB

Remember that without https in front of the URL OEM will not respond.https://ec2-99-55-139-99.eu-west-1.compute.amazonaws.com:1158/em