AWS common command line interface

The AWS Common Command Line interface can be found here:

http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

SQS message retention and visibility

There are two important attributes (http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Query_QuerySetQueueAttributes.html)  set when working with AWS SQS.  The first is the MessageRetentionPeriod and the other is the VisibilityTimeout.  These are two very different and distinct attributes.  The MessageRetentionPeriod is the length of time (in seconds) the message will stay in the queue (unless it is deleted).  The value can be 60 (1 minute) to 1209600 (14 days).  The VisibilityTimeout is the length of time no other applications can NOT see the message while an application is processing the message.  This can be set to be 0 to 43200 (12 hours). The longer the time this is set the longer you expect one process to be working on this message.

SQS automatically deletes messages that have been in a queue for more than maximum message retention period. The default message retention period is 4 days. However, you can set the message retention period to a value from 60 seconds to 1209600 seconds (14 days) with SetQueueAttributes

More on why messages are not deleted once read and the visibility of a message while it is being processed by an application:
http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AboutVT.html

AWS Auto Scaling for batch processing

Can I use Auto Scaling for scaling out a set of servers that run periodically (auto scaling of zero) ?  This is not the typical auto scaling use case but it is possible.  This would most likely be used for use cases such as batch processing where instances would be spun up to support processing that happens periodically.  Here are some ways to solve this use case:

• Auto Scaling, SQS and CloudWatch : You can but it would require to use AWS SQS with Auto Scaling and you would set up an alarm that uses SQS instead of CloudWatch or a custom metrics.  See more here:
  http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/as-using-sqs-queue.html
  It is possible to set up an auto scaling group with min EC2 size of ):
  http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/US_BasicSetup.html. You can issue this command: as-create-auto-scaling-group my-test-asg --launch-configuration my-test-lc --availability-zones us-east-1a --min-size 0 --max-size 10 --desired-capacity 1
• OpsWorks : OpsWorks provides a graphical tool for scheduling time-based auto scaling (e.g. “start an instance every Monday at 8:00am and shut it down at 5:00p.m.”).
• Auto Scaling and Cron : The auto scaling API supports a cron-style "recurrence" attribute for scheduled actions:
  
  
  http://docs.aws.amazon.com/AutoScaling/latest/APIReference/API_PutScheduledUpdateGroupAction.html

AWS exploring and managing services using Eclipse

There is an open source AWS plug-in for the Eclipse. You can access and administer your AWS infrastructure, including Amazon S3, Amazon DynamoDB, Amazon EC2, and more while developing your application.  Download here:

http://aws.amazon.com/eclipse/