Using aws-cli with Eucalyptus

Just before the holidays, Amazon released awscli, a new command-line interface for managing AWS resources.  The code is based on botocore, the core python library for the next major version of boto.  I took awscli for a spin to see if it worked with the Eucalyptus Community Cloud, and as is often the case, the answer was ... almost.

First, it's useful to understand the fundamental problems that awscli was trying to address.  The most obvious is profiles.  Cloud users deal with multiple regions, accounts, users, etc., and keeping separate configurations for each one is a hassle.  awscli uses a section-based config file format which allows for multiple profiles, each of which can reference it's own region, access keys, etc.

Another problem that this new code solves is the centralization of region and service data into JSON files which are easy to read, write, and parse.  See _regions.json and _services.json in botocore for examples.

What I found was that rather than trying to alter the existing data files, what I really wanted was a eucalyptus "provider" with its own JSON files.  I'll spare you all my trial-and-error, and simply explain what worked:

1. git clone https://github.com/boto/botocore.git
2. git clone https://github.com/a13m/aws-cli.git (note that this is my fork -- upstream is https://github.com/aws/aws-cli.git )
3. Install botocore and aws-cli however you prefer ( I use "python setup.py install --user" in each directory)
4. create a provider data directory, and a "euca" directory inside it.  I'll use /var/tmp/providers as the top directory.
5. create _regions.json and _services.json under the "euca" directory (the linked examples here should work for ECC verbatim)
6. symlink to botocore/data/aws/ec2.json and botocore/data/aws/iam.json in the euca provider directory
7. Create your ~/.awsconfig file (or whatever you'd like to call it):
   

   [default]
   aws_access_key_id=XXXXXXXXXXXXXXXXXXXX
   aws_secret_access_key=XXXXXXXXXXXXXXXXXX
   region=ecc
   provider_name=euca

8. export AWS_CONFIG_FILE=$HOME/.awsconfig
9. export AWS_DATA_PATH=/var/tmp/providers
10. try some commands, such as:

    aws ec2 create-volume --size 1 --availability-zone partner01
    aws ec2 describe-volumes
    aws ec2 describe-images

It may take a couple of iterations for the patch I've proposed to be accepted upstream, but in the meantime, I hope this is useful information.  As I've mentioned in the pull request, the solution is not ideal, as it requires that your default profile in a config file reference the euca provider, but I went for the least invasive fix first.  Note that even with this version, you can use profiles to group all of your eucalyptus cloud credentials into a single config file, and then have a second file for AWS profiles.  Switching back and forth is just a matter of setting AWS_CONFIG_FILE.