Thursday, September 3, 2020

AWS CLI installation in 10 minutes

Image

Almost every Cloud provider has a Command Line Interface (CLI) which is a unified tool to manage cloud resources. In previous post I described how to configure Oracle Cloud infrastructure CLI. This time my focus is its AWS equivalent as I intend to explore different ways of provisioning instances in all major Cloud shops (OCI,AWS,Azure,GCP),  and since my next article will depend on AWS-CLI, I will start with the configuration piece which I found easier than for oci-cli.


Context: This is part of a series of 4 articles that will build quick hands-on experiences on AWS provisioning.

  1. Install and configure aws-cli.
  2. Launch an instance using aws-cli.
  3. Launch an Instance using Terraform.
  4. Launch an instance using  aws ansible modules.

Requirement


Whether you install aws cli on windows or on Linux the basic install will always require 2 elements:


I. AWS CLI Installation (version1)

  • Windows

    1- Download and execute the following aws-cli installer https://s3.amazonaws.com/aws-cli/AWSCLISetup.exe
    2- Follow the on-screen instructions. The AWS CLI version 1 will automatically install the correct version as follows
        C:\Program Files\Amazon\AWSCLI
      for 64-bit system
        C:\Program Files (x86)\Amazon\AWSCLI for a 32-bit system
    .
    3- Open Environment Variable window and add the bin directory to the PATH variable
              ==> ”C:\Program Files\Amazon\AWSCLI\bin”


    - Run the version command to confirm that AWS CLI was installed correctly.

    C:\Users\kosse> aws --version
    aws-cli/1.18.130 Python/3.6.0 Windows/10 botocore/1.17.53 

    C:\Users\kosse> where aws C:\Program Files\Amazon\AWSCLI\bin\aws.exe
  • Linux/MacOS (bundled installer) 

    I also had aws-cli installed on my windows subsystem for Linux (WSL) where the installation was done as follows :
    brokedba~$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
    brokedba~$ unzip awscli-bundle.zip
    brokedba~$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
    --- Run the version command to confirm it was installed correctly. brokedba~$ aws --version aws-cli/1.18.130 Python/2.7.12 Linux/4.4.0-18362-Microsoft botocore/1.17.53
  • Installation using Python Package manager (pip) :

    You might also install aws cli using if the following pip command (already installed for 2.7.9+) .
  • brokedba~$ pip --version
    pip 20.2.2 from /home/brokedba/.local/lib/python2.7/site-packages/pip (python 2.7) 
    brokdba:~$ pip install awscli
    --- Run the version command to confirm it was installed correctly. brokedba~$ aws --version aws-cli/1.18.130 Python/2.7.12 Linux/4.4.0-18362-Microsoft botocore/1.17.53
    brokedba@brokdba:~$ which aws
    /usr/local/bin/aws

II. Configure AWS CLI


Once your AWS free Tier account is created, python and aws cli installed you will need to gather the required credentials as shown in the below setup tasks:

  • Create the Access Key
    AWS-CLI will need the access key to make API calls to AWS.
    On your Console, go to the profile menu on the top right of the page and click on My security Credentials.
                                                                           AWS Credentials
  • Click Create Access Keys under "Access keys" section. You have the right to 2 Access keys as a Free Tier user

                                              Create access key
  • Download the Access key
    Click Download Key file as it's only available at creation and not later. The csv file will contain the key Id and secret key.
     

                                                            New access key
Note: You can always delete keys to recreate new ones if you reach the max amount of access keys or lost the key file.    

  • Run AWS configure

    Now that you have installed aws cli along with the access key info gathered in your csv file, you can finally configure your aws-cli with just the key id and the access key (region and output format are not credentials).To do so run the following:
    $ aws configure 
    Access Key ID:
    AKxxxxxxxxxxx
    Secret Access Key:
    Dsxxxxxxxxxxxxxxxxxxxxxxx
    Default region name [us-east-1]:
    Default output format [table]:
    
    You can also add a profile when you have multiple aws accounts to manage
    $ aws configure --profile brokedba
    Access Key ID:
    AKxxxxxxxxxxx
    Secret Access Key:
    Dsxxxxxxxxxxxxxxxxxxxxxxx
    Default region name [us-east-1]:
    Default output format [table]:

  • - Below are few information related to the current configuration and the files that were updated during setup:
      Config files :
         ~/.aws/credentials
      ==> Supported by all SDKs and contain credentials only
         ~/.aws/config          ==> Supported by CLI only and can contain credentials

    $ aws configure list
          Name                    Value             Type    Location
          ----                    -----             ----    --------
       profile                             None    None
    access_key     ****************J2WA shared-credentials-file
    secret_key     ****************H5Bn shared-credentials-file
        region                us-east-1      config-file    ~/.aws/config
    $ cat ~/.aws/credentials [default] aws_access_key_id = AKIXXXXXXXXXXXXXXXXXJ2WA aws_secret_access_key = DsXXXXXXXXXXXXXX5Bn [brokedba] aws_access_key_id = AKIXXXXXXXXXXXXXXXXXJ2WA aws_secret_access_key = DsXXXXXXXXXXXXXX5Bn $ cat ~/.aws/config [default] output = table region = us-east-1 [profile brokedba] output = table region = us-east-1

III.Test your first API request


Few notions worth reminding before hitting the terminal with your favorite  aws-cli requests :    
A. Command structure : is based on the below components

    $ aws <AWS service> <operation to perform> [one or more options & parameters]   

       Parameters: 
        Will be followed by their values, for example when specifying an instance id we want to describe or defining a name for a created
        key- pair. The value type can also vary (string, integer, JSON, list, binary,…)

  •  Options :
              
    1- “-- output” : will format AWS CLI output into Json, yaml, Table, or text (raw).
               2- “-- query” : Allows to choose the list of fields to return in the response. It can be used  to do some simple filtering.
               3- “ -- filters” : Is the condition used to specify which resources you want described or listed.


B. Filters vs Query
:
The --query option relies on JMSPath and its filtering is done at client side while --filters does it at server level which is way faster and more efficient. I personally use filters to narrow my research and query to specify which field I want to display.

- To demonstrate the nuance, here’s an example where we filter an aws region using each option (filters and query)

    $ aws ec2 describe-regions --query 'Regions[?RegionName==`us-west-2`]' 
    $ aws ec2 describe-regions --filters "Name=region-name,Values=us-west-2"


Examples


There are few requests that you can run to test your connectivity and practice with aws-cli. Below describe-* commands are good examples to start with.

  • Describe and list aws regions using describe-regions subcommand and --query option
    $ aws ec2 describe-regions --query 'Regions[]'
    -------------------------------------------------------------------------------
    |                               DescribeRegions                               |
    +-----------------------------------+-----------------------+-----------------+
    |             Endpoint              |      OptInStatus      |   RegionName    |
    +-----------------------------------+-----------------------+-----------------+
    |  ec2.eu-north-1.amazonaws.com     |  opt-in-not-required  |  eu-north-1     |
    |  ec2.ap-south-1.amazonaws.com     |  opt-in-not-required  |  ap-south-1     |
    |  ec2.eu-west-3.amazonaws.com      |  opt-in-not-required  |  eu-west-3      |
    |  ec2.eu-west-2.amazonaws.com      |  opt-in-not-required  |  eu-west-2      |
    |  ec2.eu-west-1.amazonaws.com      |  opt-in-not-required  |  eu-west-1      |
    |  ec2.ap-northeast-2.amazonaws.com |  opt-in-not-required  |  ap-northeast-2 |
    |  ec2.ap-northeast-1.amazonaws.com |  opt-in-not-required  |  ap-northeast-1 |
    |  ec2.sa-east-1.amazonaws.com      |  opt-in-not-required  |  sa-east-1      |
    |  ec2.ca-central-1.amazonaws.com   |  opt-in-not-required  |  ca-central-1   |
    |  ec2.ap-southeast-1.amazonaws.com |  opt-in-not-required  |  ap-southeast-1 |
    |  ec2.ap-southeast-2.amazonaws.com |  opt-in-not-required  |  ap-southeast-2 |
    |  ec2.eu-central-1.amazonaws.com   |  opt-in-not-required  |  eu-central-1   |
    |  ec2.us-east-1.amazonaws.com      |  opt-in-not-required  |  us-east-1      |
    |  ec2.us-east-2.amazonaws.com      |  opt-in-not-required  |  us-east-2      |
    |  ec2.us-west-1.amazonaws.com      |  opt-in-not-required  |  us-west-1      |
    |  ec2.us-west-2.amazonaws.com      |  opt-in-not-required  |  us-west-2      |
    +-----------------------------------+-----------------------+-----------------+
  • List the access keys for an aws account :
    $ aws iam list-access-keys --query  "AccessKeyMetadata"
    ------------------------------------------------------------------------
    |                            ListAccessKeys                            |
    +-----------------------+------------------------+---------+-----------+
    |      AccessKeyId      |      CreateDate        | Status  | UserName  |
    +-----------------------+------------------------+---------+-----------+
    |  AXXXXXXXXXXXXXXXXXWA |  2020-06-25T07:13:44Z  |  Active |  brokedba |
    |  AXXXXXXXXXXXXXXXXZOA |  2020-09-02T00:24:17Z  |  Active |  brokedba |
    +-----------------------+------------------------+---------+-----------+
  • List the existing buckets within the s3 account:
  • $ aws s3 ls
    2020-06-07 01:51:08 brokebucket
    2020-06-13 20:01:06 brokereportbucket
  • Describe existing instances in the default region and give a custom name for each field inside the braces:

  • $ aws ec2 describe-instances --query 'Reservations[].Instances[].{VPCID:VpcId,Subnet:SubnetId,image:ImageId,Rootdev:RootDeviceName,AZ:Placement.AvailabilityZone,PrivIP:PrivateIpAddress}'
    ----------------------------------------- | DescribeInstances | +----------+----------------------------+ | AZ | us-east-1a | | PrivIP | 192.168.10.45 | | Rootdev | /dev/sda1 | | Subnet | subnet-08b49f9682c5da2b6 | | VPCID | vpc-096b461ebe9d06ff3 | | image | ami-01861c2f0a2adfdb7 | +----------+----------------------------+
  • Note: If you don’t like the table output you can always go for a text or Json  layout using --output option

Aliases

AWS has made a CLI alias repository available in their GitHub.Some of them can help get a grasp of common queries like describing security groups, open public ports,running instances etc. You can quickly install it by running the below commands:
 
$ git clone https://github.com/awslabs/awscli-aliases.git
$ mkdir -p ~/.aws/cli 
$ cp awscli-aliases/alias ~/.aws/cli/alias


IV. Upgrade to Version 2

AWS CLI version 2 is available since last February and is the recommended version. The upgrade to v2 is unfortunately not direct as the existing v1 has to be uninstalled first but the configuration will still be in place after upgrading (No pip install possible in v2).
You can quickly upgrade by following the bellow steps:

  • Windows

    - Uninstall aws-cli v1: Type appwiz.cpl in your cmd box and & hit uninstall for the entry named “AWS Command Line Interface”
    - Download & Run awscli v2 installer (64bits only): https://awscli.amazonaws.com/AWSCLIV2.msi 
    • C:\Users\kosse>  aws --version
      aws-cli/2.0.45 Python/3.7.7 Windows/10 exe/AMD64
  • Linux
    • 1-– uninstall v1
      $
      sudo rm -rf /usr/local/aws
      $ sudo rm /usr/local/bin/aws
      2-- install v2
      $ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
      $ unzip awscliv2.zip
      $ sudo ./aws/install -i /usr/local/aws-cli -b /usr/local/bin

      3-- Run the version command to confirm v2 was installed correctly

      $ aws --version
      aws-cli/2.0.45 Python/3.7.3 Linux/4.4.0-18362-Microsoft exe/x86_64.ubuntu.16
  • Enable autocomplete (v2)
    • $ complete -C aws_completer aws
  • Explore awscli wizard (v2)



Conclusion:


In this tutorial we learned how to install and configure aws-cli v1 which took 5 minutes then upgraded to v2 that took,well... few more ;). We also described the command syntax and tried few describe requests using aws-cli.
The new features of aws-cli version2 (interactivity,SSO,autocomplete,wizards…) seem to bring more value to the tool which makes it worth a try.   
Just remember to use --filters as a condition and --query as a select to reduce the overhead/response time on your cli requests. Finally, feel free to consult AWS CLI Command Reference for more details and examples on aws-cli requests.

Cheers.

2 comments:

  1. wonderful article. Very interesting to read this article.
    DevOps Training
    DevOps Online Training

    ReplyDelete
  2. Would you be interested in trading links or maybe guest writing a blog post or vice-versa?
    aws online training

    ReplyDelete