Geekly Articles each Day
⬆️ ⬇️

awless.io, mighty CLI for AWS

image

Some very clearly prefer the CLI (source: Imgur)



Disclaimer : This is a translation of the article, the original can be found here .



While cloud technologies were gaining popularity, the Amazon Web Services (AWS) platform, which today provides work for many Internet giants and sets standards for how many cloud services should be organized, has become the undisputed leader in the cloud computing market. To understand how to use the ever-growing number of services and the opportunities they provide, the DevOps engineer must keep up with the pace of cloud computing development that Amazon is asking, and this is becoming a real challenge.

')

DevOps often wonder how to better manage the fast growing AWS infrastructure. They ponder whether it is better to use GUI ( AWS console ) or CLI ( AWS CLI ) for this. Or maybe use both?



We present to you the third option, created to facilitate the work of DevOps: awless.io is a new open source CLI for AWS, written in Go. The project was approved by Jeff Barr, a rock star for AWS, and in just a few months gathered more than 2.5K stars on GitHub. In this article, I will explain why Wallix, the company I work for, created awless.io, and why we believe that this tool can help DevOps.



What, by the way, are these tweets talking about:





Love and hate GUI



image

AWS provides many services.



System administrators tend to hate the GUI. The Linux Foundation considers them to be a lot of newbies. Is the AWS Console an exception? The Console provides the ability to manage all the available services and allows you to use wizards to solve various tasks. We appreciate the features provided by the Console, especially when performing one-time tasks, because visualization of the process in this case is very useful. But the solution of standard, repetitive tasks using the GUI takes too much time and can ruffle experienced users.



image

We are in the sixth step in the AWS Console and we still have not created an instance



Using the GUI is difficult to automate actions, leave notes for yourself or interact with a colleague to perform a particular operation. Remember how you last explained to your colleague (or, for example, your mother) by phone, how to do something using the web interface, while digging through it? This is really annoying.



CLI aws difficulties



CLI, on the other hand, is great for routine tasks. After pre-setting using the CLI, you can do anything you want without manually entering the system and without touching the mouse.



For example, this command is enough to run the instance t1.micro: 



find \ -name "sws" > aws ec2 run-instances --image-id ami-xxxxxxxx --count 1 --instance-type t1.micro \ --key-name keypair-xxxxxxxx --security-groups sg-xxxxxxxx


Note that remembering the Amazon Machine Image ID (AMI) or the name of a key pair may not be easy, but this problem is solved using the alias shell command. For example, first we find the necessary security group: 



 > aws ec2 describe-security-groups --group-name "awless-secgroup"


and we get the group ID from the output, which by default should look something like this: 



 { "SecurityGroups": [ { "IpPermissionsEgress": [ { "PrefixListIds": [], "FromPort": 80, "IpRanges": [ { "CidrIp": "0.0.0.0/0" } ], "ToPort": 80, "IpProtocol": "tcp", "UserIdGroupPairs": [], "Ipv6Ranges": [] }, { "PrefixListIds": [], "FromPort": 443, "IpRanges": [ { "CidrIp": "0.0.0.0/0" } ], "ToPort": 443, "IpProtocol": "tcp", "UserIdGroupPairs": [], "Ipv6Ranges": [] } ], "Description": "awless-secgroup", "IpPermissions": [ { "PrefixListIds": [], "FromPort": 22, "IpRanges": [ { "CidrIp": "0.0.0.0/0" } ], "ToPort": 22, "IpProtocol": "tcp", "UserIdGroupPairs": [], "Ipv6Ranges": [] } ], "GroupName": "awless-secgroup", "VpcId": "vpc-00b68c65", "OwnerId": "519101999238", "GroupId": "sg-687a5d0e" } ] }


Although this approach seems to be more “advanced” than using the AWS Console, it is not always convenient and effective. When using aws, CLI has to constantly switch from entering commands to performing other actions, such as copying the output of previous commands or editing JSON snippets. Existing open source alternatives to aws CLI only slightly alter the syntax of operations, add highlighting, autocompletion, and other whistles.



Tools like AWS CloudFormation are convenient for creating stacks, but writing the appropriate templates is a separate task, and the templates themselves cannot always replace the manual execution of several commands. Once we were tormented by trying to find resources created using CloudFormats using aws CLI. Guessing the right place in kilobytes of JSON, where you need to register Id, is not an easy task. In the end we gave up and used the GUI.



Awless.io phenomenon



awless.io is an alternative CLI for AWS, in which ease of use comes to the fore, in front of exhaustive functionality. With this tool, we strive to simplify 90 percent of possible tasks by modifying existing commands.



awless.io is not just a front-end for AWS CLI: this is a new console interface written on Go using the official AWS Go SDK. This is a fully open-source project released under the Apache license. Currently awless.io supports the following AWS services: IAM, EC2 (including ELBv2), S3, Route53, RDS, SQS, SNS, Cloudfront, Cloudwatch, CloudFormation, and Lambda. In the near future, expansion of this list is planned.



Originally, awless.io was developed by the Wallix innovation department of my company to solve a number of internal problems. De facto, Wallix sponsored the development, but essentially awless.io is a project created by the developer community, and your participation in it is strongly encouraged.



Benefits of awless.io



When creating awless.io, we tried to follow the principles laid down by popular tools such as git; Most commands are: 



 awless [verb] [noun] [parameter=value ...]


For example, all of these are possible commands: 



 #    awless list users #    awless create instance #     awless attach user name=my-user-name group=my-group #   awless run https://example.com/create_vpc.aws


The basic principles of awless.io include:





awless.io emphasizes the use of the same verbs (such as list , create , delete ) in the syntax of different operations, which results in a small (and easy to remember) set of commands used. Syntax is unified for different services. For example, you can write: 



 awless list [instances|users|buckets|records]


instead: 



 aws ec2 describe-instances aws iam list-users aws s3api list-buckets aws route53 list-resource-record-sets


Getting started with awless.io



awless.io is available on github. We also distribute it as compiled packages and through Homebrew for macOS: 



 brew tap wallix/awless; brew install awless


Building from source is very simple, provided that Go is installed on the system: 



 go get -u github.com/wallix/awless


When you first start, awless.io will ask for credentials or, if aws-cli is installed and configured on your system, it will automatically reconfigure them from its configuration. In addition, it is worth setting autocompletion for bash or zsh .



Explore AWS Infrastructure



awless.io provides DevOps with a wide range of options for navigating the AWS deployed infrastructure. The two main commands for this are list and show . For each type of resource, awless.io first displays the most relevant information.



Listing objects



awless.io allows you to list all kinds of AWS objects, such as users, security groups, or instances: 



 > awless list users | ID ▲ | NAME | LASTUSED | CREATED | |-----------------------|-----------|----------|-----------| | AIDAI37GETCRRVNLDKFYQ | hbinsztok | 8 days | 10 months | | AIDAI3FX74KAQQKIOM2XY | awless-ts | 13 weeks | 4 months | […]


If the width of the terminal does not allow to display the data completely, then the columns are cut off automatically, and a warning is displayed on the screen that such information has been hidden: 



 > awless list securitygroups | ID â–˛ | VPC | INBOUND | |-------------|--------------|-------------------------------------| | sg-1d578b64 | vpc-00b68c65 | [0.0.0.0/0](tcp:1433) | | | | [0.0.0.0/0;::/0](tcp:443) | | | | [::/0;0.0.0.0/0](tcp:80) | | | | [::/0;0.0.0.0/0](tcp:2242) | | | | [::/0;0.0.0.0/0](tcp:22) | | | | [0.0.0.0/0](tcp:3389) | | sg-30abc756 | vpc-00b68c65 | [0.0.0.0/0](tcp:443) | | | | [0.0.0.0/0](tcp:80) | [...] Columns truncated to fit terminal: 'Outbound', 'Name', 'Description'


The output is easily and naturally sorted and filtered: 



 > awless list instance --sort uptime --filter zone=eu-west-1b | ID | ZONE | NAME | STATE | |---------------------|------------|---------------------|---------| | i-03a1f22b36ddb0739 | eu-west-1b | blog-test | running | | i-05adde5a930da5ed3 | eu-west-1b | authoringtest | stopped | | i-06b8f07c1e4afb49a | eu-west-1b | AwlessWithScheduler | running | [...] Columns truncated to fit terminal: 'Type', 'Public IP', 'Private IP', 'Uptime', 'KeyPair'


Output format can be customized. By default, data is displayed in a readable table (the result is displayed in markdown, which can be passed as input to a third-party tool, for example, in pandoc ). Data can also be output in csv, json or tsv formats: 



 > awless list subnets --format csv ID,Name,CIDR,Zone,Default,Vpc,Public,State subnet-0c41ad68,,172.31.0.0/20,eu-west-1a,true,vpc-00b68c65,true,available subnet-2486b17c,demo-env-subnet,10.0.0.0/24,eu-west-1c,false,vpc-67b25c00,true,available [...]


One of the distinguishing features of awless.io is the ability to work with the infrastructure represented as an RDF graph and perform complex queries locally. Using a standardized format allows you to use existing tools, such as Protégé , to perform graph queries. The graph saved locally allows you to work with the infrastructure to some extent even without accessing the Internet: 



 > ping amazon.com ping: cannot resolve amazon.com: Unknown host > awless list instances --local | ID â–˛ | ZONE | NAME | STATE | |---------------------|------------|---------------------|---------| | i-01b17116fed0a09bd | eu-west-1c | windows-demo-env | running | | i-05c0a275adb5484fa | eu-west-1c | git.inno.wallix.com | running | | i-06b8f07c1e4afb49a | eu-west-1b | AwlessWithScheduler | running | [...]<br />


Retrieving Resource Information



The show command allows you to get information about any resource: 



 > awless show i-03a1f22b36ddb0739 | PROPERTY ▲ | VALUE | |-------------------|---------------------| | Architecture | x86_64 | | Hypervisor | xen | | ID | i-03a1f22b36ddb0739 | | Image | ami-70edb016 | | KeyPair | keypair-blog | | Name | blog-test | | NetworkInterfaces | [eni-10e4352d] | | Private IP | 10.0.20.245 | | Public IP | 34.252.193.183 |<br /> | RootDevice | /dev/xvda | | RootDeviceType | ebs | | SecurityGroups | [sg-a2a282c4] | | State | running | | Subnet | subnet-472a1731 | | Type | t2.micro | | Uptime | 46 hours | | Vpc | vpc-96c7eef2 | | Zone | eu-west-1b | eu-west-1[region] ↳ @awless-test-vpc[vpc] ↳ @awless-public-subnet[subnet] ↳ @blog-test[instance] Depending on: keypair-blog[keypair], @default[securitygroup], vol-0f82cd93824f456e5[volume] Siblings: @AwlessWithScheduler[instance]


This command displays the characteristics and dependencies of the cloud resource. For example, in the case of an instance, the command will display its Virtual Private Cloud (VPC), subnet, associated objects, and information about the mounted volume, key pair, and instance guppah. The list of properties will vary depending on the type of resource: 



 > awless show AIDAI3FX74KAQQKIOM2XY | PROPERTY â–˛ | VALUE | |------------------|-----------------------------------------------| | Arn | arn:aws:iam::519101999238:user/awless-test-usr| | Created | 4 months | | ID | AIDAI3FX74KAQQKIOM2XY | | Name | awless-test-user | | PasswordLastUsed | 2 weeks | | Path | / | Depending on: @PepsWallixS3Assume[policy], @PepsWallixS3[policy]


One-liners and templates



awless.io is a great tool for viewing and analyzing cloud resources. But it also provides a powerful template engine that supports infrastructure operations (creating, editing, and deleting resources).



One-liners



You can use awless.io templates by entering single-line commands, or by using scripts. To create an instance, key pair or resource of any other type, two simple commands are enough: 



 > awless create instance > awless create keypair


awless.io does not require entering all the necessary information at once. Instead, it asks for data as the operation progresses, giving you the option of auto-completion as they are entered.



Let's first create a key pair that we use to connect over SSH to the instances that we will create right after: 



 > awless create keypair Please specify (Ctrl+C to quit, Tab for completion): keypair.name? keypair-blog create keypair name=keypair-blog Confirm? (y/n): y [info] Generating locally a RSA 4096 bits keypair… [info] 4096 RSA keypair generated locally and stored in '/Users/henri/.awless/keys/keypair-blog.pem' OK keypair = keypair-blog [info] Revert this template with `awless revert 01BHAE1C0CPMNYQ0KFPRQQ7XYA`


We entered only the awless create keypair command , the name of the key pair of the keypair-blog and confirmed its creation.



awless.io uses both command line input and default options that can be configured (for example, AMI and instance type); awless.io asks for user confirmation before making changes to the existing infrastructure.



Now we easily create an instance using our new pair of keys: 



 > awless create instance keypair=@keypair-blog Please specify (Ctrl+C to quit, Tab for completion): instance.name? instance-blog instance.subnet? <Tab> @awless-private-subnet @awless-public-subnet @demo-env-subnet instance.subnet? @demo-env-subnet create instance count=1 image=ami-70edb016 keypair=keypair-blog name=instance-blog subnet=subnet-2486b17c type=t2.micro Confirm? (y/n): y [info] create tag 'Name=instance-blog' on 'i-094ceffee40892f66' done [info] create instance 'i-094ceffee40892f66' done OK instance = i-094ceffee40892f66 [info] Revert this template with `awless revert 01BHAGCXY80FPK5FR73GTWWSD2`


To connect to an instance directly, you can use the ssh command: 



 > awless ssh instance-blog awless could not validate the authenticity of '34.252.193.183:22' (unknown host) ecdsa-sha2-nistp256 public key fingerprint is SHA256:Ay8ODMNQiEt1U6BoTsPyIajoaHZ047v7uNjTV1SSQmk. Do you want to continue connecting and persist this key to '/Users/henri/.ssh/known_hosts' (yes/no)? yes [info] Login as 'ec2-user' on '34.252.193.183', using keypair '/Users/henri/.awless/keys/keypair-blog.pem' with ssh client '/usr/bin/ssh' __| __|_ ) _| ( / Amazon Linux AMI ___|\___|___| https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/ 18 package(s) needed for security, out of 52 available Run “sudo yum update” to apply all updates. Amazon Linux version 2017.03 is available. [ec2-user@ip-10–0–20–245 ~]$


There is no need to pass the keys as arguments to the command, since they are stored locally, there is no need to specify a username, because awless.io guesses it yourself. The devil is in the details, and awless.io is doing everything possible to make using CLI as convenient as possible.



In addition to simplified short commands, other new and useful functions are at your service. If you carefully read the previous output of the command, you might have noticed that with each successfully completed action, revert Id is associated. Indeed, changes resulting from the execution of most awless.io templates (one-liners and full-fledged scripts) can be very easily undone.



Like git, awless.io keeps a log of the operations performed: 



 > awless log [...] ID: 01BHAE1C0CPMNYQ0KFPRQQ7XYA, Date: May 29 17:43:50, Author: user/hbinsztok, Region: eu-west-1 OK create keypair name=keypair-blog[keypair-blog] ID: 01BHAGCXY80FPK5FR73GTWWSD2, Date: May 29 18:25:06, Author: user/hbinsztok, Region: eu-west-1 OK create instance count=1 image=ami-70edb016 keypair=keypair-blog name=instance-blog subnet=subnet-2486b17c type=t2.micro [i-094ceffee40892f66]


And just like commits, operations can be easily rolled back: 



 > awless revert 01BHAGCXY80FPK5FR73GTWWSD2 delete instance id=i-094ceffee40892f66 Confirm? (y/n): y OK delete instance


Templates



Previously, we used templates that contained only one command (one-liners). In general, templates are a generalization and are a sequence of awless.io commands. 



 #        securitygroup = create securitygroup vpc={instance.vpc} description={securitygroup.description} name=ssh-from-internet #     22       update securitygroup id=$securitygroup inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=22 #     keypair = create keypair name={keypair.name} #     ,    ,     create instance subnet={instance.subnet} image={instance.image} type={instance.type} keypair=$keypair name={instance.name} count=1 securitygroup=$securitygroup


We added the following options for building a chain of commands:





Templates are an extremely effective and useful feature of awless.io. Using them, for example, you can easily:





To execute the template as an argument, you can pass the file name in the official awless-templates repository (with the repo: prefix), URL or the name of the local template file: 



 > awless run repo:dynamic_autoscaling_watching_CPU


Try it yourself! Especially after that, you can use the awless revert command, which we discussed above, to remove all the resources created as a result of running templates (instances, security groups, target groups, load balancers, etc.).



Please note that for now we have implemented only a draft version of the template template, and before the release of awless.io 1.0, changes may be introduced and new functions added. After release 1.0, all new versions will be accompanied by utilities to simplify migrations.



Command Overview



You can get help on awless.io functions like this: 



 > awless help > awless [verb] -h


These tables contain the main keywords used in commands and templates.



And this is just the beginning.



Today, after more than 20 alpha releases released since the launch of the project in January 2017, we are presenting awless.io 0.1.0. This project is still under development. We plan to improve our product in many ways, in particular, to add support for an even greater number of AWS services and functions. Please, if your favorite service (still) is not supported, open new or add existing issues on GitHub. Here is a development plan and functionality that we plan to add over the next year:





We will be happy for your help in the implementation of the project, including if you tell your friends about it. And in order to stay up to date with all the innovations, follow the news on the official @awlessCLI twitter.

Source: https://habr.com/ru/post/330502/

More articles:



All Articles