Command Line Apps in Python

Python is a great language for automation. Ansible, SaltStack and Fabric are Python based. Most DevOps tools provide an SDK for Python.
Cloud providers like AWS, Google Cloud Platform (GCP) and
Azure provide an SDK for Python. The Python standard library provides
capabilities that support automation. Python is a good alternative to bash. Python scripts can be tested and reused.
Python scripts that are available as a Command Line Interface (CLI) application enable DevOps. It is very easy to create
a CLI in Python. Lets see how that works!

Buckets

We will create a small application that uses the AWS SDK for Python. The
application is available in Github and is called buckets. We will incrementally
add features to buckets. This way we learn how to create CLI apps.

Hello-Buckets

Python supports creating CLI apps. The package click enables Python programmers
to create beautiful CLI apps. The branch 01-hello-buckets
shows what is necessary to create a simple CLI.

import click

@click.command()
def main():
    """Bucket, your interface to S3"""
    print("Hello, I am Buckets")

The simplest CLI app contains a single function that has been decorated with @click.command(). The annotation tells
click to create a CLI app. When we run python buckets/buckets.py --help it prints the following:

Usage: buckets.py [OPTIONS]

  Bucket, your interface to S3

Options:
  --help  Show this message and exit.

And when we run the application python buckets/buckets.py it prints:

Hello, I am Buckets

Package, Install and Run

A CLI app should run standalone. The project 01-hello-buckets
contains setup.py that is used to package the CLI app.
With make dist we create an installation binary. With make install we install the binary. We can now type buckets or buckets --help
to run the CLI app. Buckets can now be used in DevOps pipelines. With make uninstall we uninstall buckets.

AWS S3

AWS Simple Storage Service (S3) is a massive storage service. AWS provides an SDK called boto3
to interface with S3. The branch 02-boto-buckets shows how to create a command
to display all S3 buckets in your AWS account.

import click
import boto3

client = boto3.client('s3')

@click.command()
def main():
    """Bucket, your interface to S3"""
    rows = []
    rows.append(['BucketName', 'CreationDate'])
    for bucket in client.list_buckets()['Buckets']:
        rows.append([bucket['Name'], bucket['CreationDate'].__str__()])

    print(render_table(rows))

After make dist and make install, buckets retrieves all buckets from S3 and displays them:

BucketName                                       CreationDate
===============================================  =========================
                                   dennisvriend  2018-08-15 19:47:35+00:00
                               dennisvriend.com  2018-06-04 18:56:29+00:00

Multiple Commands

The branch 03-multiple-commands shows how to create multiple commands.
Buckets now supports ls, du and count as commands.

import click
import boto3

client = boto3.client('s3')

@click.group()
def main():
    """Bucket, your interface to S3"""

@main.command()
def ls():
    """Print all buckets"""
    rows = []
    rows.append(['BucketName', 'CreationDate'])
    for bucket in client.list_buckets()['Buckets']:
        rows.append([bucket['Name'], bucket['CreationDate'].__str__()])

    print(render_table(rows))

@main.command()
def du():
    """Print usage per bucket"""
    rows = []
    rows.append(['BucketName', 'CreationDate', 'Number of objects', 'Bytes'])
    for bucket in client.list_buckets()['Buckets']:
        num_objects, total_bytes = get_bucket_size(bucket['Name'])
        rows.append([bucket['Name'], bucket['CreationDate'].__str__(), num_objects, total_bytes])

    print(render_table(rows))

@main.command()
def count():
    """Count the number of buckets"""
    sum = 0
    for _ in client.list_buckets()['Buckets']:
        sum += 1
    print(f'{str(sum)} buckets')

When we run python buckets/buckets.py --help it prints the following:

Usage: buckets.py [OPTIONS] COMMAND [ARGS]...

  Bucket, your interface to S3

Options:
  --help  Show this message and exit.

Commands:
  count  Count the number of buckets
  du     Print usage per bucket
  ls     Print all buckets

When we run python buckets/buckets.py count, the command count is executed. The method main has been decorated with

@click.group(). This tells click to create a CLI app. We can add multiple commands by decorating
other methods. Notice that we annotate the other methods with the name @main.command(). Click will add these commands to the

group main. By adding commands to the group you can add multiple commands and execute them via the CLI.

Conclusion

It is very easy to create CLI apps in Python by decorating functions. We use the @click.command() for a simple CLI. We use
the @click.group() to decorate a method – eg. main – to be a placeholder for commands. We then decorate other methods with eg.

@main.command() to denote that these methods belong to the group main. This way we can add multiple commands to the CLI app.
We have covered the basics of creating CLI apps with Python. Click has much more to offer us. Next time we will extend buckets
with parameters, default values, flags and environment variables.