/ Cloud Automation

AWS EC2 Automation Using Bash Scripts

These bash scripts will allow you to automagically SSH into newly provisioned EC2 containers on AWS, as well as terminate instances and commit to GitHub with a few simple bash commands.

Since it already pushes to GitHub, you might as well setup an automated build on Docker Hub for the integration there.

#!/bin/sh
# Edits one of the dotfiles and then re-sources it
#
# USAGE:
#	$ rc # edit .bashrc
#	$ rc aliases # edit .bash_aliases
#	$ rc input # edit .inputrc

NAME=${1:-bash}

for file in "$NAME" ".$NAME" ".${NAME}rc" ".bash_$NAME"; do
	path="$HOME/$file"
	if [ -f "$path" ]; then
		echo "editing $file"
		# FIXME: It's source to this script and not to shell
		nano "$path" && . "$path"
		exit
	fi
done

echo "no file found with that name"
exit 1

You can put these scripts inside of a dotfiles directory like .ec2 with all of your EC2 configuration parameters.

ec2_instance_setup.sh
instance_installs.sh
temrinate_instance.sh

Create and SSH

#!/bin/bash

# Authorize TCP, SSH & ICMP for default Security Group
#ec2-authorize default -P icmp -t -1:-1 -s 0.0.0.0/0
#ec2-authorize default -P tcp -p 22 -s 0.0.0.0/0

# The Static IP Address for this instance:
IP_ADDRESS=$(cat ~/.ec2/ip_address)

# Create new t1.micro instance using ami-af7e2eea (64 bit Ubuntu 10.10 Maverick Meerkat)
# using the default security group and a 16GB EBS datastore as /dev/sda1.
# EC2_INSTANCE_KEY is an environment variable containing the name of the instance key.
# --block-device-mapping ...:false to leave the disk image around after terminating instance
EC2_RUN_RESULT=$(ec2-run-instances --instance-type t1.micro --group default --region us-west-1 --key $EC2_INSTANCE_KEY --block-device-mapping "/dev/sda1=:16:true" --instance-initiated-shutdown-behavior stop --user-data-file instance_installs.sh ami-af7e2eea)

INSTANCE_NAME=$(echo ${EC2_RUN_RESULT} | sed 's/RESERVATION.*INSTANCE //' | sed 's/ .*//')

times=0
echo
while [ 5 -gt $times ] && ! ec2-describe-instances $INSTANCE_NAME | grep -q "running"
do
  times=$(( $times + 1 ))
  echo Attempt $times at verifying $INSTANCE_NAME is running...
done

echo

if [ 5 -eq $times ]; then
  echo Instance $INSTANCE_NAME is not running. Exiting...
  exit
fi

ec2-associate-address $IP_ADDRESS -i $INSTANCE_NAME

echo
echo Instance $INSTANCE_NAME has been created and assigned static IP Address $IP_ADDRESS
echo

# Since the server signature changes each time, remove the server's entry from ~/.ssh/known_hosts
# You may not need to do this if you're using a Reserved Instance?
ssh-keygen -R $IP_ADDRESS

# SSH INTO NEW EC2 INSTANCE
ssh -i $EC2_HOME/$EC2_INSTANCE_KEY.pem [email protected]$IP_ADDRESS

I titled this file ec2_instance_setup.sh, but it really doesn't matter what you name it. Just make sure you include instance_installs.sh along with this one.

You’ll notice this script uses ec2-run-instances with the –user-data-file parameter, and that is where the magic happens. Once the instance is running, the script you pass in that parameter is automatically run. I intend to use that script to:

  • Setup my home directory structure.
  • Run a package installer to add everything I need from the repository.
  • Perform other miscellaneous setup like creating users, assigning permissions, setting up and starting daemons, etc.

Additional Perks

  • The script serves as a reminder of every step taken to create, start and configure the instance.
  • Provides a configuration management tool for EC2.
  • You don’t have to pay storage charges for AWS Snapshots.

EC2 Configuration Parameters

#!/bin/bash
set -e

function usage
{
  echo
  echo "$(basename $0) spools up a new ec2 instance"
  echo
  echo "usage: $(basename $0) -k key [[[-a ip_address] [-s script]] | [-h]]"
  echo "       -a, --ami          : the Amazon Machine Image (AMI) name"
  echo "       -d, --disks        : Comma separated list of block device mappings"
  echo "                          :   e.g. '/dev/sda1=:8:false,/dev/sda1=:1:true'"
  echo "                          :   (see ec2-run-instances -h option --block-device-mapping for format)"
  echo "       -h, --help         : displays the information you're reading now"
  echo "       -i, --ip-address   : the IP Address to assign to the new instance"
  echo "       -s, --script       : the shell script to run after the instance is created"
  echo "       -t, --instance-type: e.g. t1.micro, m1.small, m1.large, etc."
  echo
}

if [ ! $EC2_HOME ] || [ ! $EC2_INSTANCE_KEY ]; then
  echo
  echo "An EC2_HOME environment variable set to the EC2 installation directory (usually"
  echo "~/.ec2) and an EC2_INSTANCE_KEY environment variable set to the EC2 keypair "
  echo "name must be defined. The EC2 instance key must reside in the directory specified"
  echo "in EC2_HOME. You will need to create the environment variable based on the requirements"
  echo "of your particular Operating System."
  echo
  exit 1
fi

# Authorize TCP, SSH & ICMP for default Security Group
#ec2-authorize default -P icmp -t -1:-1 -s 0.0.0.0/0
#ec2-authorize default -P tcp -p 22 -s 0.0.0.0/0

# Loop through command line params and capture values
while [ "$1" != "" ]; do
  case $1 in
    -i | --ip-address )    shift
                           IP_ADDRESS=$1
                           ;;
    -s | --script )        shift
                           INSTALL_SCRIPT=$1
                           ;;
    -h | --help )          usage
                           exit
                           ;;
    -a | --ami )           shift
                           AMI=$1
                           ;;
    -d | --disks )         shift
                           DISKS=$1
                           ;;
    -t | --instance-type ) shift
                           INSTANCE_TYPE=$1
                           ;;
    * )                    usage
                           exit 1
  esac

  shift
done

echo

if [ ! $AMI ]; then
  AMI="ami-af7e2eea"
  echo "No AMI specified. Using Ubuntu 11.10 Oneiric (ami-af7e2eea) by default..."
else
  echo "Using specified ami: ($AMI)"
fi

if [ ! $DISKS ]; then
  # --block-device-mapping ...:false to leave the disk image around after terminating the instance
  BLOCK_DEVICE_MAPPINGS='--block-device-mapping "/dev/sda1=:8:true"'
  echo "No disks specified. Using 8GB image not deleted on instance termination by default..."
else
  IFS=$','
  for disk in $DISKS; do BLOCK_DEVICE_MAPPINGS="--block-device-mapping \"$disk\" $BLOCK_DEVICE_MAPPINGS"; done
  unset IFS
  echo "Using specified disk mappings: ($BLOCK_DEVICE_MAPPINGS)"
fi

if [ ! $INSTANCE_TYPE ]; then
  INSTANCE_TYPE="t1.micro"
  echo "No instance-type specified. Using t1.micro by default..."
else
  echo "Using specified instance-type: ($INSTANCE_TYPE)"
fi

echo
echo "Starting your new instance. Please wait..."

if [ $INSTALL_SCRIPT ]; then
  echo "The script $INSTALL_SCRIPT will be run once the instance is created."
  SCRIPT_PARAM="--user-data-file $INSTALL_SCRIPT"
fi

MAX_SECONDS_TO_WAIT=181
SECONDS_TO_ADD=5

# Create new instance
EC2_RUN_RESULT=$(ec2-run-instances --instance-type $INSTANCE_TYPE --group default --key $EC2_INSTANCE_KEY $BLOCK_DEVICE_MAPPINGS --instance-initiated-shutdown-behavior stop $SCRIPT_PARAM $AMI)

INSTANCE_NAME=$(echo ${EC2_RUN_RESULT} | sed 's/RESERVATION.*INSTANCE //' | sed 's/ .*//')

SECONDS_TO_WAIT=0

echo

while [ $MAX_SECONDS_TO_WAIT -gt $SECONDS_TO_WAIT ] && ! ec2-describe-instances $INSTANCE_NAME | grep -q "running"
do
  SECONDS_TO_WAIT=$(( $SECONDS_TO_WAIT + $SECONDS_TO_ADD ))
  echo "$INSTANCE_NAME not running. Waiting $SECONDS_TO_WAIT seconds before checking again..."
  sleep $(echo $SECONDS_TO_WAIT)s
done

if [ $MAX_SECONDS_TO_WAIT -lt $SECONDS_TO_WAIT ]; then
  echo "Instance $INSTANCE_NAME is taking too long to enter the running state. Exiting..."
  exit 1
fi

echo
echo "Instance $INSTANCE_NAME is now running."

DESCRIBE_INSTANCE=$(ec2-describe-instances $INSTANCE_NAME)
INSTANCE_FQDN=$(echo ${DESCRIBE_INSTANCE} | sed -E 's/RESERVATION.*ami-.{9}//' | sed -E 's/\ .*//')

if [ $IP_ADDRESS ]; then
  echo "Associating it with IP Address $IP_ADDRESS..."
  ec2-associate-address $IP_ADDRESS -i $INSTANCE_NAME
fi

# Sleep for a bit... ssh seems to fail if started too soon.
echo "Please wait..."
sleep 20s

# SSH details for my BRAND NEW EC2 INSTANCE! WooHoo!!!
echo "You can ssh into your new instance with the following command:"
echo "ssh -o StrictHostKeyChecking=no -i $EC2_HOME/$EC2_INSTANCE_KEY.pem [email protected]$INSTANCE_FQDN"

Quick SSH into AWS EC2

Instead of typing ssh -i ec2-keypair.pem [email protected] each time, you can add an alias to your User/.ssh/config file:

Host aws
IdentityFile ~/.ssh/ec2-keypair.pem
HostName yourPublicDNS
User ec2-user

Once you place the ec2-keypair.pem in the default directory, you can now just connect with:

ssh aws

EC2 Termination Script

#!/bin/bash

# function to display help information
function usage
{
  echo
  echo "temrinate_instance.sh [image_name]"
  echo
  echo "removes the server's entry from known_hosts and terminates the instance"
  echo
}

if [ ! $1 ]; then
  usage
  exit 1
fi

echo "Getting $1's Fully Qualified Domain Name..."

DESCRIBE_INSTANCE=$(ec2-describe-instances $1)
INSTANCE_FQDN=$(echo ${DESCRIBE_INSTANCE} | sed -E 's/RESERVATION.*ami-.{9}//' | sed -E 's/\ .*//')

echo "Removing $INSTANCE_FQDN from known_hosts..."
ssh-keygen -R $INSTANCE_FQDN

echo "Terminating $INSTANCE_FQDN..."
ec2-terminate-instances $1

Here's one last shortcut for git commits.

#!/bin/sh
# - git adds all files
# - commits them with the given message
# - copies the commit message to the clipboard
#
# USAGE:
#	- commit "my message"

git add -A && git commit -m "$1" && echo "$1" | clip
Seth Bergman

Seth Bergman

I'm a Software Engineer focused on helping companies scale. I love learning about software architecture, containers, open source programming and automation.

Read More