AWS EC2 Automation Using Bash Scripts
These bash scripts will allow you to automagically SSH into newly provisioned EC2 containers on AWS, as well as terminate instances and commit to GitHub with a few simple bash commands.
Since it already pushes to GitHub, you might as well setup an automated build on Docker Hub for the integration there.
#!/bin/sh
# Edits one of the dotfiles and then re-sources it
#
# USAGE:
# $ rc # edit .bashrc
# $ rc aliases # edit .bash_aliases
# $ rc input # edit .inputrc
NAME=${1:-bash}
for file in "$NAME" ".$NAME" ".${NAME}rc" ".bash_$NAME"; do
path="$HOME/$file"
if [ -f "$path" ]; then
echo "editing $file"
# FIXME: It's source to this script and not to shell
nano "$path" && . "$path"
exit
fi
done
echo "no file found with that name"
exit 1
You can put these scripts inside of a dotfiles directory like .ec2
with all of your EC2 configuration parameters.
ec2_instance_setup.sh
instance_installs.sh
temrinate_instance.sh
Create and SSH
#!/bin/bash
# Authorize TCP, SSH & ICMP for default Security Group
#ec2-authorize default -P icmp -t -1:-1 -s 0.0.0.0/0
#ec2-authorize default -P tcp -p 22 -s 0.0.0.0/0
# The Static IP Address for this instance:
IP_ADDRESS=$(cat ~/.ec2/ip_address)
# Create new t1.micro instance using ami-af7e2eea (64 bit Ubuntu 10.10 Maverick Meerkat)
# using the default security group and a 16GB EBS datastore as /dev/sda1.
# EC2_INSTANCE_KEY is an environment variable containing the name of the instance key.
# --block-device-mapping ...:false to leave the disk image around after terminating instance
EC2_RUN_RESULT=$(ec2-run-instances --instance-type t1.micro --group default --region us-west-1 --key $EC2_INSTANCE_KEY --block-device-mapping "/dev/sda1=:16:true" --instance-initiated-shutdown-behavior stop --user-data-file instance_installs.sh ami-af7e2eea)
INSTANCE_NAME=$(echo ${EC2_RUN_RESULT} | sed 's/RESERVATION.*INSTANCE //' | sed 's/ .*//')
times=0
echo
while [ 5 -gt $times ] && ! ec2-describe-instances $INSTANCE_NAME | grep -q "running"
do
times=$(( $times + 1 ))
echo Attempt $times at verifying $INSTANCE_NAME is running...
done
echo
if [ 5 -eq $times ]; then
echo Instance $INSTANCE_NAME is not running. Exiting...
exit
fi
ec2-associate-address $IP_ADDRESS -i $INSTANCE_NAME
echo
echo Instance $INSTANCE_NAME has been created and assigned static IP Address $IP_ADDRESS
echo
# Since the server signature changes each time, remove the server's entry from ~/.ssh/known_hosts
# You may not need to do this if you're using a Reserved Instance?
ssh-keygen -R $IP_ADDRESS
# SSH INTO NEW EC2 INSTANCE
ssh -i $EC2_HOME/$EC2_INSTANCE_KEY.pem [email protected]$IP_ADDRESS
I titled this file ec2_instance_setup.sh
, but it really doesn't matter what you name it. Just make sure you include instance_installs.sh
along with this one.
You’ll notice this script uses ec2-run-instances
with the –user-data-file
parameter, and that is where the magic happens. Once the instance is running, the script you pass in that parameter is automatically run. I intend to use that script to:
- Setup my home directory structure.
- Run a package installer to add everything I need from the repository.
- Perform other miscellaneous setup like creating users, assigning permissions, setting up and starting daemons, etc.
Additional Perks
- The script serves as a reminder of every step taken to create, start and configure the instance.
- Provides a configuration management tool for EC2.
- You don’t have to pay storage charges for AWS Snapshots.
EC2 Configuration Parameters
#!/bin/bash
set -e
function usage
{
echo
echo "$(basename $0) spools up a new ec2 instance"
echo
echo "usage: $(basename $0) -k key [[[-a ip_address] [-s script]] | [-h]]"
echo " -a, --ami : the Amazon Machine Image (AMI) name"
echo " -d, --disks : Comma separated list of block device mappings"
echo " : e.g. '/dev/sda1=:8:false,/dev/sda1=:1:true'"
echo " : (see ec2-run-instances -h option --block-device-mapping for format)"
echo " -h, --help : displays the information you're reading now"
echo " -i, --ip-address : the IP Address to assign to the new instance"
echo " -s, --script : the shell script to run after the instance is created"
echo " -t, --instance-type: e.g. t1.micro, m1.small, m1.large, etc."
echo
}
if [ ! $EC2_HOME ] || [ ! $EC2_INSTANCE_KEY ]; then
echo
echo "An EC2_HOME environment variable set to the EC2 installation directory (usually"
echo "~/.ec2) and an EC2_INSTANCE_KEY environment variable set to the EC2 keypair "
echo "name must be defined. The EC2 instance key must reside in the directory specified"
echo "in EC2_HOME. You will need to create the environment variable based on the requirements"
echo "of your particular Operating System."
echo
exit 1
fi
# Authorize TCP, SSH & ICMP for default Security Group
#ec2-authorize default -P icmp -t -1:-1 -s 0.0.0.0/0
#ec2-authorize default -P tcp -p 22 -s 0.0.0.0/0
# Loop through command line params and capture values
while [ "$1" != "" ]; do
case $1 in
-i | --ip-address ) shift
IP_ADDRESS=$1
;;
-s | --script ) shift
INSTALL_SCRIPT=$1
;;
-h | --help ) usage
exit
;;
-a | --ami ) shift
AMI=$1
;;
-d | --disks ) shift
DISKS=$1
;;
-t | --instance-type ) shift
INSTANCE_TYPE=$1
;;
* ) usage
exit 1
esac
shift
done
echo
if [ ! $AMI ]; then
AMI="ami-af7e2eea"
echo "No AMI specified. Using Ubuntu 11.10 Oneiric (ami-af7e2eea) by default..."
else
echo "Using specified ami: ($AMI)"
fi
if [ ! $DISKS ]; then
# --block-device-mapping ...:false to leave the disk image around after terminating the instance
BLOCK_DEVICE_MAPPINGS='--block-device-mapping "/dev/sda1=:8:true"'
echo "No disks specified. Using 8GB image not deleted on instance termination by default..."
else
IFS=$','
for disk in $DISKS; do BLOCK_DEVICE_MAPPINGS="--block-device-mapping \"$disk\" $BLOCK_DEVICE_MAPPINGS"; done
unset IFS
echo "Using specified disk mappings: ($BLOCK_DEVICE_MAPPINGS)"
fi
if [ ! $INSTANCE_TYPE ]; then
INSTANCE_TYPE="t1.micro"
echo "No instance-type specified. Using t1.micro by default..."
else
echo "Using specified instance-type: ($INSTANCE_TYPE)"
fi
echo
echo "Starting your new instance. Please wait..."
if [ $INSTALL_SCRIPT ]; then
echo "The script $INSTALL_SCRIPT will be run once the instance is created."
SCRIPT_PARAM="--user-data-file $INSTALL_SCRIPT"
fi
MAX_SECONDS_TO_WAIT=181
SECONDS_TO_ADD=5
# Create new instance
EC2_RUN_RESULT=$(ec2-run-instances --instance-type $INSTANCE_TYPE --group default --key $EC2_INSTANCE_KEY $BLOCK_DEVICE_MAPPINGS --instance-initiated-shutdown-behavior stop $SCRIPT_PARAM $AMI)
INSTANCE_NAME=$(echo ${EC2_RUN_RESULT} | sed 's/RESERVATION.*INSTANCE //' | sed 's/ .*//')
SECONDS_TO_WAIT=0
echo
while [ $MAX_SECONDS_TO_WAIT -gt $SECONDS_TO_WAIT ] && ! ec2-describe-instances $INSTANCE_NAME | grep -q "running"
do
SECONDS_TO_WAIT=$(( $SECONDS_TO_WAIT + $SECONDS_TO_ADD ))
echo "$INSTANCE_NAME not running. Waiting $SECONDS_TO_WAIT seconds before checking again..."
sleep $(echo $SECONDS_TO_WAIT)s
done
if [ $MAX_SECONDS_TO_WAIT -lt $SECONDS_TO_WAIT ]; then
echo "Instance $INSTANCE_NAME is taking too long to enter the running state. Exiting..."
exit 1
fi
echo
echo "Instance $INSTANCE_NAME is now running."
DESCRIBE_INSTANCE=$(ec2-describe-instances $INSTANCE_NAME)
INSTANCE_FQDN=$(echo ${DESCRIBE_INSTANCE} | sed -E 's/RESERVATION.*ami-.{9}//' | sed -E 's/\ .*//')
if [ $IP_ADDRESS ]; then
echo "Associating it with IP Address $IP_ADDRESS..."
ec2-associate-address $IP_ADDRESS -i $INSTANCE_NAME
fi
# Sleep for a bit... ssh seems to fail if started too soon.
echo "Please wait..."
sleep 20s
# SSH details for my BRAND NEW EC2 INSTANCE! WooHoo!!!
echo "You can ssh into your new instance with the following command:"
echo "ssh -o StrictHostKeyChecking=no -i $EC2_HOME/$EC2_INSTANCE_KEY.pem [email protected]$INSTANCE_FQDN"
Quick SSH into AWS EC2
Instead of typing ssh -i ec2-keypair.pem [email protected]
each time, you can add an alias to your User/.ssh/config
file:
Host aws
IdentityFile ~/.ssh/ec2-keypair.pem
HostName yourPublicDNS
User ec2-user
Once you place the ec2-keypair.pem
in the default directory, you can now just connect with:
ssh aws
EC2 Termination Script
#!/bin/bash
# function to display help information
function usage
{
echo
echo "temrinate_instance.sh [image_name]"
echo
echo "removes the server's entry from known_hosts and terminates the instance"
echo
}
if [ ! $1 ]; then
usage
exit 1
fi
echo "Getting $1's Fully Qualified Domain Name..."
DESCRIBE_INSTANCE=$(ec2-describe-instances $1)
INSTANCE_FQDN=$(echo ${DESCRIBE_INSTANCE} | sed -E 's/RESERVATION.*ami-.{9}//' | sed -E 's/\ .*//')
echo "Removing $INSTANCE_FQDN from known_hosts..."
ssh-keygen -R $INSTANCE_FQDN
echo "Terminating $INSTANCE_FQDN..."
ec2-terminate-instances $1
Here's one last shortcut for git commits.
#!/bin/sh
# - git adds all files
# - commits them with the given message
# - copies the commit message to the clipboard
#
# USAGE:
# - commit "my message"
git add -A && git commit -m "$1" && echo "$1" | clip