Bash Scripting — Automate DevOps Workflows Like a Pro

Introduction

Why This Matters

Bash is the universal glue of DevOps. CI/CD pipelines, deployment scripts, cron jobs, backup routines, and health checks are all written in Bash because it is present on virtually every Linux system without dependencies. A well-written Bash script with proper error handling, logging, and idempotency behaves reliably across environments. A poorly written one causes silent failures that take hours to debug in production.

Script Foundations

#!/bin/bash
# Always start with a shebang line
 
# Essential safety options
set -euo pipefail
# -e: exit immediately if a command fails
# -u: treat unset variables as errors
# -o pipefail: pipe fails if any command in it fails
 
# Trap for cleanup and error messages
trap 'echo "ERROR: Script failed at line $LINENO" >&2; cleanup' ERR EXIT
 
cleanup() {
    # Remove temporary files on exit
    rm -f /tmp/deploy-$$.tmp 2>/dev/null
}
 
# Script metadata
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"
readonly VERSION="1.0.0"
 
echo "[$SCRIPT_NAME v$VERSION] Starting at $(date '+%Y-%m-%d %H:%M:%S')"

Variables and Substitution

# Variable assignment (no spaces around =)
APP_NAME="my-service"
VERSION="${1:-latest}"              # $1 or default "latest"
ENV="${2:?'Environment required'}"  # $2 or error if unset
 
# Read-only constants
readonly CONFIG_DIR="/etc/my-app"
readonly LOG_FILE="/var/log/deploy.log"
 
# Command substitution
CURRENT_DATE=$(date +%Y%m%d)
GIT_HASH=$(git rev-parse --short HEAD)
CONTAINER_ID=$(docker ps -qf "name=my-app")
 
# String manipulation
FILE="deploy-2024-01-15.tar.gz"
BASENAME="${FILE%.tar.gz}"      # remove suffix: deploy-2024-01-15
EXT="${FILE#*.}"                # remove prefix: tar.gz
UPPER="${APP_NAME^^}"           # uppercase: MY-SERVICE
LOWER="${APP_NAME,,}"           # lowercase
 
# Array variables
SERVERS=("web1.example.com" "web2.example.com" "web3.example.com")
echo "Total servers: ${#SERVERS[@]}"
echo "First server: ${SERVERS[0]}"
for server in "${SERVERS[@]}"; do
    echo "Deploying to $server"
done

Conditionals and Comparisons

# String comparisons
if [[ "$ENV" == "production" ]]; then
    echo "Production deployment"
elif [[ "$ENV" == "staging" ]]; then
    echo "Staging deployment"
else
    echo "Unknown environment: $ENV" >&2
    exit 1
fi
 
# File checks
[[ -f "$CONFIG_FILE" ]] || { echo "Config not found: $CONFIG_FILE" >&2; exit 1; }
[[ -d "$LOG_DIR" ]] || mkdir -p "$LOG_DIR"
[[ -x "$DEPLOY_SCRIPT" ]] || chmod +x "$DEPLOY_SCRIPT"
[[ -r "$CERT_FILE" ]] || { echo "Certificate not readable" >&2; exit 1; }
 
# Numeric comparisons
RETRY_COUNT=0
MAX_RETRIES=5
if (( RETRY_COUNT >= MAX_RETRIES )); then
    echo "Max retries exceeded" >&2
    exit 1
fi
 
# Check command exists
if ! command -v docker &>/dev/null; then
    echo "docker not found. Install Docker first." >&2
    exit 1
fi
 
# Check exit code of previous command
aws s3 ls s3://my-bucket &>/dev/null
if [[ $? -ne 0 ]]; then
    echo "Bucket not accessible" >&2
    exit 1
fi
# Simpler: use || directly
aws s3 ls s3://my-bucket &>/dev/null || { echo "Bucket not accessible" >&2; exit 1; }

Loops

# For loop over list
for env in development staging production; do
    echo "Checking $env"
    ./check-env.sh "$env"
done
 
# For loop over files
for config in /etc/my-app/*.conf; do
    [[ -f "$config" ]] || continue  # skip if no files match glob
    echo "Validating: $config"
    validate_config "$config"
done
 
# While loop with counter
MAX_WAIT=60
elapsed=0
while ! curl -sf http://localhost:3000/health &>/dev/null; do
    if (( elapsed >= MAX_WAIT )); then
        echo "Service failed to start after ${MAX_WAIT}s" >&2
        exit 1
    fi
    echo "Waiting for service... (${elapsed}s)"
    sleep 5
    (( elapsed += 5 ))
done
echo "Service is healthy"
 
# Loop over command output
while IFS= read -r line; do
    echo "Processing: $line"
done < <(aws ec2 describe-instances --query 'Reservations[*].Instances[*].InstanceId' --output text)

Functions

# Logging helper
log() {
    local level="$1"
    shift
    echo "[$(date '+%H:%M:%S')] [$level] $*" | tee -a "$LOG_FILE"
}
 
log_info() { log "INFO" "$@"; }
log_warn() { log "WARN" "$@" >&2; }
log_error() { log "ERROR" "$@" >&2; }
 
# Retry function
retry() {
    local max_attempts="$1"
    local delay="$2"
    shift 2
    local attempt=1
 
    until "$@"; do
        if (( attempt >= max_attempts )); then
            log_error "Command failed after $max_attempts attempts: $*"
            return 1
        fi
        log_warn "Attempt $attempt failed. Retrying in ${delay}s..."
        sleep "$delay"
        (( attempt++ ))
    done
}
 
# Deploy function
deploy_service() {
    local service="$1"
    local version="$2"
    local env="${3:-production}"
 
    log_info "Deploying $service:$version to $env"
 
    # Pull new image
    retry 3 10 docker pull "myregistry.example.com/$service:$version"
 
    # Graceful rolling restart
    docker service update \
        --image "myregistry.example.com/$service:$version" \
        --update-delay 30s \
        --update-parallelism 1 \
        "$service"
 
    log_info "Deployment complete: $service:$version"
}

Argument Parsing

#!/bin/bash
set -euo pipefail
 
usage() {
    cat << EOF
Usage: $0 [OPTIONS] ENVIRONMENT
 
Deploy application to target environment.
 
Arguments:
  ENVIRONMENT   Target environment (development|staging|production)
 
Options:
  -v VERSION    Image version to deploy (default: latest)
  -d            Dry run — show what would happen without executing
  -h            Show this help message
 
Examples:
  $0 production
  $0 -v 1.2.3 staging
  $0 -d production
EOF
    exit 0
}
 
# Parse flags
VERSION="latest"
DRY_RUN=false
 
while getopts "v:dh" opt; do
    case $opt in
        v) VERSION="$OPTARG" ;;
        d) DRY_RUN=true ;;
        h) usage ;;
        *) echo "Unknown option: -$OPTARG" >&2; usage ;;
    esac
done
shift $(( OPTIND - 1 ))
 
# Validate positional arguments
ENV="${1:?'ERROR: ENVIRONMENT argument required'}"
[[ "$ENV" =~ ^(development|staging|production)$ ]] || {
    echo "ERROR: Invalid environment '$ENV'" >&2
    usage
}
 
if $DRY_RUN; then
    echo "[DRY RUN] Would deploy version $VERSION to $ENV"
else
    deploy_service "my-api" "$VERSION" "$ENV"
fi

Common Mistakes

Not using set -euo pipefail — scripts continue silently after failed commands
Using [ ] instead of [[ ]] — [[ ]] handles spaces in variables correctly without quoting issues
Forgetting to quote variables: rm -rf $DIR vs rm -rf "$DIR" — unquoted variables split on whitespace
Using $(cat file) instead of < file or $(<file) — unnecessary subprocess overhead
Not redirecting error output: command 2>/dev/null when you want to suppress or redirect errors separately

Best Practices

Use shellcheck to lint scripts before deployment — it catches most common Bash mistakes
Write idempotent scripts — running the same script twice should produce the same result
Use mktemp for temporary files: TMP=$(mktemp /tmp/deploy.XXXXXX) and clean up in trap
Log with timestamps and levels — silent scripts are hard to debug when they run as cron jobs
Store scripts in version control alongside application code — treat infrastructure scripts as first-class code

Key Takeaways

set -euo pipefail is the most important line in any production Bash script — enables immediate error detection
[[ ]] is preferred over [ ] for conditionals — supports regex matching and handles special characters safely
Always quote variable expansions: "$VAR", "${ARRAY[@]}" — prevents word splitting and globbing issues
Functions improve reusability and testability — extract retry logic, logging, and deployment steps into named functions
trap 'cleanup' ERR EXIT ensures cleanup code runs whether the script succeeds, fails, or is interrupted
Use getopts for flag parsing — standardized, handles errors correctly, and is available on all POSIX systems
shellcheck is the Bash equivalent of a linter — run it in CI to catch bugs before they reach production
Redirecting to &2 (echo "ERROR" >&2) sends error messages to stderr — allows callers to separate stdout from errors

Introduction#

Why This Matters#

Script Foundations#

Variables and Substitution#

Conditionals and Comparisons#

Loops#

Functions#

Argument Parsing#

Common Mistakes#

Best Practices#

Key Takeaways#

Related reading