Advanced Bash Scripting: Automating Complex Tasks in Linux

Bash scripts are the quickest way to automate routine server work such as backups, log handling and maintenance. This guide covers the techniques that make a script reliable: functions, arrays, cleanup traps, error handling, scheduling, safe file handling, parallel jobs and debugging.

1. Before you start

You should already be comfortable with variables, loops and if statements, and know how to run a script with bash script.sh or make it executable with chmod +x script.sh. Begin each script with a shebang line:

#!/bin/bash

2. Functions

Functions group repeated code under one name. Arguments arrive as $1, $2 and so on, and local keeps variables inside the function.

greet() {
  local first="$1"
  local second="$2"
  echo "First argument: $first"
  echo "Second argument: $second"
}

greet "Hello" "World"

Use functions for repeated checks, such as testing that a file exists or that a command succeeded, and call them throughout the script.

3. Arrays

An array holds a list of values in one variable. Indexes start at 0.

fruits=("apple" "banana" "cherry")

echo "${fruits[1]}"        # banana
echo "${#fruits[@]}"       # 3, the number of elements

for fruit in "${fruits[@]}"; do
  echo "I like $fruit"
done

Always write "${array[@]}" with the quotes, so items that contain spaces stay whole. Arrays suit lists of servers, IP addresses or file names.

4. Cleaning up with trap

trap runs a command when the script exits or receives a signal, so temporary files are removed even after an error or Ctrl+C.

tmpfile=$(mktemp)

cleanup() {
  rm -f "$tmpfile"
}
trap cleanup EXIT

# Work with "$tmpfile" here. It is removed however the script ends.

A trap on EXIT also runs after INT and TERM in Bash, so one trap is enough. Use mktemp rather than a fixed name in /tmp, which another user could create first.

5. Error handling and exit status

These options make a script stop at the first problem instead of carrying on with bad data:

set -e            # exit when a command fails
set -u            # treat unset variables as an error
set -o pipefail   # a pipeline fails if any command in it fails
# or all three at once:
set -euo pipefail

Check important commands explicitly and send errors to standard error:

if ! cp /source/file /destination/; then
  echo "Failed to copy file" >&2
  exit 1
fi

set -e does not trigger for commands inside an if condition or before ||, so check critical steps yourself as above.

6. Scheduling with cron

Edit the current user's schedule with crontab -e. This line runs a script at 02:00 every day and appends its output to a log:

0 2 * * * /root/scripts/backup.sh >> /var/log/backup.log 2>&1

Cron runs with a minimal PATH, so use full paths to scripts and commands, or set PATH at the top of the crontab. Make the script executable with chmod +x first.

7. Log rotation

On AlmaLinux and Rocky Linux, logrotate is installed and runs daily. Adding a small config file is safer than moving logs by hand, because it can compress, keep a set number of copies and handle applications that keep the log file open.

# /etc/logrotate.d/myapp
/var/log/myapp.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    copytruncate
}

Test it with logrotate -d /etc/logrotate.d/myapp. If you must rotate in a script, quote paths, create the archive directory, and tell the application to reopen its log:

#!/bin/bash
set -euo pipefail

log_file="/var/log/myapp.log"
archive_dir="/var/log/archive"

mkdir -p "$archive_dir"
if [[ -f "$log_file" ]]; then
  mv "$log_file" "$archive_dir/myapp_$(date +%F).log"
  touch "$log_file"
  systemctl reload myapp   # or restart, so the app writes to the new file
fi

8. Automated backups

This script copies a directory and dumps a MySQL or MariaDB database into a dated folder.

#!/bin/bash
set -euo pipefail

backup_dir="/backups/$(date +%F)"
mkdir -p "$backup_dir"

# Copy a directory
rsync -a /important/data/ "$backup_dir/data/"

# Dump a database. Credentials come from /root/.my.cnf, not the command line.
mysqldump --single-transaction database_name > "$backup_dir/database_name.sql"

Store the database user and password in /root/.my.cnf with permissions 600:

[client]
user=backup_user
password=your_password

A password typed after -p on the command line can be seen by other users in the process list. --single-transaction gives a consistent dump of InnoDB tables without locking them. Copy backups to a second location too; a backup on the same disk does not survive a disk failure.

9. Faster and safer commands

Count matches with one command instead of two:

grep -c "pattern" file      # instead of: grep "pattern" file | wc -l

When deleting files found by find, let find do the work. Piping plain find output into xargs rm breaks on file names that contain spaces.

find . -name "*.log" -delete
# or, for any other command:
find . -name "*.log" -print0 | xargs -0 rm -f

10. Running tasks in parallel

Start jobs in the background with & and wait for them all to finish:

task1 &
task2 &
wait

To process a list with up to four jobs at once, use xargs -P:

xargs -a file_list.txt -d '\n' -n 1 -P 4 ./process_file.sh

Don't run more parallel jobs than the server can handle; disk-heavy tasks often get slower, not faster.

11. Debugging

Trace a whole script by running it with bash -x script.sh, or switch tracing on and off around one section:

set -x    # print each command before it runs
# commands to inspect
set +x    # stop tracing

shellcheck, available from the EPEL repository (dnf install epel-release then dnf install ShellCheck), finds quoting mistakes and common bugs before you run the script.

If you need a server where you can run scheduled scripts as root, see VPS plans compared.

Frequently asked questions

Stuck on a script on your server? Open a support ticket or use our 24/7 live chat.