Course Outline (Part 14)

Welcome to Part 14 of the Linux Bash Course. In this section, we cover security hardening, script performance optimization, and the role of Bash in modern DevOps environments.

Chapter 40: Best Practices & Security

40.1 Writing Defensive Shell Scripts: Using set -e, -u, and -o pipefail

To prevent scripts from continuing execution after an error occurs, include safety flags at the beginning of your code:

  • set -e: Immediately terminates the script if any command returns a non-zero exit status.
  • set -u: Exits the script if it references a variable that has not been defined, preventing accidental execution of commands like rm -rf /$UNDEFINED_VAR.
  • set -o pipefail: Propagates pipeline error codes. If a command in a pipeline fails (e.g., fail_cmd | echo "test"), the entire pipeline returns the error status instead of the final command’s exit code.

40.2 Preventing Shell Injection Vulnerabilities by Proper Quoting

Failing to wrap variables in double quotes can allow attackers to inject arguments or shell metacharacters.

  • Vulnerable: rm $1 (If $1 contains * or spaces, it deletes unexpected files).
  • Secure: rm -- "$1" (The -- flag tells the command that any subsequent arguments are filenames, even if they start with a hyphen, and double quotes prevent word splitting).

40.3 Securing Sensitive Credentials: Environment Variables and Vaults

  • Never hardcode passwords, API keys, or database credentials in your scripts.
  • Best Practice 1: Use local system environment variables passed at runtime (e.g., DB_PASS=$DB_PASS ./backup.sh).
  • Best Practice 2: Fetch secrets dynamically from secure managers (like HashiCorp Vault or AWS Secrets Manager) using CLI tools during execution.

40.4 Restricting Script Permissions and Running with Non-Root Users

  • Set script permissions to the minimum necessary level. Use chmod 700 script.sh to restrict read, write, and execute permissions to the file owner.
  • Run scripts under a dedicated, low-privilege service account instead of root. Use sudo or runuser only for commands that require elevated privileges.

40.5 Validating Inputs, User Paths, and Directory Destinations

Validate inputs before processing files:

  • Use -d to verify that directories exist before writing to them.
  • Check that integer inputs are numbers: 
    if [[ ! "$input" =~ ^[0-9]+$ ]]; then
    echo "Error: Input must be a positive integer" >&2
    exit 1
fi

40.6 Managing Temporary Files Securely: mktemp and cleanup traps

Use the mktemp utility to create temporary files with random, secure names under /tmp/. Use a trap command to clean up these files when the script exits or receives an interruption signal.

  • Syntax:

    mktemp [options] [template]
trap 'command_to_clean' SIGNALS

  • Example Command:

    tempfile=$(mktemp /tmp/script_XXXXXX.tmp)
trap 'rm -f "$tempfile"' EXIT

  • Expected Output: (Creates a file named /tmp/script_a8f9B1.tmp and sets a cleanup trap for when the script exits)

  • Flag & Command Breakdown:

    • mktemp: Secure file creation utility.
    • /tmp/script_XXXXXX.tmp: Template path. The XXXXXX placeholders are replaced with random characters.
    • trap: Registers a command to run when the shell receives signals like EXIT, INT, or TERM.

40.7 Running Static Analysis on Scripts with shellcheck

shellcheck is a static analysis tool that highlights warnings, bugs, and syntax issues in shell scripts.

  • Syntax:

    shellcheck script.sh

  • Example Command:

    shellcheck -s bash /home/suresh/Documents/TechBlog/src/content/courses/linux-bash/scratch/bad_script.sh

  • Expected Output:

    In bad_script.sh line 4:
echo $variable
     ^-- SC2086 (info): Double quote to prevent globbing and word splitting.

  • Flag & Command Breakdown:

    • shellcheck: Auditing utility.
    • -s bash: Specifies the shell dialect to check against.

40.8 Secure Shell Configuration: Hardening /etc/ssh/sshd_config

Hardening your SSH configuration protects remote terminal sessions. Use sshd -t to test configuration changes.

  • Set PermitRootLogin no to block direct root logins.
  • Set PasswordAuthentication no to require key-based authentication.
  • Set Port to a non-standard port (like 2222) to reduce automated scan attempts.

40.9 Verifying File Integrity with Checksums: sha256sum

Use SHA-256 checksums to verify that files have not been modified or corrupted during transfer.

  • Syntax:

    sha256sum filename
sha256sum -c checksum_file

  • Example Command:

    sha256sum backup.tar.gz > backup.sha256
sha256sum -c backup.sha256

  • Expected Output:

    backup.tar.gz: OK

  • Flag & Command Breakdown:

    • sha256sum: Calculates SHA-256 hashes.
    • -c: Verifies checksums listed in the specified configuration file.

40.10 Auditing script executions and user commands with system logs

Audit logs track system actions. You can read these logs in real time using tail or journalctl.

  • Syntax:

    tail -f /var/log/auth.log

  • Example Command:

    sudo tail -n 2 /var/log/auth.log

  • Expected Output:

    Jun 12 08:30:05 server sudo:   suresh : TTY=pts/0 ; PWD=/home/suresh ; USER=root ; COMMAND=/usr/bin/apt update

  • Flag & Command Breakdown:

    • tail: Prints the end of files.
    • -n 2: Shows only the last two log entries.

Chapter 41: Performance Tuning

41.1 Profiling Shell Scripts and Timing execution using time

Use the time utility to measure the execution time of commands and scripts.

  • Syntax:

    time command_to_measure

  • Example Command:

    time sleep 1.5

  • Expected Output:

    real	0m1.503s
user	0m0.001s
sys	0m0.002s

real = Actual clock time elapsed.
user = CPU time spent in user-space code.
sys  = CPU time spent in system-space (kernel) calls.

  • Flag & Command Breakdown:

    • time: Built-in shell timing command.

41.2 Optimizing Loops: Replacing subshells with built-in commands

  • Forking subshells inside loops (e.g. $(echo "$var" | tr 'A' 'a')) slows down execution because it creates a new process for each iteration.
  • Optimized: Use Bash parameter expansion, which runs directly in the parent process (e.g., ${var,,}).

41.3 Streamlining Text Processing Pipelines: Combining grep, sed, and awk

  • Avoid chaining multiple text utilities together (e.g., cat file | grep text | awk '{print $2}' | sed 's/a/b/').
  • Optimized: Combine these operations into a single command using awk or sed (e.g., awk '/text/{gsub(/a/, "b", $2); print $2}' file), which processes the input file in a single pass.

41.4 Minimizing Forking and External Process overhead

Running external commands inside loops can slow down execution.

  • Slow: 
    for i in {1..1000}; do
    expr $i + 1 > /dev/null
done
  • Fast: 
    for i in {1..1000}; do
    ((i++))
done

Using built-in arithmetic operations ((i++)) runs inside the shell without spawning external processes like expr.

41.5 Using Parallelism in Bash: Background jobs and the wait command

You can speed up independent tasks by running them in the background using & and waiting for them to finish using wait.

  • Syntax:

    task1 &
task2 &
wait

  • Example Command:

    sleep 2 & sleep 2 & wait

  • Expected Output: (The terminal blocks for 2 seconds instead of 4 seconds because the tasks run in parallel)

  • Flag & Command Breakdown:

    • &: Tells the shell to run the command in the background.
    • wait: Blocks execution until all background jobs have finished.

41.6 GNU Parallel: Scaling command execution across CPU cores

parallel runs commands in parallel across multiple CPU cores, offering more control than standard background jobs.

  • Syntax:

    parallel command ::: arguments

  • Example Command:

    parallel gzip ::: file1.log file2.log file3.log

  • Expected Output: (Files are compressed concurrently across available CPU cores)

  • Flag & Command Breakdown:

    • parallel: GNU Parallel utility.
    • gzip: Command to run in parallel.
    • :::: Separator indicating the list of arguments to process.

41.7 Optimizing Disk I/O with Buffering and block sizes

  • When reading or writing files, use larger buffer sizes to reduce disk I/O overhead.
  • When using dd, use bs=64K or bs=1M instead of the default 512 bytes to write data in larger chunks.
  • Avoid writing output to disk inside loops. Instead, buffer output in memory and write it all at once when the loop completes.

41.8 Memory Management and avoiding large variable allocations in memory

  • Bash is not designed to store massive datasets in memory.
  • Avoid loading large log files into variable strings (e.g., logs=$(cat large.log)).
  • Best Practice: Stream data line-by-line using file descriptors and loops (e.g., while read -r line; do ... done < large.log) to keep memory usage low and constant.

41.9 Caching Command Results using associative arrays and temporary variables

If your script queries slow system resources or remote APIs repeatedly, cache the results in memory using associative arrays:

declare -A dns_cache

get_ip() {
    local host="$1"
    if [[ -z "${dns_cache[$host]:-}" ]]; then
        dns_cache[$host]=$(dig +short "$host")
    fi
    echo "${dns_cache[$host]}"
}

41.10 Diagnosing script bottlenecks and resource locks with system tools

Use standard system monitoring tools to diagnose bottlenecks:

  • top / htop: Check if CPU usage is high or if processes are blocked.
  • iotop: Monitor I/O write operations to see if disk storage is slow.
  • strace -p PID: Trace system calls made by a running process to see where it is spending its time.

Chapter 42: The Future of Bash

42.1 Bash Release History and Features in Bash 4.x and 5.x

  • Bash 4.x introduced associative arrays (declare -A), recursive globbing (globstar), and the mapfile command.
  • Bash 5.x added improvements like the $EPOCHSECONDS variable, local variable attributes, and better compatibility with POSIX standards.

42.2 Modern Shell Alternatives: Zsh, Fish, and NuShell

  • Zsh: Extends Bash with advanced tab completion, theme support, and spelling correction. It is now the default shell on macOS.
  • Fish: A user-friendly shell that provides syntax highlighting and auto-suggestions out of the box, though it is not POSIX-compliant.
  • NuShell: A modern shell that structures command output as data tables, making it easier to filter and manipulate.

42.3 The Role of Bash in Cloud-Native Infrastructures and Kubernetes

In containerized environments, lightweight shells like Bash are often used to write entrypoint scripts, build containers, and automate tasks inside Kubernetes pods (e.g., kubectl exec -it pod -- /bin/bash).

42.4 Bash in CI/CD Automation Pipelines (GitHub Actions, GitLab CI)

CI/CD platforms use shell scripts to define build steps. A single shell script can run tests, build binaries, and deploy applications across different environments:

- name: Run Build Script
  run: |
    chmod +x build.sh
    ./build.sh

42.5 Writing POSIX-Compliant Scripts for Cross-Platform compatibility

POSIX compliance ensures your scripts can run across different Unix-like systems (like macOS, BSD, and Alpine Linux).

  • Use #!/bin/sh as the shebang line.
  • Avoid using Bash-specific features like [[ ... ]] (use [ ... ] instead), arrays, and double-bracket regex matches.

42.6 Interfacing Bash with APIs: Parsing JSON outputs with jq

jq is a command-line utility for parsing and filtering JSON data.

  • Syntax:

    jq 'filter' file.json

  • Example Command:

    echo '{"user": "suresh", "role": "admin"}' | jq -r '.user'

  • Expected Output:

    suresh

  • Flag & Command Breakdown:

    • jq: JSON processor.
    • -r: Raw output mode (removes quotes around string values).
    • '.user': Filter to extract the value of the user key.

42.7 Scripting in containers: Alpine shell (ash) vs Ubuntu (bash)

  • Alpine Linux uses the BusyBox ash shell to keep container image sizes small (around 5MB). ash supports basic POSIX shell features but lacks advanced Bash features.
  • Ubuntu/Debian containers include full bash shells, which are larger but support all Bash scripting features.

42.8 The Windows Subsystem for Linux (WSL) and Bash on Windows

WSL allows developers to run a full Linux environment—including the Bash shell and command-line utilities—directly on Windows without the overhead of a traditional virtual machine.

42.9 Integration of Bash with scripting languages (Python, Go, Node.js)

While shell scripts are great for system automation, they can become hard to maintain as they grow in complexity.

  • Use Bash to handle system-level startup tasks, environment setup, and input redirection.
  • Delegate complex data processing or API integrations to languages like Python, Go, or Node.js by calling them from your shell script: python3 -c "import sys; print(sys.version)"

42.10 Why Shell Scripting remains a critical skill for DevOps and SysAdmins

Shell scripting is a fundamental skill for DevOps engineers and system administrators. It allows them to automate repetitive tasks, manage cloud infrastructure, write deployment scripts, and troubleshoot issues on remote servers where graphical user interfaces are not available.

External Resources