Welcome to Part 15 of the Linux Bash Course. This is the final part of our course. It contains quick reference guides, troubleshooting tips, obscure commands, and a final capstone project.

---

Chapter 43: Quick Reference & Cheat Sheets

43.1 Bash Operators and Special Variables Cheat Sheet

• $0: Name of the script currently executing.
• $1 to $9: Positional arguments passed to the script.
• $#: Number of positional arguments passed.
• $*: All positional arguments as a single string.
• $@: All positional arguments as separate strings (safe for quoting).
• $?: Exit status of the most recently executed command (0 for success).
• $$: Process ID (PID) of the current shell.
• $!: Process ID of the most recently executed background command.

43.2 File Test Operators Quick Reference

Used inside conditional expressions to check file attributes:

• -e file: True if the file exists.
• -f file: True if the file exists and is a regular file.
• -d file: True if the file exists and is a directory.
• -s file: True if the file exists and is not empty.
• -r file: True if the file is readable.
• -w file: True if the file is writable.
• -x file: True if the file is executable.
• -h / -L file: True if the file is a symbolic link.

43.3 String and Numeric Comparison Operators Reference

• String comparisons (use double brackets [[ ... ]]):
  • == / =: Equal to.
  • !=: Not equal to.
  • <: Less than (alphabetical order).
  • >: Greater than (alphabetical order).
  • -z str: True if the string is empty.
  • -n str: True if the string is not empty.
• Numeric comparisons (use -eq style inside brackets, or standard symbols inside double parentheses (( ... ))):
  • -eq / ==: Equal to.
  • -ne / !=: Not equal to.
  • -lt / <: Less than.
  • -le / <=: Less than or equal to.
  • -gt / >: Greater than.
  • -ge / >=: Greater than or equal to.

43.4 Regular Expressions in Bash Reference

Within double brackets [[ ... ]], you can perform regular expression matching using the =~ operator:

if [[ "$email" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]]; then
    echo "Valid Email"
fi

• ^: Matches the start of the string.
• $: Matches the end of the string.
• +: Matches one or more occurrences of the preceding pattern.
• . (escaped as \.): Matches a literal dot.

43.5 Standard Exit Codes and Meanings Cheat Sheet

• 0: Success.
• 1: General catchall for errors.
• 2: Misuse of shell builtins (syntax error).
• 126: Command invoked cannot execute (permission issue).
• 127: Command not found.
• 128: Invalid argument to exit.
• 130: Script terminated by Control-C (SIGINT).
• 255: Exit status out of range.

43.6 Common Command-Line Shortcuts and Hotkeys Cheat Sheet

• Ctrl + A: Moves the cursor to the beginning of the line.
• Ctrl + E: Moves the cursor to the end of the line.
• Ctrl + U: Clears the line from the cursor to the beginning.
• Ctrl + K: Clears the line from the cursor to the end.
• Ctrl + W: Deletes the word before the cursor.
• Ctrl + R: Opens a search prompt to search through your command history.
• Ctrl + L: Clears the screen (same as the clear command).

43.7 Emacs vs Vi Editing Modes in Terminal Cheat Sheet

• By default, Bash uses Emacs-style keyboard shortcuts for line editing.
• To switch to Vi-style shortcuts, run set -o vi. You can then press Esc to enter Command mode and use keys like h, j, k, l to navigate and i to enter Insert mode.
• To switch back to Emacs-style shortcuts, run set -o emacs.

43.8 Redirection and File Descriptors Cheat Sheet

• 0: Standard Input (stdin).
• 1: Standard Output (stdout).
• 2: Standard Error (stderr).
• >: Redirects stdout, overwriting the destination.
• >>: Redirects stdout, appending to the destination.
• 2>: Redirects stderr.
• &>: Redirects both stdout and stderr.
• 2>&1: Redirects stderr to stdout.
• <: Redirects file contents to stdin.

43.9 Cron Scheduling Syntax Quick Reference

┌───────────── minute (0 - 59)
│ ┌───────────── hour (0 - 23)
│ │ ┌───────────── day of month (1 - 31)
│ │ │ ┌───────────── month (1 - 12)
│ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday)
│ │ │ │ │
* * * * *  command_to_execute

• */15 * * * *: Runs every 15 minutes.
• 0 9 * * 1-5: Runs at 9:00 AM, Monday through Friday.
• 0 0 1 * *: Runs at midnight on the first day of every month.

43.10 Basic ANSI Color Escape Codes Reference

Use these escape codes in echo -e statements to format command output:

• Black: \033[0;30m
• Red: \033[0;31m
• Green: \033[0;32m
• Yellow: \033[0;33m
• Blue: \033[0;34m
• Purple: \033[0;35m
• Cyan: \033[0;36m
• Reset formatting: \033[0m

---

Chapter 44: Obscure but Useful Commands

44.1 Locating command sources: The type and whereis commands

type displays whether a command is a shell builtin, an alias, a function, or an external file. whereis locates the binary, source, and manual page files for a command.

• Syntax:

  type command_name
  whereis command_name

• Example Command:

  type cd
  type ls

• Expected Output:

  cd is a shell builtin
  ls is aliased to `ls --color=auto'

• Flag & Command Breakdown:

  • type: Displays information about command types.

44.2 Inspecting executable dependencies: The ldd command

ldd lists the shared libraries required by a program or shared library.

• Syntax:

  ldd path_to_binary

• Example Command:

  ldd /bin/ls

• Expected Output:

  linux-vdso.so.1 (0x00007ffe09bf7000)
  libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f9c2d1b0000)
  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9c2cf88000)

• Flag & Command Breakdown:

  • ldd: Prints shared library dependencies.

44.3 Executing commands with limits: The timeout utility

timeout runs a command and terminates it if it is still running after a specified duration.

• Syntax:

  timeout duration command

• Example Command:

  timeout 3s sleep 10

• Expected Output: (Exits after 3 seconds with exit code 124)

• Flag & Command Breakdown:

  • timeout: Runs a command with a time limit.
  • 3s: Time duration (3 seconds).
  • sleep 10: The command to run.

44.4 Monitoring file adjustments: The watch command

watch runs a command repeatedly at regular intervals, letting you monitor changes in real time.

• Syntax:

  watch [options] command

• Example Command:

  watch -n 1 -d free -m

• Expected Output: (Displays the output of free -m in a full-screen window, updating it every 1 second and highlighting differences)

• Flag & Command Breakdown:

  • watch: Runs a program periodically, showing output full-screen.
  • -n 1: Runs the command every 1 second (default is 2).
  • -d: Highlights the differences between updates.

44.5 Tracking system call behaviors: The strace command

strace intercepts and records the system calls made by a process, which is useful for debugging.

• Syntax:

  strace [options] command

• Example Command:

  strace -e openat ls >/dev/null

• Expected Output:

  openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3

• Flag & Command Breakdown:

  • strace: Traces system calls and signals.
  • -e openat: Limits the trace to the openat system call.

44.6 Listing open files: The lsof command

lsof (List Open Files) displays information about files opened by processes on the system.

• Syntax:

  lsof [options]

• Example Command:

  sudo lsof -i :22

• Expected Output:

  COMMAND PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  sshd    781 root    3u  IPv4  19253      0t0  TCP *:ssh (LISTEN)

• Flag & Command Breakdown:

  • lsof: Lists open files.
  • -i :22: Limits the output to files opened by processes using port 22 (SSH).

44.7 Generating random numbers and UUIDs in Bash

• You can generate a random integer between 0 and 32767 using the $RANDOM shell variable: echo $RANDOM
• To generate a unique 128-bit identifier (UUID), read from /proc/sys/kernel/random/uuid: cat /proc/sys/kernel/random/uuid

44.8 Printing formatted strings: The printf command

printf formats and prints text, offering more control over output formatting than the echo command.

• Syntax:

  printf format_string arguments

• Example Command:

  printf "| %-10s | %04d |\n" "suresh" 42

• Expected Output:

  | suresh     | 0042 |

• Flag & Command Breakdown:

  • printf: Formats and prints data.
  • %-10s: Formats the first argument as a left-aligned string padded to 10 characters.
  • %04d: Formats the second argument as a decimal integer padded with leading zeros to 4 digits.
  • \n: Prints a newline character.

44.9 Displaying and clearing user history: The history command

history displays the command history list with line numbers.

• Syntax:

  history [options]

• Example Command:

  history 3

• Expected Output:

  1050  ls -la
  1051  cd /var/log
  1052  history 3

• Flag & Command Breakdown:

  • history: Accesses the command history.
  • 3: Returns only the 3 most recent history entries.

44.10 Terminal screen sharing and persistence: The screen and tmux utilities

tmux (Terminal Multiplexer) lets you run multiple terminal sessions in a single window. These sessions run in the background, so they persist even if you lose your SSH connection.

• Syntax:

  tmux [command]

• Example Command:

  tmux new -s mysession
  # To detach: press Ctrl+B, then press D
  # To reattach:
  tmux attach -t mysession

• Expected Output: (Opens a new persistent terminal session named mysession)

• Flag & Command Breakdown:

  • tmux: Launches the terminal multiplexer.
  • new -s mysession: Creates a new session with the name mysession.

---

Chapter 45: Scripting with External Tools

45.1 Making HTTP requests with curl and wget in scripts

curl and wget are used to query remote APIs and download dependencies in scripts.

• Syntax:

  curl [options] URL

• Example Command:

  curl -s -w "%{http_code}" -o response.json https://api.github.com/users/octocat

• Expected Output:

  200

• Flag & Command Breakdown:

  • curl: Command Line URL client.
  • -s: Silent mode (suppresses progress meters).
  • -w "%{http_code}": Appends the HTTP response status code to the output.
  • -o response.json: Saves the response payload to a file.

45.2 Parsing and editing JSON payloads using jq

jq is the standard tool for parsing JSON data in shell environments.

• Syntax:

  jq 'filter' file.json

• Example Command:

  echo '{"status": "ok", "code": 200}' | jq '.status'

• Expected Output:

  "ok"

• Flag & Command Breakdown:

  • jq: Command line JSON processor.
  • '.status': Extracts the value associated with the status key.

45.3 Manipulating YAML configurations using yq

yq works like jq but is designed for parsing YAML files.

• Syntax:

  yq 'filter' file.yaml

• Example Command:

  echo -e "version: 1.0\nname: app" | yq '.name'

• Expected Output:

  app

• Flag & Command Breakdown:

  • yq: Command line YAML processor.
  • '.name': Extracts the value of the name key.

45.4 Parsing XML and HTML formats with xmllint

xmllint formats, validates, and parses XML files using XPath queries.

• Syntax:

  xmllint --xpath 'xpath_expression' file.xml

• Example Command:

  echo '<user><name>Suresh</name></user>' | xmllint --xpath 'string(/user/name)' -

• Expected Output:

  Suresh

• Flag & Command Breakdown:

  • xmllint: XML processing utility.
  • --xpath: Evaluates the specified XPath query.
  • -: Tells the command to read from standard input.

45.5 Reading and writing database tables with CLI clients (sqlite3, psql)

Database command-line interfaces let you run SQL queries directly from your scripts.

• Syntax:

  sqlite3 database_file "SQL query"

• Example Command:

  sqlite3 users.db "CREATE TABLE IF NOT EXISTS users(name TEXT); INSERT INTO users VALUES('suresh'); SELECT * FROM users;"

• Expected Output:

  suresh

• Flag & Command Breakdown:

  • sqlite3: CLI interface for SQLite databases.

45.6 Generating interactive terminal dialogs with dialog and whiptail

whiptail displays interactive dialog boxes (like menus, message boxes, and input prompts) in the terminal.

• Syntax:

  whiptail --title "title" --msgbox "message" height width

• Example Command:

  whiptail --title "Setup Complete" --msgbox "The packages were installed successfully." 8 45

• Expected Output: (Displays a text-based confirmation dialog box with an “Ok” button)

• Flag & Command Breakdown:

  • whiptail: Dialog box utility.
  • --title: Sets the dialog box title.
  • --msgbox: Specifies the message text.
  • 8 45: Dimensions of the dialog box (8 lines high, 45 characters wide).

45.7 Sending SMTP emails directly from script tasks with mail and sendmail

You can send automated email alerts from your scripts using the mail command.

• Syntax:

  mail -s "Subject" email_address

• Example Command:

  echo "The backup job completed successfully." | mail -s "Backup Status" [email protected]

• Expected Output: (No output in the console. Sends the email via the local mail server)

• Flag & Command Breakdown:

  • mail: Sends and receives email.
  • -s: Specifies the email subject line.

45.8 Orchestrating remote servers with ssh inside loops

You can run commands on multiple remote servers in parallel by wrapping SSH calls in loops:

for server in 192.168.1.100 192.168.1.101; do
    ssh -o ConnectTimeout=5 -n suresh@"$server" "uname -r" &
done
wait

• -n: Redirects stdin to /dev/null, which is required to prevent SSH from consuming the loop’s input.

45.9 Syncing folders efficiently across hosts with rsync

rsync copies files securely between directories or remote systems, reducing bandwidth usage by sending only the differences between files.

• Syntax:

  rsync [options] source destination

• Example Command:

  rsync -avz --delete /home/suresh/documents/ remote_server:/backups/documents/

• Expected Output:

  sending incremental file list
  notes.txt
  
  sent 120 bytes  received 35 bytes  310.00 bytes/sec
  total size is 1024  speedup is 6.61

• Flag & Command Breakdown:

  • rsync: File sync utility.
  • -a: Archive mode (recursively copies directories and preserves file permissions, symlinks, and timestamps).
  • -v: Verbose output.
  • -z: Compresses file data during transfer.
  • --delete: Deletes files from the destination directory if they no longer exist in the source directory.

45.10 Automating interactive prompts using expect scripts

expect is a tool for automating interactive command-line programs (like SSH connections or command prompts that ask for password inputs).

• A basic expect script structure:

  #!/usr/bin/expect -f
  spawn ssh [email protected]
  expect "password:"
  send "my_secure_password\r"
  interact

---

Chapter 46: Troubleshooting Common Errors

46.1 Fixing “bash: ./script.sh: Permission denied”

• Cause: The script file does not have execute permissions.
• Fix: Run chmod +x script.sh to make the file executable, or run the script using the interpreter directly: bash script.sh.

46.2 Fixing “bash: ./script.sh: /bin/bash^M: bad interpreter”

• Cause: The script was saved with Windows carriage return characters (\r\n) instead of Linux line endings (\n).
• Fix: Convert the file to Unix line endings using the dos2unix utility:

  dos2unix script.sh

46.3 Troubleshooting “command not found” errors

• Cause: The command is not installed on the system, or its directory is not included in the active shell’s $PATH variable.
• Fix: Install the command using your package manager (e.g. sudo apt install command_name), or specify the absolute path to the executable (e.g. /usr/local/bin/mytool).

46.4 Resolving “syntax error: unexpected end of file”

• Cause: A block construct was left open (e.g., a for loop without a matching done, a then without a matching fi, or unmatched quotes).
• Fix: Use static code analysis tools like shellcheck to find the line where the code block was left open, and make sure all loops, conditionals, and quotes are closed.

46.5 Fixing “unary operator expected” errors in conditional checks

• Cause: A variable inside a single-bracket conditional test [ "$var" == "val" ] was empty or undefined, causing the expression to fail syntax validation.
• Fix: Wrap variables in double quotes, or switch to double brackets [[ $var == "val" ]] which handle empty variables safely.

46.6 Resolving variable scoping issues and the local keyword

• By default, all variables in Bash are global, meaning they can be modified from anywhere in the script.
• Best Practice: Use the local keyword when defining variables inside functions to keep them scoped to that function:

  my_func() {
      local temp_var="value"
  }

46.7 Troubleshooting pipeline and redirection errors

• If you notice pipeline outputs are missing, make sure you are redirecting standard error (2>) as well as standard output.
• Avoid writing to the same file you are reading from in a pipeline (e.g., cat list.txt | grep "suresh" > list.txt), as this will truncate the file before it is read. Use a temporary file instead.

46.8 Fixing script hangings and identifying infinite loops

• If a script is hanging, check if it is waiting for user input (e.g., a read command) or blocked by a command that is waiting for a network response (e.g., curl without timeouts).
• Press Ctrl + Z to suspend the script, check its state using ps, and terminate it using kill -9 %1.

46.9 Resolving dynamic array expansion and index errors

• Standard array expansions like $my_array only return the first element (index 0) of the array.
• Fix: Use the ${my_array[@]} syntax to expand all elements in the array, and use ${#my_array[@]} to get the array’s total length.

46.10 Debugging script outputs using set -x and set -v

You can trace how a script runs by enabling debug flags:

• set -v: Prints input lines as they are read by the shell.
• set -x: Prints each command and its expanded arguments before it is executed.
• You can enable these flags directly when running a script: bash -x script.sh.

---

Chapter 47: Final Exam & Capstone

47.1 Final Capstone Project Overview: Automating server setups

In this capstone project, you will build a production-ready system configuration script. The script will validate the user environment, install packages, partition a disk, set up service accounts, configure a systemd service, and schedule backups.

47.2 Capstone Step 1: Validating user environments and OS distributions

Verify that the script is running with superuser privileges and on a supported Linux distribution:

if [ "$EUID" -ne 0 ]; then
    echo "Error: This script must be run as root." >&2
    exit 1
fi

if [ ! -f /etc/debian_version ]; then
    echo "Error: This script is only supported on Debian/Ubuntu systems." >&2
    exit 1
fi

47.3 Capstone Step 2: Designing custom interactive user setup menus

Create a text-based setup menu using a case loop:

echo "=== SERVER SETUP MENU ==="
echo "1. Run Full Auto-Setup"
echo "2. Perform Dry-Run"
echo "3. Exit"

47.4 Capstone Step 3: Installing required system tools and packages

Update your package index and install the dependencies required for the setup:

apt-get update -y
apt-get install -y nginx curl rsync sqlite3

47.5 Capstone Step 4: Formatting and partitioning secondary disks

Format the secondary disk /dev/sdb with the EXT4 filesystem:

if lsblk /dev/sdb >/dev/null 2>&1; then
    mkfs.ext4 -F /dev/sdb
    mkdir -p /data
    mount /dev/sdb /data
fi

47.6 Capstone Step 5: Creating application users, groups, and folders

Create a service group and user account for the application, then configure folder ownership:

groupadd -r appgroup
useradd -r -g appgroup -d /var/www/app -s /usr/sbin/nologin appuser
mkdir -p /var/www/app
chown -R appuser:appgroup /var/www/app

47.7 Capstone Step 6: Deploying service configs and systemd scripts

Create a custom systemd service unit file for the application at /etc/systemd/system/webapp.service:

[Unit]
Description=Web Application
After=network.target

[Service]
Type=simple
User=appuser
ExecStart=/usr/bin/python3 -m http.server 8080 --directory /var/www/app
Restart=on-failure

[Install]
WantedBy=multi-user.target

Run systemctl daemon-reload, then enable and start the service.

47.8 Capstone Step 7: Configuring automated backup schedules with cron

Create a cron job that runs a backup script every day at midnight and saves the cron configuration under /etc/cron.d/webapp-backup:

0 0 * * * root tar -zcvf /backups/app-$(date +\%F).tar.gz /var/www/app

47.9 Capstone Step 8: Adding monitoring alerts and log tracing

Add logging statements to track the setup progress, and write error alerts to /var/log/syslog using the logger utility if any step fails.

47.10 Capstone Step 9: Running tests, validating setup, and execution

Assemble the components into a single script named setup_server.sh. Run the script, check its output, and verify that the systemd service is active:

chmod +x setup_server.sh
sudo ./setup_server.sh
systemctl status webapp.service

---

Congratulations!

You have completed the Linux Bash Scripting & Administration Course. You now have the skills to write robust shell scripts, automate system tasks, manage network configurations, and troubleshoot issues on Linux systems.

External Resources

• GNU Bash Reference Manual
• Advanced Bash-Scripting Guide
• Debian Administrator’s Handbook