Course Outline (Part 4)

Welcome to Part 4 of the Linux Bash course. In this part, we transition to advanced administration tasks: utilizing the powerful awk scripting tool, securing files and directories with the Linux ownership and permission model, and managing concurrent programs using process control and jobs.

If you are running multi-container scripts or setting user groups for system directories, you will find our guide on Installing Docker on Ubuntu highly relevant. For full details on terminal control and shell parameters, consult the official GNU Bash Manual.

Chapter 10: Advanced Text Tools

10.1 Introduction to the awk Language

awk is a complete Turing-complete programming language designed for pattern scanning and processing. Named after its creators (Aho, Weinberger, and Kernighan), it reads files line-by-line, automatically splits each line into fields (columns) using whitespace as the default separator, and executes action blocks on lines matching specified patterns.

10.2 Basic awk Syntax and Printing Columns

By default, awk assigns the entire line to $0, and each column field to $1, $2, $3, and so on.

Syntax:

awk 'pattern { action }' file

Example Command:

echo "suresh web-developer techblog 2026" | awk '{print $1, $3}'

Expected Output:

suresh techblog

Flag & Command Breakdown:

  • awk: Pattern scanning programming language utility.
  • '{print $1, $3}': Action block. Instructs awk to print field 1 (suresh) and field 3 (techblog) separated by a space.

10.3 Using Custom Field Separators: awk -F

If fields are separated by characters other than spaces or tabs (e.g. commas or colons), define a custom field separator using the -F flag.

Syntax:

awk -F'separator' '{action}' file

Example Command:

awk -F':' '{print $1 " uses shell " $7}' /etc/passwd | head -n 3

Expected Output:

root uses shell /bin/bash
daemon uses shell /usr/sbin/nologin
bin uses shell /usr/sbin/nologin

Flag & Command Breakdown:

  • -F':': Instructs awk to split columns on the colon symbol.
  • $1 and $7: Prints the username and login shell path.

10.4 Filtering Lines with awk Conditions

You can add matching filters before action blocks so that the code only executes on matching lines.

Syntax:

awk 'condition {action}' file

Example Command:

awk '$3 >= 8080 {print "Alert: port " $1 " is set to " $3}' config.txt

Expected Output:

Alert: port port is set to 8080

(Evaluates the 3rd field as a number, executing print only if the value is 8080 or larger)

10.5 awk Builtin Variables (NR, NF, FS)

awk exposes internal variables to monitor the file processing status:

  • NR: Number of Records (represents the current line number, 1-indexed).
  • NF: Number of Fields (counts how many columns exist on the current line).
  • FS: Field Separator variable (stores the active separator, defaults to space).

Syntax:

awk '{print NR, NF}' file

Example Command:

awk '{print "Line " NR " has " NF " fields"}' config.txt

Expected Output:

Line 1 has 1 fields
Line 2 has 1 fields
Line 3 has 1 fields

10.6 Translating Characters: The tr Command

The tr command translates, squeezes, or deletes characters from standard input and prints the result on standard output.

Syntax:

tr [options] set1 [set2]

Example Command:

echo "hello linux world" | tr 'a-z' 'A-Z'

Expected Output:

HELLO LINUX WORLD

Flag & Command Breakdown:

  • tr: Translates lowercase range characters to uppercase range characters.

10.7 Formatting Columns: The column Command

The column command formats lists and files into clean, aligned text tables.

Syntax:

column [options] [file...]

Example Command:

column -t -s':' /etc/passwd | head -n 3

Expected Output:

root    x  0  0  root  /root      /bin/bash
daemon  x  1  1  daemon  /usr/sbin  /usr/sbin/nologin
bin     x  2  2  bin     /sbin      /usr/sbin/nologin

Flag & Command Breakdown:

  • -t: Determines the number of columns the input contains and formats them as a table.
  • -s':': Specifies the character used to delimit fields (colon).

10.8 Merging Files Line-by-Line: The paste Command

The paste command merges corresponding lines of multiple files side-by-side.

Syntax:

paste [options] file1 file2...

Example Command:

paste -d',' names.txt ages.txt

(Outputs lines of names.txt and ages.txt side-by-side, separated by a comma)

10.9 Joining Files on Common Fields: The join Command

The join command joins lines of two sorted files on a common field (similar to a SQL join operation).

Syntax:

join [options] file1 file2

Example Command:

join -t',' -1 1 -2 1 user_emails.txt user_phones.txt

(Joins both files on the 1st field of file1 and 1st field of file2 using a comma delimiter)

10.10 Generating Argument Lists: The xargs Command

The xargs command reads items from standard input (separated by spaces or newlines) and executes a specified command using those items as arguments. It allows commands that do not support standard input redirection to consume inputs.

Syntax:

command | xargs [options] target_command

Example Command:

echo "file1.txt file2.txt" | xargs rm -v

Expected Output:

removed 'file1.txt'
removed 'file2.txt'

Flag & Command Breakdown:

  • xargs: Converts standard input stream words into command arguments for rm -v.

Chapter 11: File Permissions & Ownership

11.1 Understanding the Linux Multi-User Model

Linux is designed as a secure multi-user operating system. Multiple users can log in simultaneously, each owning private home spaces. The kernel protects files and hardware devices by enforcing ownership boundaries:

  • Users: Every process runs under a specific User ID (UID).
  • Groups: Users are gathered into logical security groups sharing Group IDs (GIDs) to grant team permissions.
  • Root: The superuser account (UID 0) overrides all standard permission rules.

11.2 Decoding ls -l Output

When listing files with ls -l, look at the 10-character permission block (e.g. -rwxr-xr--):

  • Character 1 (File Type): - = regular file, d = directory, l = symbolic link.
  • Characters 2-4 (Owner/User permissions): Controls what the file owner can do.
  • Characters 5-7 (Group permissions): Controls what members of the file’s group can do.
  • Characters 8-10 (Others/World permissions): Controls what everyone else on the system can do.
  • Letters: r = read, w = write, x = execute.

11.3 User, Group, and Others

The three categories of users are abbreviated in symbolic commands:

  • u (User/Owner)
  • g (Group)
  • o (Others)
  • a (All categories combined)

11.4 Changing Permissions: Symbolic Mode

Using the symbolic mode of chmod, you can add (+), subtract (-), or explicitly assign (=) permissions.

Syntax:

chmod [category][operator][permission] file...

Example Command:

chmod u+x,g-w,o=r script.sh

Expected Output: (No terminal output, but ls -l script.sh confirms permissions were changed)

Flag & Command Breakdown:

  • chmod: Change file mode bits.
  • u+x: Adds execute permission for the file owner.
  • g-w: Removes write permission for the group.
  • o=r: Sets others’ permissions to exactly read-only (clears write and execute).

11.5 Changing Permissions: Octal/Numeric Mode

Numeric mode represents permissions using three octal digits (0-7), where each digit represents User, Group, and Others respectively:

  • r (Read) = 4
  • w (Write) = 2
  • x (Execute) = 1
  • Sum permissions together: rwx = 4+2+1 = 7. rx = 4+1 = 5. r = 4.

Syntax:

chmod [octal_number] file...

Example Command:

chmod 754 script.sh

(Sets Owner to Read/Write/Execute (7), Group to Read/Execute (5), and Others to Read-Only (4))

11.6 Changing File Owner: The chown Command

Only the superuser (root) can change file ownership.

Syntax:

chown [options] owner[:group] file...

Example Command:

sudo chown suresh:developers server.log

Expected Output: (Changes owner of server.log to user suresh and group to developers)

Flag & Command Breakdown:

  • chown: Changes ownership.
  • suresh: The target user owner.
  • :developers: Optional group assignment.

11.7 Changing File Group: The chgrp Command

Changes the group ownership of files. Regular users can use it if they own the file and belong to the target group.

Syntax:

chgrp [options] group file...

Example Command:

chgrp developers documents/

11.8 Understanding Default Permissions: The umask Command

The umask (User Mask) command defines default permissions applied when creating new files or directories. The mask values are subtracted from a system base level:

  • Base level for files: 666 (rw-rw-rw-)
  • Base level for directories: 777 (rwxrwxrwx) If umask is set to 022, new files get permission 644 (666 - 022) and directories get 755 (777 - 022).

Syntax:

umask [octal_mask]

Example Command:

umask 027

(Sets mask so new files are created with 640 (rw-r-----) and directories with 750 (rwxr-x---))

11.9 Special Permissions: SUID and SGID

  • SUID (Set User ID): When applied to an executable file, users run the program with the permissions of the file’s owner (typically root). Marked with an s in place of the owner’s x.
  • SGID (Set Group ID): When applied to directories, new files created inside inherit the group ownership of the directory rather than the group of the creating user.

11.10 Special Permissions: The Sticky Bit

When applied to a directory, the Sticky Bit prevents users from deleting or renaming files inside unless they own the file, directory, or have root access. Commonly used on /tmp. Marked with a t at the end of the permission string.

Chapter 12: Process Management

12.1 What is a Process?

A process is an active instance of a running program.

  • PID: Process ID, a unique positive integer assigned by the kernel.
  • PPID: Parent Process ID. The ID of the process that spawned it.
  • Process States:
    • R (Running / Runnable): Actively executing or waiting in CPU queue.
    • S (Interruptible Sleep): Waiting for an event or resource.
    • T (Stopped): Suspended or debugged (e.g. by pressing Ctrl+Z).
    • Z (Zombie): Terminated but still listing in the process table waiting for its parent to read its exit status.

12.2 Listing Processes: The ps Command

The ps command reports snapshots of current processes. The standard options are BSD style (aux) or UNIX style (-ef).

Syntax:

ps [options]

Example Command:

ps aux | grep bash

Expected Output:

suresh    1024  0.0  0.1  25142  9420 pts/0    Ss   13:00   0:00 /bin/bash
suresh    1450  0.0  0.0  14212  2140 pts/0    S+   13:30   0:00 grep bash

Flag & Command Breakdown:

  • a: Show processes for all users.
  • u: Display user-oriented column layouts (user, CPU%, MEM%, virtual size).
  • x: Show processes without controlling terminals (background daemons).

12.3 Detailed Process Tree: The pstree Command

pstree shows running processes in a tree structure, illustrating parent-child relationships.

Syntax:

pstree [options] [user_or_pid]

Example Command:

pstree -p suresh

Expected Output:

systemd(1)───sshd(820)───bash(1024)───pstree(1460)

Flag & Command Breakdown:

  • -p: Shows PIDs in parentheses alongside process names.

12.4 Interactive Process Monitoring: The top Command

The top command provides a real-time, interactive view of the running system, showing resource usage (CPU, Memory, Swap) and a list of processes sorted by CPU consumption.

Syntax:

top

(Inside top: q to quit, M to sort by Memory, P to sort by CPU)

12.5 Next-Generation Process Monitor: The htop Command

htop is a modern interactive process viewer. Unlike top, it supports color displays, scrolling horizontally/vertically, and mouse clicking to kill/select processes easily.

Syntax:

htop

12.6 Background Processes: Running Commands with &

Append an ampersand & to run a command in the background, freeing up your terminal prompt immediately.

Syntax:

command &

Example Command:

sleep 100 &

Expected Output:

[1] 1482

Flag & Command Breakdown:

  • [1]: Job number assigned to this command.
  • 1482: System Process ID (PID) assigned to this instance.

12.7 Listing Jobs: The jobs Command

The jobs command displays status information for tasks launched from the current shell session.

Syntax:

jobs [options]

Example Command:

jobs -l

Expected Output:

[1]+  1482 Running                 sleep 100 &

Flag & Command Breakdown:

  • -l: Lists process IDs (PIDs) alongside the job status and commands.

12.8 Suspending a Process: Ctrl+Z

Pressing Ctrl+Z sends the SIGTSTP signal to the active foreground process, suspending execution and returning control back to the prompt.

Example:

suresh@techblog-server:~$ sleep 200
^Z
[2]+  Stopped                 sleep 200

12.9 Bringing Jobs to the Foreground: The fg Command

The fg command resumes a suspended or background job, bringing it to the active foreground.

Syntax:

fg [%job_number]

Example Command:

fg %1

Expected Output:

sleep 100
*(Terminal blocks again as sleep 100 is now running in foreground)*

12.10 Sending Jobs to the Background: The bg Command

The bg command resumes a suspended job, keeping it running in the background.

Syntax:

bg [%job_number]

Example Command:

bg %2

Expected Output:

[2]+ sleep 200 &