Shell Scripting Basics in Linux

1. What is a Script?

A script is a list of commands stored in a file that can be executed as a program.

• Unlike compiled programs, scripts are interpreted at runtime.
• They are written in scripting languages like:
  • Bash (Bourne Again SHell)
  • Python
  • Perl
  • Ruby

Key Characteristics

2. Common Use Cases for Shell Scripts

Shell scripting is widely used in system administration and automation:

Shell scripts help save time and reduce human error by automating routine tasks.

3. The Shebang Line (#!) – Interpreter Directive

The first line of any shell script should be the shebang, also known as the interpreter directive.

Purpose

Tells the system which interpreter to use when running the script.

Syntax

#!/path/to/interpreter [optional_argument]

Examples

Without a shebang, the script may run using the current shell — which could lead to unexpected behavior.

4. Creating Your First Shell Script: “Hello World”

Step-by-step Instructions

Step 1: Create an empty file

touch hello_world.sh

.sh is a common extension for shell scripts (not required, but good practice).

Step 2: Add the shebang and command

echo '#!/bin/bash' >> hello_world.sh
echo 'echo "Hello World"' >> hello_world.sh

This creates a script that runs in Bash and prints “Hello World”.

Step 3: Make the script executable

chmod +x hello_world.sh

Before this, the file was readable and writable, but not executable.

Use ls -l hello_world.sh to check permissions:

-rw-r--r-- 1 user group 0 Jan 1 12:00 hello_world.sh   # Not executable

After chmod +x:

-rwxr-xr-x 1 user group 0 Jan 1 12:00 hello_world.sh   # Now executable

Step 4: Run the script

./hello_world.sh

Output:

Hello World

5. Understanding File Permissions

Linux uses a permission model to control access to files.

File Permission Breakdown

Using ls -l, you’ll see something like:

-rwxr-xr-x 1 user group 0 Jan 1 12:00 hello_world.sh

• -rwx → Owner (user) has Read, Write, Execute
• r-x → Group has Read and Execute
• r-x → Others have Read and Execute

Changing Permissions with chmod

chmod +x filename.sh     # Add execute permission for all users
chmod u+x filename.sh    # Add execute only for owner
chmod go-r filename.sh   # Remove read for group and others

6. Summary of Key Concepts

7. Quick Reference Table

8. Final Tips

✅ Always start your shell script with a shebang line (#!/bin/bash)
✅ Use descriptive names for your scripts
✅ Test small scripts before scaling them up
✅ Keep scripts simple and modular
✅ Comment your code to explain what it does
✅ Use chmod to make scripts executable
✅ Run scripts with ./script.sh after setting permissions

9. Bonus: Anatomy of a Simple Bash Script

#!/bin/bash
# This is a comment
echo "Welcome to Shell Scripting!"

Save this as welcome.sh, make it executable, and run it!

🧾 Reading: A Brief Introduction to Shell Variables

In this reading, you were introduced to one of the most powerful and essential features in shell scripting — shell variables. These allow you to store and manipulate data directly in the terminal or within scripts.

🎯 Learning Objectives Recap

After completing this reading, you can now:

✅ Describe what shell variables are
✅ Create and use your own shell variables
✅ Read user input into a variable using read

💡 What Is a Shell Variable?

A shell variable is a named storage location that holds a value — such as a string, number, or command output — for use in the shell environment.

🔹 Example:

firstname=Jeff
echo $firstname

Output:

Jeff

• firstname=Jeff → Assigns the value Jeff to the variable firstname
• $firstname → Retrieves (expands) the value stored in the variable

⚠️ No spaces around the = when assigning values:

# Correct
name=Linux

# Incorrect
name = Linux  ❌

🧩 Creating Shell Variables

🔹 Direct Assignment

You can create a variable by simply giving it a name and assigning a value:

greeting="Hello World"
echo $greeting

Output:

Hello World

Quotes help preserve whitespace and special characters.

🔹 Reading User Input with read

The read command lets you capture input from the user and store it in a variable.

Example:

read lastname
Grossman
echo $lastname

Output:

Grossman

This is especially useful in scripts, where you want to interactively gather information from users.

🧑‍💻 Combining Variables

You can combine multiple variables in one command:

echo $firstname $lastname

Output:

Jeff Grossman

This makes it easy to build dynamic messages or paths based on user input or system state.

📝 Summary Table

🧠 Why Shell Variables Matter

Shell variables are essential because they allow you to:

• Store temporary data during a session
• Make scripts interactive with user input
• Customize behavior of shell scripts
• Reuse values without hardcoding them

They’re the foundation of automation in Bash scripting!

🛠️ Real-World Use Cases

📜 Exercise 1 – Create and Execute a Basic Shell Script

In this hands-on exercise, you created your first Bash shell script called greet.sh. You used:

• Shell variables to store user input
• The echo command to print messages
• Comments (#) to document the script
• The read command to accept input from the user

You then executed it using bash greet.sh.

✅ Step-by-Step Breakdown

🔹 1.1 Create a New Script File

Steps:

1. Opened a new file in the editor (via File > New File)
2. Saved it as: greet.sh
3. Pasted the following Bash script:

# This script accepts the user's name and prints
# a message greeting the user

# Print the prompt message on screen
echo -n "Enter your name :"

# Wait for user to enter a name, and save the entered name into the variable 'name'
read name

# Print the welcome message followed by the name
echo "Welcome $name"

# The following message should print on a single line. Hence the usage of '-n'
echo -n "Congratulations! You just created and ran your first shell script "
echo "using Bash on IBM Skills Network"

💡 Tip: The -n option in echo prevents the automatic newline — useful when printing across multiple lines.

4. Saved the file using File > Save

🔹 1.2 Execute the Script

Step 1: Open a Terminal

• Clicked Terminal > New Terminal to open a terminal window

Step 2: Check File Permissions

ls -l greet.sh

Make sure it exists and is readable.

Step 3: Run the script

bash greet.sh

What Happens:

• It prompts:

  Enter your name :

• You type your name and press Enter
• It responds with:

  Welcome [YourName]
  Congratulations! You just created and ran your first shell script using Bash on IBM Skills Network

✅ Success! You’ve run your first Bash script.

🧠 Why This Matters

This simple script introduces several key scripting concepts:

These are the building blocks of Bash scripting, which powers automation in Linux environments.

🛠️ Real-World Scripting Tips

• Make scripts executable:

  chmod +x greet.sh
  ./greet.sh

• Always test your scripts before sharing them
• Use comments to help others understand your code
• Use descriptive variable names like username, not x

🔧 Exercise 2 – Using a Shebang Line

In this exercise, you took your first Bash script (greet.sh) to the next level by:

• Adding a shebang line (#!) to specify the interpreter
• Making it executable, so you can run it like a regular command: ./greet.sh

This is a foundational step toward writing portable and reusable shell scripts.

📋 Step-by-Step Breakdown

🔹 2.1 Find the Path to the Bash Interpreter

You used the which command to locate where bash is installed:

which bash

Output:

/bin/bash

This tells you that the Bash shell is located at /bin/bash.

🔹 2.2 Add the Shebang Line to Your Script

You opened greet.sh and added the shebang line at the very top:

#!/bin/bash

This line tells the system:

“Use /bin/bash to interpret this script.”

Your updated script now looks like:

#!/bin/bash

# This script accepts the user's name and prints
# a message greeting the user

echo -n "Enter your name :"
read name
echo "Welcome $name"
echo -n "Congratulations! You just created and ran your first shell script "
echo "using Bash on IBM Skills Network"

✅ This makes your script self-contained and portable — any system with Bash can now execute it.

🔹 2.3 Make the Script Executable

First, check current permissions:

ls -l greet.sh

It likely looked like:

-rw-r--r-- 1 user group 200 Apr 5 10:00 greet.sh

No execute permission yet!

Add execute permission for the owner:

chmod u+x greet.sh

Or make it executable for everyone:

chmod +x greet.sh

Verify the new permissions:

ls -l greet.sh

Now it should show:

-rwxr-xr-x 1 user group 200 Apr 5 10:00 greet.sh

The x means the file is now executable.

🔹 2.4 Run the Script Like a Command

You executed your script using:

./greet.sh

• ./ tells the shell to look in the current directory
• greet.sh is your self-executable script

You were prompted:

Enter your name :

After entering your name, you saw:

Welcome [YourName]
Congratulations! You just created and ran your first shell script using Bash on IBM Skills Network

✅ Congratulations again — you’ve successfully made your script runnable as a command!

📌 Summary Table

🧠 Why This Matters

By adding the shebang line and making the script executable, you’ve learned how to:

• Turn a text file into a real Linux command
• Define which interpreter should run the script
• Improve script portability across systems
• Control who can run the script via permissions

These are essential steps in creating custom automation tools in Linux.

💡 Pro Tips

• Always test scripts with:

  bash greet.sh

  Before making them executable — helps catch syntax errors early.
• Use descriptive names for your scripts
• Keep your scripts in a dedicated folder like ~/scripts/ and add it to your PATH later
• Use comments liberally to explain what your script does

Filters, Pipes, and Variables in Linux Shell

1. Introduction

This video introduces three powerful features of the Linux shell:

✅ Filters – commands that process input and produce output
✅ Pipes (|) – a way to chain filters together
✅ Shell & Environment Variables – for storing and managing values

These tools are essential for building complex command chains and managing configuration in scripts and interactive sessions.

2. What Are Filters?

Definition

A filter is a command or program that:

• Accepts input from standard input (stdin) (e.g., keyboard)
• Processes or transforms the data
• Sends output to standard output (stdout) (e.g., terminal)

Common Filter Commands

Example of a Filter Chain

ls | sort -r

Takes output from ls and sorts it in reverse order.

3. The Pipe Operator (|) – Chaining Filters Together

Purpose

The pipe operator allows you to connect the output of one command as input to another, forming a pipeline.

Syntax

command1 | command2 | command3

Examples

Example 1: List variables and show only the first few

set | head -n 4

set lists all shell variables; head -n 4 shows only the first four lines.

Example 2: List environment variables matching a pattern

env | grep GREE

env lists all environment variables; grep GREE filters for those containing “GREE”.

Example 3: Count number of processes

ps | wc -l

Counts how many running processes there are.

4. Shell Variables

What Are Shell Variables?

Variables defined within a shell session that are not available to child processes (like subshells or other programs).

Creating a Shell Variable

GREETINGS=hello
AUDIENCE=world

⚠️ No spaces around the = sign.

Accessing Variable Values

Use the $ symbol to access variable contents:

echo $GREETINGS $AUDIENCE

Output:

hello world

Listing All Shell Variables

set

Shows all shell variables and functions visible in the current shell.

Removing a Shell Variable

unset AUDIENCE

Removes the variable AUDIENCE.

5. Environment Variables

What Are Environment Variables?

Environment variables are like shell variables but with global scope — they are visible to the current shell and any child processes.

Promoting a Shell Variable to Environment Variable

export GREETINGS

Now GREETINGS is an environment variable.

Listing All Environment Variables

env

Or filter using grep:

env | grep GREETINGS

Confirms GREETINGS is now an environment variable.

6. Key Differences Between Shell and Environment Variables

7. Summary Table: Commands for Working with Filters, Pipes, and Variables

| Task | Command | Description | | ---------------------------------- | ------------ | -------------------------------------- | ---------------------------------------- | | Run two commands in sequence | cmd1 | cmd2 | Output of cmd1 becomes input of cmd2 | | Count lines in a directory listing | ls | wc -l | Lists files and counts them | | View only top 4 shell variables | set | head -n 4 | Lists shell vars and shows top 4 | | Create a shell variable | VAR=value | No spaces around = | | Access variable value | echo $VAR | Use $ to get value | | Remove a variable | unset VAR | Deletes variable from shell | | Make shell var an env var | export VAR | Makes var available to child processes | | List all environment variables | env | Shows global variables | | Filter environment variables | env | grep PATTERN | Search for specific variables |

8. Full “Hello World” Example Using Variables

GREETINGS=hello
AUDIENCE=world
echo $GREETINGS $AUDIENCE
# Output: hello world

unset AUDIENCE
export GREETINGS
env | grep GREETINGS
# Output confirms GREETINGS is now an environment variable

9. Final Tips

• Use pipes to build powerful one-liners.
• Use filters to transform and analyze data.
• Use shell variables for temporary values inside scripts.
• Use environment variables to pass values to sub-processes or configure applications.
• Always check your syntax when assigning variables — no spaces around =.
• Combine these tools for automation, analysis, and scripting.

🔗 Examples of Pipes in Linux

Pipes (|) are one of the most powerful features in Linux and Bash scripting. They allow you to connect commands, using the output of one as input for the next.

🎯 Learning Objectives Recap

After this reading, you can now:

✅ Describe what pipes are
✅ Use pipes to combine commands like sort, uniq, tr, etc.
✅ Apply pipes to extract information from strings and files
✅ Extract specific data (like Bitcoin price) from JSON or URLs using grep and pipes

🔧 What Are Pipes?

A pipe takes the output of one command and feeds it as input to another.

🔁 Basic Syntax:

command1 | command2 | command3

There’s no limit on how many commands you can chain together!

💡 This is known as a pipeline — a sequence of processes where the output of each step becomes the input of the next.

🔄 Pipe Examples: Combining Commands

You used sort and uniq together to clean up text and remove duplicates.

Example: Remove all duplicate lines from pets.txt

sort pets.txt | uniq

• sort: puts identical lines together
• uniq: removes consecutive duplicates

Together, they give you a list of unique animals:

cat
dog
goldfish
parrot

📄 Applying Commands to Strings and Files

Some tools like tr work with standard input only — so we use pipes to apply them to strings and file contents.

Replace vowels in a string with _:

echo "Linux and shell scripting are awesome!" | tr "aeiou" "_"

Output:

L_n_x _nd sh_ll scr_pt_ng _r_ _w_s_m_!

Convert entire file to uppercase:

cat pets.txt | tr "[a-z]" "[A-Z]"

Combine with sort and uniq:

sort pets.txt | uniq | tr "[a-z]" "[A-Z]"

Result:

CAT
DOG
GOLDFISH
PARROT

🪣 Extracting Data from JSON Files

You practiced extracting structured data from a JSON file using grep and regular expressions.

Given JSON content in Bitcoinprice.txt:

{
  "coin": {
    "price": 57907.78008618953,
    ...
  }
}

Extract the "price" field:

cat Bitcoinprice.txt | grep -oE "\"price\"\s*:\s*[0-9]*\.?[0-9]*"

Explanation:

• -o → show only matched part
• -E → enable extended regex
• \"price\" → match "price"
• \s* → match any amount of whitespace
• : → match colon
• [0-9]*\.?[0-9]* → match numbers with optional decimal point

This gives:

"price" : 57907.78008618953

🌐 Extracting Info from URLs

You can combine curl or wget with grep to extract live data from web APIs or pages.

Example: Get current Bitcoin price from an API:

curl -s https://api.coinstats.app/public/v1/coins/bitcoin | grep -oE "\"price\": *[0-9]+\.?[0-9]*"

Or if you have the JSON saved locally:

cat Bitcoinprice.txt | grep -oE "\"price\": *[0-9]+\.?[0-9]*"

📋 Summary Table: Useful Pipe Combinations

| Task | Command | | ------------------------------ | ------------------------------------------ | --------------- | | Sort and deduplicate lines | sort file.txt | uniq | | Replace characters in a string | echo "text" | tr "from" "to" | | Convert text to uppercase | tr "[a-z]" "[A-Z]" | | Extract price from JSON | grep -oE "\"price\": *[0-9.]*" file.json | | Download and parse data | curl example.com/data.json | grep pattern |

🧠 Why Pipes Matter

Using pipes lets you:

• Automate complex text processing
• Chain small tools into powerful pipelines
• Extract and transform data efficiently
• Build custom scripts that process logs, config files, or API responses

They’re at the heart of Unix philosophy: Do one thing well, and connect it with others.

✅ Final Takeaways

| Concept | Description | | ------------------- | ------------------------------------------------------------------- | --------------------------------------------------- | | Pipes ( |) | Connect outputs of one command to inputs of another | | Filters | Tools like grep, uniq, tr act on streamed input | | Data extraction | Use grep with regex to pull values from structured text like JSON | | Automation | Combine curl + grep to extract live data from the web |

Useful Features of the Bash Shell

1. Introduction

✅ Metacharacters – special characters with built-in meanings
✅ Quoting & Escaping – control how the shell interprets commands
✅ I/O Redirection – manage input and output streams
✅ Command Substitution – insert results from one command into another
✅ Command Line Arguments – pass values to scripts
✅ Running Commands in Batch or Concurrent Mode

These tools allow for powerful automation, customization, and efficiency.

2. Metacharacters – Special Characters in Bash

Definition

Metacharacters are symbols interpreted by the shell to perform specific operations.

3. Quoting – Literal vs Interpreted Text

A. Single Quotes ('...')

• Treats everything inside as literal text
• Ignores variable substitution and metacharacter interpretation

Example:

echo '$HOME is /home/user'

Output:

$HOME is /home/user

B. Double Quotes ("...")

• Allows some interpretation, such as:
  • Variable expansion ($VAR)
  • Command substitution ($(...) or backticks)

Example:

echo "$HOME is my home directory"

Output (if $HOME=/home/user):

/home/user is my home directory

C. Backslash (\) – Escaping Metacharacters

Use backslash to prevent interpretation of special characters.

Example:

echo "The cost is \$1 only"

Output:

The cost is $1 only

4. Input/Output (I/O) Redirection

Used to redirect standard input, output, and error streams.

Example:

echo "line1" > eg.txt
echo "line2" >> eg.txt
cat eg.txt

Output:

line1
line2

5. Command Substitution – Run Command Inside Another

Allows you to run a command and use its output as part of another command.

Syntax Options

| Syntax | Description | | --------------- | ----------------------- | ---------------------------- | | $(command) | Preferred modern syntax | echo "Current dir: $(pwd)" | | `command` | Older backtick syntax | echo "Today is date" |

Example:

here=$(pwd)
echo "You are in: $here"

Output:

You are in: /home/user

6. Command Line Arguments

When running a script, you can pass arguments to it using the command line.

Accessing Arguments in Script

Example:

./myscript.sh arg1 arg2

Inside myscript.sh:

echo "Script name: $0"
echo "First argument: $1"
echo "Second argument: $2"
echo "All arguments: $@"
echo "Number of arguments: $#"

7. Running Commands Sequentially vs Concurrently

A. Batch Mode – Sequential Execution

Commands are executed one after another.

Example:

sleep 2; echo "Done"

• sleep 2 runs first.
• echo "Done" runs after it finishes.

B. Concurrent Mode – Parallel Execution

Use & to run a command in the background.

Example:

long_process.sh & echo "Continuing..."

• long_process.sh runs in the background.
• echo runs immediately without waiting.

8. Summary Table

9. Full Example: Combining Concepts

#!/bin/bash
# This is a comment

message="Script started at $(date)"
echo "$message" > log.txt

echo "Writing to log file..." >> log.txt

# Try something that causes an error
ls /invalid_path 2> error.txt

# Print contents of files
cat log.txt
cat error.txt

Output:

Script started at [current date]
Writing to log file...
ls: cannot access '/invalid_path': No such file or directory

10. Final Tips

• Use metacharacters to write compact and powerful commands.
• Use quoting to control whether variables and wildcards get expanded.
• Use redirection to capture outputs or errors into files.
• Use command substitution to dynamically insert command results.
• Use command line arguments to make your scripts more flexible.
• Use batch mode for sequential execution, concurrent mode for background tasks.
• Combine these features for efficient and readable shell scripts.

🧠 Examples of Bash Shell Features – Summary & Highlights

In this reading, you explored several powerful features of the Bash shell, which are essential for writing clean, effective, and dynamic scripts.

🔤 Metacharacters in Bash

These characters have special meanings to the shell and help control how commands behave.

Examples:

# This is a comment
echo "Hello"; echo "World"  # Runs both commands
ls *.txt                      # Lists all .txt files
ls file?.txt                  # Matches file1.txt, file2.txt, etc.

“ ” ' ' \ Quoting and Escaping

Use quoting to control how the shell interprets special characters.

Examples:

touch file\ with\ space.txt   # Creates a file with spaces in name
echo "Today is $USER"         # Expands variables like $USER
echo 'Today is $USER'         # Prints literal "$USER"

📥 Input/Output Redirection

You can redirect input (<), output (> or >>), and error streams (2>, 2>>) to or from files.

Examples:

ls > files.txt          # Saves ls output to files.txt
ls >> files.txt         # Appends more data
grep "error" log.txt > errors.txt  # Saves only matching lines
sort < data.txt         # Sorts content from data.txt

💡 Command Substitution

Allows you to capture and reuse command outputs as values in your scripts.

Syntax:

• Backticks: `command` (legacy)
• Preferred: $(command) (modern and nested-friendly)

Example:

here=$(pwd)
cd /tmp
cd $here

This lets you store and reuse results dynamically in scripts — great for automation!

📝 Command Line Arguments

Arguments passed to a script when it’s executed. You can access them using positional parameters.

Example:

./myscript.sh arg1 arg2

Inside the script:

echo "First argument: $1"
echo "Second argument: $2"
echo "All arguments: $@"

Positional Parameters:

🔄 Why These Bash Features Matter

✅ Real-World Examples

1. Save and restore directory:

current_dir=$(pwd)
cd /tmp
cd "$current_dir"

2. Log output and errors separately:

my_script.sh > output.log 2> error.log

3. Pass and use arguments in a script:

#!/bin/bash
echo "Script name: $0"
echo "First argument: $1"
echo "Number of arguments: $#"

Run it:

chmod +x myscript.sh
./myscript.sh Hello World!

Output:

Script name: ./myscript.sh
First argument: Hello
Number of arguments: 2

🛠️ Best Practices

• Always quote variables to avoid word splitting:

  echo "$USER"

• Use $(...) over backticks for better readability and nesting
• Use > carefully — it overwrites by default
• Use >> to append instead
• Include comments using # for clarity and documentation

Great job learning these key Bash shell features!
You now know how to:

• Control character interpretation with quoting
• Manipulate input/output with redirection
• Make scripts dynamic with command substitution
• Build flexible scripts that accept command-line arguments

🧠 Exercise 1 – Using Conditional Statements and Logical Operators in Bash

In this exercise, you learned how to use conditional logic in a Bash script — specifically using the if, elif, and else structure.

You built a script that:

• Asks the user a yes/no question
• Stores their response
• Responds differently depending on whether they typed "y" or "n"
• Handles invalid input gracefully

This is a foundational skill for writing interactive shell scripts.

✅ Step-by-Step Breakdown

🔹 1.1 Create a New Bash Script

Create and make it executable:

echo '#!/bin/bash' > conditional_script.sh
chmod u+x conditional_script.sh

This creates a new script file with execute permissions for the owner.

🔹 1.2 Prompt the User and Store Their Response

Add code to ask a question and store input:

#!/bin/bash
echo 'Are you enjoying this course so far?'
echo -n "Enter \"y\" for yes, \"n\" for no: "
read response

• echo prints messages to the screen
• echo -n prevents a newline after printing (keeps prompt on same line)
• read response stores user input in a variable called response

🔹 1.3 Use a Conditional Block Based on User Input

Add the conditional logic:

if [ "$response" == "y" ]; then
    echo "I'm pleased to hear you are enjoying the course!"
    echo "Your feedback regarding what you have been enjoying would be most welcome!"

elif [ "$response" = "n" ]; then
    echo "I'm sorry to hear you are not enjoying the course."
    echo "Your feedback regarding what we can do to improve the learning experience"
    echo "for this course would be greatly appreciated!"

else
    echo "Your response must be either 'y' or 'n'."
    echo "Please re-run the script to try again."
fi

📝 Explanation:

🧪 Sample Outputs

If user enters y:

I'm pleased to hear you are enjoying the course!
Your feedback regarding what you have been enjoying would be most welcome!

If user enters n:

I'm sorry to hear you are not enjoying the course.
Your feedback regarding what we can do to improve the learning experience
for this course would be greatly appreciated!

If user enters something else:

Your response must be either 'y' or 'n'.
Please re-run the script to try again.

📋 Summary Table

🛠️ Real-World Uses of Conditional Logic

Conditional statements like this are used for:

• Validating user input in setup scripts
• Checking system conditions before performing actions
• Controlling flow in automation tools
• Writing interactive CLI apps in Bash

💡 Pro Tips

• Always quote variables to avoid unexpected behavior:

  if [ "$response" = "y" ]; then

• Use [[ ... ]] instead of [ ... ] for advanced features (like regex matching) in modern Bash
• You can simplify responses by converting input to lowercase:

  response="${response,,}"

🧮 Exercise 2 – Performing Basic Math and Logical Comparisons in Bash

In this hands-on exercise, you created a Bash script that:

• Accepts two integers from the user
• Performs basic math: addition and multiplication
• Compares results using numeric conditional operators
• Outputs comparison results to the user

This helps you understand how to work with arithmetic operations, variables, and numerical logic in shell scripting.

✅ Step-by-Step Breakdown

🔹 2.1 Create a Bash Script for Input and Calculations

You created a script that takes two integers as input:

#!/bin/bash
echo -n "Enter an integer: "
read n1
echo -n "Enter another integer: "
read n2

Then calculated:

sum=$(($n1 + $n2))
product=$(($n1 * $n2))

And printed the result:

echo "The sum of $n1 and $n2 is $sum"
echo "The product of $n1 and $n2 is $product."

📝 The syntax $(()) is used in Bash for arithmetic expansion — it evaluates mathematical expressions.

🔹 2.2 Add Logic to Compare Results

You added an if-elif-else block to compare the sum and product:

if [ $sum -lt $product ]; then
   echo "The sum is less than the product."
elif [[ $sum == $product ]]; then
   echo "The sum is equal to the product."
elif [ $sum -gt $product ]; then
   echo "The sum is greater than the product."
fi

⚠️ Note: You used -lt, -gt, and == for numeric comparisons.

🧪 Sample Output Scenarios

Case 1: Sum < Product

Enter an integer: 3
Enter another integer: 5
The sum of 3 and 5 is 8
The product of 3 and 5 is 15.
The sum is less than the product.

Case 2: Sum == Product

Enter an integer: 2
Enter another integer: 2
The sum of 2 and 2 is 4
The product of 2 and 2 is 4.
The sum is equal to the product.

Case 3: Sum > Product

Enter an integer: 5
Enter another integer: 1
The sum of 5 and 1 is 6
The product of 5 and 1 is 5.
The sum is greater than the product.

📋 Summary Table

💡 Pro Tips

• Always use spaces around operators:

  # Correct
  if [ $a -gt $b ]
  
  # Incorrect
  if [$a-gt$b] ❌

• Validate input (optional enhancement):

  if ! [[ "$n1" =~ ^[0-9]+$ && "$n2" =~ ^[0-9]+$ ]]; then
      echo "Please enter valid integers."
      exit 1
  fi

  This checks if both inputs are integers before doing calculations.

• Use let for inline assignments:

  let sum=n1+n2

• Or use alternative syntax:

  (( sum = n1 + n2 ))

🤓 Why This Matters

You’re now able to:

• Handle user input
• Perform basic math in Bash scripts
• Use conditional logic based on numerical comparisons
• Write more advanced scripts that can handle dynamic user data

These skills form the foundation for writing interactive, logical, and automated command-line tools in Linux.

🧮 Exercise 3 – Using Arrays and For Loops in Bash Scripts

In this exercise, you learned how to:

• Download and inspect a CSV file
• Extract columns into arrays using cut
• Create a new array by performing arithmetic on other arrays
• Combine data back into a new CSV report using paste

This is a powerful demonstration of how Bash scripting can be used for basic data processing — especially when working with structured text files like CSVs.

✅ Step-by-Step Breakdown

🔹 3.1 Download the CSV File

You downloaded a sample dataset:

csv_file="https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-LX0117EN-SkillsNetwork/labs/M3/L2/arrays_table.csv"
wget $csv_file

This gave you a file named arrays_table.csv.

🔹 3.2 View the CSV Content

You inspected it with:

cat arrays_table.csv

Example content:

column_0,column_1,column_2
A,10,50
B,20,40
C,30,30
D,40,20
E,50,10

Each line represents a row, and each column holds different types of data.

🔹 3.3 Parse Columns into Arrays

You extracted each column using cut and stored them in Bash arrays:

column_0=($(cut -d "," -f 1 $csv_file))
column_1=($(cut -d "," -f 2 $csv_file))
column_2=($(cut -d "," -f 3 $csv_file))

💡 This uses command substitution ($(...)) to populate arrays from command output.

Then you printed one of them:

echo "${column_0[@]}"

Output:

column_0 A B C D E

Note: The first element is the header (column_0, column_1, etc.)

🔹 3.4 Create a New Array as a Difference Between Two Columns

You created a new array, column_3, which stores the difference between column_2 and column_1.

Initialize with header:

column_3=("column_3")

Count lines in the CSV:

nlines=$(cat $csv_file | wc -l)

Loop through rows and calculate:

for ((i=1; i<$nlines; i++)); do
  column_3[$i]=$((column_2[i] - column_1[i]))
done

Then you printed the result:

echo "${column_3[@]}"

Example output:

column_3 40 20 0 -20 -40

⚠️ Bash supports only integer math — so this works perfectly for numeric columns.

🔹 3.5 Combine New Column with Original CSV

You wrote the new column to a temporary file:

echo "${column_3[0]}" > column_3.txt
for ((i=1; i<nlines; i++)); do
  echo "${column_3[$i]}" >> column_3.txt
done

Then merged it with the original CSV:

paste -d "," $csv_file column_3.txt > report.csv

Now report.csv contains all four columns!

Example output:

column_0,column_1,column_2,column_3
A,10,50,40
B,20,40,20
C,30,30,0
D,40,20,-20
E,50,10,-40

📋 Summary Table

🧠 Why This Matters

This exercise introduced you to:

• Array manipulation in Bash
• Looping through indexed elements
• Performing basic arithmetic
• Working with CSV data via shell commands

These skills are useful for:

• Processing logs or flat-file databases
• Automating simple data transformations
• Building lightweight reporting tools without external dependencies

💡 Pro Tips

• Use ${array[@]} to access all elements in an array
• Remember array indexing starts at 0
• Always quote variables to avoid issues:

  echo "${column_0[@]}"

• You can validate that arrays have the same length before merging:

  [[ ${#column_0[@]} -eq ${#column_3[@]} ]] && echo "Arrays match in size"

🛠️ Real-World Uses

🎉 Conclusion – Great Job Completing the Lab!

You’ve successfully completed a hands-on lab that dives into advanced Bash scripting logic — and you’re now equipped with powerful tools to write more intelligent, dynamic, and efficient scripts.

✅ What You Learned

Here’s what you accomplished in this lab:

🧠 Why This Matters

The skills you practiced are not just academic — they’re essential for:

• Automation: Build scripts that make decisions based on input
• Data Wrangling: Parse and transform structured text files (like logs or CSVs)
• Reporting: Generate clean output files dynamically from raw data
• Scripting Efficiency: Know when to use arrays vs. direct file writing

💡 Even though Bash isn’t a full-fledged programming language like Python, it’s incredibly useful for quick automation, system management, and integration tasks.

🛠️ Real-World Applications

Here’s how these concepts apply beyond the lab:

Scheduling Jobs Using Cron

Cron is a powerful tool in Linux/Unix systems that lets you schedule jobs (commands or scripts) to run automatically at specific times.

Key Concepts:

• Cron: The system utility that runs scheduled tasks.
• Crond: The background service (daemon) that checks and runs cron jobs.
• Crontab:
  • Short for “cron table” — a file that stores your scheduled jobs.
  • Also a command (crontab) used to edit, view, or remove these schedules.

Basic Commands:

• crontab -e : Opens the editor to edit your cron jobs.
• crontab -l : Lists all current cron jobs.
• crontab -r : Removes all cron jobs (use with caution).

Cron Syntax:

Each job has a line in the crontab file with this format:

*     *     *     *     *    command_to_run
|_____|_____|_____|_____|_____|
min   hour   day   month   day of week

You can use:

• Numbers (like 0, 15, 7)
• Asterisks (*) as wildcards meaning “any”
• Commas (,) for multiple values
• Hyphens (-) for ranges

Example:

30 15 * * 0 echo $(date) >> sundays.txt

This runs at 15:30 every Sunday, appending the current date to sundays.txt.

Tips:

• Use crontab -e to add/edit jobs.
• Save and exit the editor to activate changes.
• Keep it simple and test your scripts before scheduling.

Now you can automate routine tasks like backups, data loads, and more!

🕒 Exercise 1 - Understand crontab File Syntax

In this exercise, you learned the basics of cron, a time-based job scheduler in Linux, and how to structure entries in the crontab file.

🎯 Learning Objectives Recap

After completing this exercise, you can now:

✅ Explain what Cron is
✅ Describe the purpose of a crontab file
✅ Understand and write the five time-and-date fields that define when a task runs
✅ Use proper syntax to schedule commands at specific times

🧠 What Is Cron?

• Cron is a background system process (daemon) used to execute tasks automatically at set times.
• It’s commonly used for:
  • Backups
  • Log rotation
  • System monitoring
  • Scripted maintenance tasks

📄 What Is a Crontab File?

• A crontab (short for “cron table”) is a configuration file that defines scheduled jobs for your user account.
• Each line represents one job with a specific schedule.

🔁 How to Edit It

crontab -e

👀 How to View It

crontab -l

⏰ Crontab Time Fields

Each cron job has five time fields, followed by the command to run.

Format:

minute hour day month weekday command_to_execute

All fields must be separated by spaces or tabs
You cannot use spaces inside a field — each field is a single value or expression

🧩 Special Characters in Crontab

✅ Example Entries

📝 Sample Entry Breakdown

0 21 * * * echo "Welcome to cron" >> /tmp/echo.txt

This means:

• At minute 0
• At hour 21 (9:00 PM)
• On any day of the month
• In any month
• On any day of the week
• Run: echo "Welcome to cron" >> /tmp/echo.txt

🛠️ Why This Matters

Understanding crontab syntax is key to:

• Automating repetitive tasks
• Ensuring scripts run reliably without manual input
• Managing system health and performance over time

This knowledge gives you the ability to build powerful automation workflows directly from the terminal.

✅ Summary Table

🛠️ Real-World Examples

💡 Pro Tips

• Always test your scripts manually before adding them to crontab.
• Use absolute paths for scripts in cron (e.g., /home/user/scripts/backup.sh)
• Redirect output to a log file or /dev/null to avoid email spam:

  0 2 * * * /scripts/log_system_stats.sh >> /var/log/system_stats.log 2>&1

📋 Exercise 2 – List Cron Jobs

In this exercise, you learned how to view your current cron jobs using the crontab -l command.

This is an essential step when managing scheduled tasks — whether you’re checking what’s already set up or preparing to add new automated jobs.

✅ Step-by-Step Breakdown

🔹 Open a New Terminal

You clicked:

• Terminal → New Terminal (from the menu bar)

A new terminal window opened at the bottom of the screen.

🔹 View Current User’s Crontab

You ran:

crontab -l

This command lists all scheduled jobs for the current user.

Output example:

no crontab for theia

This message simply means that no cron jobs have been defined yet — which is normal in a fresh environment.

🧠 Why This Matters

Even if there were no existing cron jobs, knowing how to list them is critical for:

• Verifying scheduled scripts or commands
• Debugging issues with automation
• Ensuring system maintenance tasks are running
• Avoiding duplicate entries when adding new jobs

📌 Summary Table: Common Crontab Commands

💡 Pro Tips

• If you’re using sudo, you may be editing a different crontab (system-wide):

  sudo crontab -l

• Always use crontab -l before making changes — it shows what’s already scheduled.

• Save output to a file for backup:

  crontab -l > my_crontab_backup.txt

🧑‍💻 Exercise 3 – Add a Job in the Crontab File

In this exercise, you learned how to schedule recurring tasks using the crontab file. You added two cron jobs:

• One that echoes a message daily at 9:00 PM
• Another that runs a custom script (diskusage.sh) at midnight every day, logging output to a file.

This is an essential skill for automating repetitive system tasks like:

• Running backups
• Monitoring disk usage
• Logging events
• Sending reports

✅ Step-by-Step Summary

🔹 3.1 Add a Daily Cron Job

You opened your crontab file with:

crontab -e

Then added this line:

0 21 * * * echo "Welcome to cron" >> /tmp/echo.txt

What it means:

This appends "Welcome to cron" to /tmp/echo.txt every night at 9 PM.

After saving with:

• Ctrl + X
• Pressed Y to confirm
• Pressed Enter to exit

You verified the job was saved by listing the crontab:

crontab -l

🔹 3.2 Schedule a Shell Script

You created a Bash script called diskusage.sh:

#!/bin/bash
# print current date/time
date
# print disk usage stats
df -h

Then made it executable and tested it:

chmod u+x diskusage.sh
./diskusage.sh

It should have printed:

• Current time (via date)
• Disk usage in human-readable format (via df -h)

🔁 Automate It with Cron

You scheduled the script to run every day at midnight:

0 0 * * * /home/project/diskusage.sh >> /home/project/diskusage.log

⚠️ Make sure the path to the script is correct — use absolute paths!

Each day at midnight, the script will:

• Print the current timestamp
• Show disk usage statistics
• Append the output to /home/project/diskusage.log

This helps track disk usage over time and monitor system behavior automatically.

📋 Summary Table

🕒 Cron Timing Quick Reference

🧠 Why This Matters

By scheduling scripts with cron:

• You automate repetitive tasks
• You ensure important logs and checks happen without manual input
• You build monitoring systems that help maintain stability and performance

These are foundational skills for DevOps, system administration, and automation workflows.

💡 Pro Tips

• Always test your script before adding to cron:

  ./diskusage.sh

• Use full (absolute) paths in cron:

  /home/project/diskusage.sh

• Redirect both stdout and stderr for debugging:

  0 0 * * * /home/project/diskusage.sh >> /home/project/diskusage.log 2>&1

• Check system cron logs if your job doesn’t run:

  sudo grep CRON /var/log/syslog

🗑️ Exercise 4 – Remove the Current Crontab

In this exercise, you learned how to remove all scheduled cron jobs for your user using the crontab -r command.

This is a powerful and irreversible action — especially on production systems — so it’s important to understand when and how to use it safely.

✅ Step-by-Step Breakdown

🔹 Remove Your Current Crontab

You ran:

crontab -r

This command removes the current user’s entire crontab file — all scheduled jobs are deleted.

⚠️ Caution: This action cannot be undone unless you have a backup!

🔍 Verify That the Crontab Was Removed

After deletion, you verified with:

crontab -l

The expected output is:

no crontab for theia

This confirms that no cron jobs are currently scheduled for this user.

📋 Summary Table

🧠 Why This Matters

Knowing how to remove or reset your crontab is useful in several scenarios:

• Cleaning up old or unused jobs
• Troubleshooting issues with scheduled tasks
• Resetting your environment during development
• Avoiding conflicts when re-scheduling scripts

Just remember:

❗ On live servers, always back up your crontab before removing it.

💡 Pro Tips

• Back up your crontab before deleting:

  crontab -l > my_crontab_backup.txt

• Restore from backup if needed:

  crontab my_crontab_backup.txt

• Remove only specific lines (instead of all) by editing:

  crontab -e

• Use sudo for system-wide cron jobs:

  sudo crontab -r -u username

🛠️ Practice Exercises – Scheduling with Cron

Here’s a clean and detailed breakdown of the practice exercise you’re working on:

🔹 1. Create a cron job that runs date >> /tmp/everymin.txt every minute

💡 Hint:

Use the crontab syntax:

*     *     *     *     *    command_to_run
|     |     |     |     |
minute hour  day  month weekday

To run something **every minute**, all five time fields should be `*`, or you can use `*` for just the minute field and set the rest to `*`.

---

### ✅ **Solution:**

#### Step 1: Open your crontab file in edit mode:
```bash
crontab -e

Step 2: Add the following line at the end of the file:

* * * * * date >> /tmp/everymin.txt

This means:

• Run the date command every minute
• Append (>>) the output to /tmp/everymin.txt

Step 3: Save and exit

• Press Ctrl + X (if using Nano)
• Type y to confirm changes
• Press Enter to save to the same file

🧪 What This Does:

Every minute, the current system date and time will be appended to /tmp/everymin.txt.

Example entry in /tmp/everymin.txt:

Sat Apr 5 10:00:00 UTC 2025

After one minute:

Sat Apr 5 10:01:00 UTC 2025

📋 Summary Table

🧠 Why This Matters

This simple example demonstrates how you can use cron to:

• Log data over time
• Monitor system events
• Automate repetitive tasks like backups, health checks, and more

You can easily expand this idea to log other useful information:

* * * * * df -h >> /tmp/everymin.txt
* * * * * free -h >> /tmp/everymin.txt

🎉 Summary – Mastering Cron Jobs in Linux

Great job completing this hands-on lab! You’ve now learned how to manage scheduled tasks using cron, one of the most powerful tools for automation in Linux.

🔧 What You Learned

Here’s a quick recap of what you practiced in this lab:

🕒 Crontab Syntax Reminder

Each line in the crontab file follows this structure:

minute hour day month weekday command_to_execute

Example:

* * * * * echo "Hello" >> /tmp/output.txt

You used this format to schedule recurring tasks like logging system time and disk usage.

📌 Best Practices & Tips

• Always test scripts manually before scheduling them with cron.
• Use absolute paths in cron — it doesn’t know about your current directory!
• Redirect output for logging or debugging:

  0 0 * * * /home/project/script.sh >> /home/project/output.log 2>&1

• Back up your crontab before editing or removing:

  crontab -l > my_crontab_backup.txt

💡 Real-World Automation Ideas

Now that you understand cron, try automating:

• Daily backups
• System health checks (CPU, memory, disk)
• Log rotation
• Sending automated reports or emails
• Updating repositories or checking software versions

📄 Module 3 Cheat Sheet – Introduction to Shell Scripting

This cheat sheet summarizes all the essential Bash scripting and task automation concepts covered in this module. Keep it handy for quick reference while writing scripts or scheduling jobs.

🐚 Basic Bash Script Setup

🔗 Pipes & Filters

| Task | Command | | ----------------------------------------------- | ------------------------- | --------- | ----- | | Pipe output from one command to another | command1 | command2 | | View first 20 lines of man ls | man ls | head -20 | | Extract column 1 from CSV and remove duplicates | cut -d "," -f1 names.csv | sort | uniq |

💬 Shell and Environment Variables

⌨️ Metacharacters & Special Symbols

📥 Input/Output Redirection

| Task | Command | | ------------------------------- | -------------------------------- | ------------------- | | Redirect output, overwrite file | echo 'Hello' > hello.txt | | Append output to file | echo 'New line' >> hello.txt | | Redirect standard error | bad_command 2> error.log | | Append standard error | bad_command 2>> error.log | | Send file content as input | tr "[a-z]" "[A-Z]" < input.txt | | Same using pipe | cat input.txt | tr "[a-z]" "[A-Z]" |

💡 Command Substitution

📎 Command Line Arguments

Example:

if [[ $# == 2 ]]; then
  echo "Exactly 2 arguments provided"
fi

⏱️ Scheduling Jobs with Cron

❓ Conditionals in Bash

Basic if syntax:

if [[ $# == 2 ]]; then
  echo "Two arguments given"
else
  echo "Not two arguments"
fi

Logical Operators:

• AND: &&

  if [ condition1 ] && [ condition2 ]

• OR: ||

  if [ condition1 ] || [ condition2 ]

➕ Arithmetic Operations

Use inside scripts or one-liners to perform basic math in Bash.

🧮 Arrays in Bash

🔁 For Loops in Bash

Loop over range:

for i in {0..5}; do
    echo "Iteration $i"
done

Loop through array items:

for item in ${my_array[@]}; do
    echo $item
done

Loop with indexing:

for ((i=0; i<${#my_array[@]}; i++)); do
    echo ${my_array[$i]}
done

🛠️ Pro Tips

• Always quote your variables:

  echo "$HOME"

• Use $(...) instead of backticks for better readability
• Prefer [[ ... ]] over [ ... ] for safer comparisons
• Use crontab -l > backup.txt before editing or removing scheduled jobs
• Combine pipes and filters to process data efficiently

🎉 Summary & Highlights – Shell Scripting Essentials

Great job completing this module! You’ve now built a strong foundation in Bash shell scripting, command chaining, and task automation.

Here’s a quick recap of what you’ve learned:

🧾 Key Concepts Covered

| Concept | Description | | -------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------ | ----------------- | | Shell Scripts | Programs that start with #!/bin/bash and are interpreted, not compiled | | Shebang Line | #!/bin/bash — tells the system which interpreter to use | | Filters + Pipes | Commands like sort, cut, uniq can be chained using | for powerful data processing | | Shell Variables | Temporary storage inside a script: my_var=value | | Environment Variables | Available to child processes; exported using export | | Metacharacters | Special characters like *, ?, #, and ; used by the shell for logic | | Quoting | Controls interpretation of metacharacters: ' ' = literal, " " = expand variables | | I/O Redirection | Send input/output to or from files using >, >>, <, 2>, etc. | | Command Substitution | Use $(command) to insert output into another command or variable | | Command-line Arguments | Pass values to your script using $1, $2, …, $@, $# | | Sequential vs Concurrent Execution | Run commands one after another or in parallel using & | | Cron Jobs | Schedule scripts to run automatically at set times using crontab -e | | Listing Cron Jobs | crontab -l shows all scheduled jobs | | Conditional Logic | Use if, elif, else, &&, | | to control flow | | Loops and Arrays | Use for loops and Bash arrays to process lists of items |

🔍 Core Syntax Reference

📌 Crontab Format

m  h  dom  mon  dow  command

• m = minute (0–59)
• h = hour (0–23)
• dom = day of month (1–31)
• mon = month (1–12)
• dow = day of week (0–6) where 0 = Sunday

Example:

0 2 * * * /home/user/scripts/backup.sh

🧮 Arithmetic in Bash

echo $((3 + 2))        # Outputs 5
echo $((-1 * -2))      # Outputs 2

🧪 Conditional Statements

if [[ $# == 2 ]]; then
  echo "Two arguments provided"
fi

🔁 For Loops

for i in {1..5}; do
  echo "Iteration $i"
done

📦 Arrays

my_array=(apple banana "orange juice" 42)
echo ${my_array[0]}     # apple
echo ${my_array[@]}     # all elements

💡 Why This Matters

You’re now equipped to:

• Write custom automation scripts
• Process and analyze log files, CSVs, and system data
• Schedule daily tasks like backups and reports
• Make decisions in scripts using conditionals and logic
• Build dynamic and reusable scripts with functions (coming soon!)

These skills are essential for:

• System administration
• DevOps workflows
• Data engineering pipelines
• Linux development and scripting