IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 23

Today's Topics

Quiz 8

The answers to Quiz 8 are posted here.

Let's take a look.

Homework 11

I have posted homework 11 here.

It is not due this Sunday.

Instead it is due Sunday, December 3rd, at 11:59 PM.

Announcement

There will be no graded quiz this week.

But there will be one the following week covering the material in the previous 3 Class Quizzes.

Tips and Examples

A Real World Script Example

Unix has daemons ...
which are processes that run in the background ...
and provide various services
The reason we can log into users3 ...
is that there is an ssh daemon, sshd ...
that listens to port 22 on users3 ...
and creates a login session when you connect to that port
Unix and Linux machines have a daemon, named cron ...
that will run commands or scripts at specific times
You create something called a cron job ...
and tell cron when to run it
When that time rolls around ...
cron will run the job
Usually, the job is a shell script
I have a client who uses a database application I created
The tables in the database are backed up every night ...
by a cron job that runs a special shell script

Let's look at this script

    $ cat -n backup_tables.sh 
         1  #!/bin/bash
         2  #
         3  # backs up all tables in the version2 database
         4  
         5  cd $v2/sql/bak
         6  rm *.bak 2> /dev/null
         7  dir=$(date +%Y-%m-%d)
         8  rm -rf $dir
         9  mysql version2 -u root -p$1 < $v2s/scripts/backup.sql
        10  mkdir $dir
        11  cp *.bak $dir
        12  date=$(date +%Y-%m-%d)
        13  echo $date ": Table backup complete"

Line 5 of the script uses the variable v2 ...
to go to the backup directory
v2 holds the absolute pathname ...
of the directory containing code for this client
I need this variable because the pathname of this directory on my machine ...
is different from the pathname of the same directory
on the machine running the application ...
in the clients office
I define this variable in ~/.bash_profile on each machine ...
to hold the correct absolute pathname for that machine
In line 6, the script removes backup files ...
all of which have a .bak extension
```
    rm *.bak 2> /dev/null
```
Copies of these files exist elsewhere ...
so this operation just removes one copy of the files
Notice that I redirect standard error to /dev/null
If I don't do this, and there are no files to delete ...
I'll get an error message that can be confusing
Line 7 sets the value of the dir variable ...
to today's date expressed in yyyy-mm-dd format
```
    dir=$(date +%Y-%m-%d)
```
This means I am using command substitution
The shell runs the date command in a subshell ...
and replaces the command on the command line ...
with the results of running the command
Line 8 removes any directory already created ...
to hold backup files for today's date
```
    rm -rf $dir
```
The script can be run more than once in a given day
So I want only the most recent backup files ...
in the directory containing today's backup

Line 9 runs a mysql script that actually creates the backup

    mysql version2 -u root -p$1 < $v2s/scripts/backup.sql

The backup files, with the .bak extension ...
are created in the backup directory
Another mysql script can restore the tables from these files
Line 10 creates a new directory whose name is today's date
```
    mkdir $dir
```
This gives me an historical archive
Line 11 copies the newly created backup files into this directory
```
    cp *.bak $dir
```
Line 12 creates the date variable ...
that also contains today's date
```
    date=$(date +%Y-%m-%d)
```
Again I use command substitution
I use this variable in line 13 to print a message ...

which will go to a log file

    echo $date ": Table backup complete"

In the command line of the cron job ...
I redirect this output to a log file
Why didn't I use the dir variable defined above?
At some time in the future ...
I might want a different date string ...
or a different name for the backup directory ...
so this gives me flexibility
I really should change the last line so it prints the message ...
only if everything goes well
To do this, I would need to use a conditional statement ...
and the special parameter $?

Review

Shell Script Control Structures

Control structures are Unix statements that change the order of execution ...
of commands within a program or script
There are two basic types of control structures
- Loops
- Conditionals

The `if ... then ...` Construct

The first conditional is the if ... then statement ...

which has the format

    if COMMAND
    then
        COMMAND_1
        COMMAND_2
        ...
    fi

COMMAND is any Unix command
COMMAND_1, COMMAND_2, ... are a series of Unix commands
The most common comman used with if is test ...
which must be followed by arguments that form a logical expression
It is used to test the truth or falsity of a condition
The keyword then must either be on a separate line from if ...
or on the same line, but separated by a semi-colon
The keyword fi must close the conditional statement
The statements between then and fi are executed ...
depending on the status code ...
given by the command that follows if
If the command following if runs without error ...
then it will return an exit status of 0 ...
which the if ... then statement treats as true

`test`

The test command evaluates a logical expression ...
given to it as an argument ...
and returns a status code of 0 ...
if the expression evaluates to true
It returns a status code of 1 ...
if the expression evaluates to false
In an if statement, a status code of 0 means true ...
and a status code greater than 0 means means false

The `test` Operators

test has a number of operators
The operators test for different conditions
When used with two arguments, the operators are placed between

Some operators work only on numbers

Operator Condition Tested

-eq Two numbers are equal

-ne Two numbers are not equal

-ge The first number is greater than, or equal to, the second

-gt The first number is greater than the second

-le The first number is less than, or equal to, the second

-lt The first number is less than the second

Operator	Condition Tested
-eq	Two numbers are equal
-ne	Two numbers are not equal
-ge	The first number is greater than, or equal to, the second
-gt	The first number is greater than the second
-le	The first number is less than, or equal to, the second
-lt	The first number is less than the second

test uses the different operators when comparing strings

Operator Condition Tested

= When placed between strings, are the two strings the same

!= When placed between strings, are the two strings not the same

Note that test uses symbols (=) when comparing strings
But letters preceded by a dash (-eq) when comparing numbers

Operator	Condition Tested
=	When placed between strings, are the two strings the same
!=	When placed between strings, are the two strings not the same

There are a couple of operators that apply only to a single string

Operator Condition Tested

-z Whether the string given as an argument has a length of 0

In these cases, the operator comes before the string

Operator	Condition Tested
-z	Whether the string given as an argument has a length of 0

Other operators apply to files and directories

Operator Condition Tested

-d Whether the argument is a directory

-e Whether the argument exits as a file or directory

-f Whether the argument is an ordinary file (not a directory)

-r Whether the argument exists and is readable

-s Whether the argument exists and has a size greater than 0

-w Whether the argument exists and is writable

-x Whether the argument exists and is executable

There are two additional operators ...
that test uses when evaluating two expressions

Operator	Condition Tested
-d	Whether the argument is a directory
-e	Whether the argument exits as a file or directory
-f	Whether the argument is an ordinary file (not a directory)
-r	Whether the argument exists and is readable
-s	Whether the argument exists and has a size greater than 0
-w	Whether the argument exists and is writable
-x	Whether the argument exists and is executable

It is placed between the two expressions

Operator Condition Tested

-a Logical AND meaning both expressions must be true

-o Logical OR meaning either of the two expressions must be true

The exclamation mark, ! is a negation operator
It changes the value of the following logical expression
- It changes a false expression to true
- And a true expression to false

Operator	Condition Tested
-a	Logical AND meaning both expressions must be true
-o	Logical OR meaning either of the two expressions must be true

New Material

Using `test` in Scripts

We can use test in an if statement

    $ if test foo = foo
    > then
    > echo "The two strings are equal"
    > fi
    The two strings are equal

But this looks very different from an if statement ...
in programming languages
To make the if statement ...
look more like a "real" programming language ...
Bash provides a synonym for test ...
a pair of square brackets, [ ]
To test whether the value of number1 ...
is greater than the value of number2

you could write either

    if test $number1 -gt $number2

    if [ $number1 -gt $number2 ]

Whenever you use [ ] ...
instead if test ...
there must be a space ...
before and after each square bracket

If you don't you will get an error message

    $ [ 5 -ne 6]
    -bash: [: missing `]'

That's because Bash reads 6] as a single token ...
which it does not understand
Putting a space between 6 and ] makes it two tokens
I find myself making this mistake all the time
The first thing to do when you get an error in a script using [ ] ...
is make sure you have spaces surrounding all your square brackets
test does not return a value to standard output
test returns true or false ...
through the exit status
If test returns an exit status of 0 ...
it means the condition was true
If test returns an exit status of 1 ...

it means the condition was false

    $ [ 5 -eq 4 ]; echo $?
    1
    $ [ 5 -ne 4 ]; echo $?
    0

Checking the Arguments to a Script

If a script must have a certain number of arguments ...
it should check to see that it has been given them ...
on the command linse
If it doesn't get the right number of arguments ..
a script should print a usage message and exit

A usage message has a standard form

    Usage:  PROGRAM_NAME  ARG1  ARG2  ...

In a usage message, the strings that follow the program name ...
should be a word or words that indicates what kind information was required
So if you had a script test_dr.sh ...
that needed the name of a directory as an argument ...
it's usage message would be
```
    Usage:  test_dr.sh  DIR_NAME
```

Let's look at an example

    $ cat examples_it244/usage_1.sh
    #! /bin/bash
    # this program demonstrates checking for arguments 
    # and printing a usage message when
    # the expected arguments are not supplied

    if test $# -eq 0
    then
        echo Usage:  $0  STRING
        exit 1
    fi
    echo Received argument $1
    
    $ examples_it244/usage_1.sh 
    Usage: examples_it244/usage_1.sh  STRING
    
    $ examples_it244/usage_1.sh foo
    Received argument foo

The script first looks at the number of arguments it gets ...
which is contained in #
If it receives zero arguments ...
the script prints a usage message ...
and then quits with an exit status of 1
Otherwise it prints the argument it was given
The usage message uses the 0 positional parameter ...
which contains the pathname that ran the script
The pathname that appears in this usage message is correct ...
but it is also confusing
What we really want in a usage message ...
is the filename part of the pathname
We can strip out everything from the pathname ...
except the name of the file ..
if we use Unix utility called basename
basename takes a pathname as an argument ...
and strips out everything ...

except for the filename

    $ basename examples_it244/usage_1.sh
    usage_1.sh

So a better version of this script would be

    $ cat examples_it244/usage_2.sh
    #! /bin/bash
    # this program demonstrates checking for arguments 
    # and printing a usage message using basename

    if test $# -eq 0
    then
    echo Usage: $(basename $0) STRING
        exit 1
    fi
    echo Received argument $1
    
    $ examples_it244/usage_2.sh
    Usage: usage_2.sh   STRING
    
    $ examples_it244/usage_2.sh foo
    Received argument foo

Here I used basename and command substitution ...
to get the name of the file ...
without the path
You don't need a usage message ...
if the script does not require arguments

The `if ... then ... else ...` Construct

The another Unix conditional is the if ... then ... else ... statement

which has the following format

    if COMMAND
    then
        COMMAND_1
        COMMAND_2
        ....
    else
        COMMAND_A
        COMMAND_B
        ...
    fi

If COMMAND returns an exit status of 0
COMMAND_1, COMMAND_2, ... will be executed
Otherwise COMMAND_A, COMMAND_B, ..., will be run

Let's look at an example

    $ cat cat_file.sh 
    #! /bin/bash
    #
    # demonstrates the use of the if ... then ... else ... construct

    if [ $# -eq 0 ]
    then
        echo Usage: $(basename $0) filename 
        exit 1
    fi
    
    if [ -f $1 ]
    then
        cat $1
    else
        echo $1 is not a file
    fi  
    
    $ ./cat_file.sh
    Usage: cat_file.sh filename
    
    $ ./cat_file.sh lines.txt 
    line 1
    line 2
    line 3
    line 4
    line 5
    
    $ ./cat_file.sh foo
    foo is not a file

The second if statement ...
does one thing if the first argument is a file ...
and another thing if it is not
The test command
```
    [ -f $1 ]
```
returns a status code of 0 ...
if the first argument is the name of a file
Otherwise it returns 1

The `if ... then ... elif ...` Construct

The if ... then ... elif ... construct ...

lets you create nested conditionals

    if COMMAND
    then
        COMMAND_1
        COMMAND_2
        ...
    elif OTHER_COMMAND
    then
        COMMAND_A
        COMMAND_B
        ...
    ...
    else
        COMMAND_N1
        COMMAND_N2
        ...
    fi

elif stands for "else if"
Notice that elif must be followed by then
The then must either be on the next line ...
or the same line separated by a semi-colon, ;
While each else statement must be ended by a fi ...
elif only requires a single fi at the end

Let's look at an example

    $ cat if_4.sh 
    #! /bin/bash
    #
    # demonstrates the if ... then ... elif ...  construction

    echo -n "word 1: "
    read word1
    echo -n "word 2: "
    read word2
    echo -n "word 3: "
    read word3
    if [ $word1 = $word2 -a $word2 = $word3 ]
    then
        echo "Match: words 1, 2 & 3"
    elif [ $word1 = $word2 ]
    then
        echo "Match: words 1 & 2"
    elif [ $word1 = $word3 ]
    then
        echo  "Match: words 1 & 3"
    elif [ $word2 = $word3 ]
    then
        echo  "Match: words 2 & 3"
    else
        echo No match
    fi
    
    $ ./if_4.sh 
    word 1: foo
    word 2: bar
    word 3: bletch
    No match
    
    $ ./if_4.sh 
    word 1: foo
    word 2: foo
    word 3: boo
    Match: words 1 & 2

Debugging Scripts

It is easy to make a mistake when writing a script
So it is a good idea to practice incremental development ...
when writing a script
This means writing a few lines of the script ...
testing it and correcting errors ...
and then writing a few more lines
To help you find bugs in your Bash script ...
you can run bash with the -x option
The -x option causes bash to print each command ...

before it executes the commands on that line

    $ cat match_three.sh 
    #! /bin/bash
    #
    # takes three stings as input and compares them

    if [ $# -lt 3 ]
    then
        echo Usage: $(basename $0)  STRING1  STRING2  STRING3
        exit 1
    fi

    if [ $1 = $2 -a $2 = $3 ]
    then
        echo All arguments match
    elif [ $1 = $2 ]
    then
        echo Arguments 1 and 2 match
    elif [ $1 = $3 ]
    then
        echo  Arguments 1 and 3 match
    elif [ $2 = $3 ]
    then
        echo  Arguments 2 and 3 match
    else
        echo No arguments match
    fi
    
    $ bash -x match_three.sh foo bar foo
    + '[' 3 -lt 3 ']'
    + '[' foo = bar -a bar = foo ']'
    + '[' foo = bar ']'
    + '[' foo = foo ']'
    + echo Arguments 1 and 3 match
    Arguments 1 and 3 match

Before bash prints a line from the script ...
it prints a plus sign, + ...
to let you know that the line is not output from the script
Notice that the script keeps running test commands ...
represented by [ ] ...
until it finds one condition that evaluates to true
Using bash with the -x option ...
you can trace the path bash takes through your script
This can help you find errors

Indenting When Writing Scripts

Control structures work by marking off certain parts of the script ...
that are executed differently from the rest of the script
Most commands in a script are executed only once ...
in the order they appear in the script
In if constructions certain blocks of commands ...
are only executed under certain conditions
In loop constructs a block of commands ...
is run more than once
Special Unix keywords set off these blocks of commands
Keywords like then, else, elif and fi
When writing scripts with control structures it is a good idea ...
to indent all lines in a block of commands ...
so it is clear that they are treated differently ...
from the rest of the script
Let's look at the arg_test.sh script ...

you wrote for Class Exercise 22

    #! /bin/bash
    #
    # responds with the number of the arguments given
    # to this script

    if test $# -eq 0 
    then
        echo "You entered no arguments"
    fi
    if test $# -eq 1 
    then
        echo "You entered 1 argument"
    fi
    if test $# -eq 2 
    then
        echo "You entered 2 arguments"
    fi
    if test $# -gt 2
    then
        echo "You entered more than 2 arguments"
    fi

Each block of commands contained in each if ... then statement ...
is clearly set off by the indent

If we did not indent, we would get this

    #! /bin/bash
    #
    # responds with the number of the arguments given
    # to this script

    if test $# -eq 0 
    then
    echo "You entered no arguments"
    fi
    if test $# -eq 1 
    then
    echo "You entered 1 argument"
    fi
    if test $# -eq 2 
    then
    echo "You entered 2 arguments"
    fi
    if test $# -gt 2
    then
    echo "You entered more than 2 arguments"
    fi

This is much harder to read than the indented version
Indenting blocks of commands in a control structure ...
is a good habit to get into

Today's Topics

Tips and Examples

Review

New Material

Quiz 8

Homework 11

Announcement

Tips and Examples

A Real World Script Example

Review

Shell Script Control Structures

The if ... then ... Construct

test

The test Operators

New Material

Using test in Scripts

Checking the Arguments to a Script

The if ... then ... else ... Construct

The if ... then ... elif ... Construct

Debugging Scripts

Indenting When Writing Scripts

Attendance

Class Quiz

The `if ... then ...` Construct

`test`

The `test` Operators

Using `test` in Scripts

The `if ... then ... else ...` Construct

The `if ... then ... elif ...` Construct