IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 5

Today's Topics

Reading Assignment

The reading assignment for this week is chapter 3 of Sobell, The Utilities.

You will find a list of all reading assignments here.

There is a link to this page on the class web page.

Homework 3

I have posted Homework 3 here.

As usual, it will be due next Sunday at 11:59 PM.

The is the first homework assignment that will require you to write a script.

Your script must meet certain requirements that I mentioned in the last class.

Let's review the requirements for homework scripts.

Tips and Examples

Do Not Fall Behind in This Course

Technical courses require that you master a great deal of detail
Unless you are an exceptional student ...
you will need to spend some time on this course ...
several times during the week
Working once a week is usually not enough
The key to success in a technical course ...
is steady, consistant, work ...
spread out over several days
Some important concepts do not sink in immediately
It may take several days for something new to sink in
Since material in this course often depends on what came before ...
it is critical that you not fall behind
in your work for this course
If you fall behind, it can be VERY hard to catch up

Read the Homework Assignments Carefully

The instructions in the homework assignments and Class Exercises are sometimes complex ...
and involve multiple steps
When this happens, you may need to read the assignment more than once ...
to make sense of what I want you to do
Take the time to read these documents slowly ...
so you understand all the details
If you don't you will make mistakes ...
and lose points
Technical work is detailed work ...
so you must learn to read documents with a lot of detail
If you do not understand what I want in an assignment ...
post a question on the class discussion area
NEVER GUESS at the meaning of something I have written

Working with More than One Window

You can have more that one ssh window open ...
at any one time
I normally run at least 4 ssh sessions on my Mac
After you have opened one PuTTY window ...
you can open you can open another ...
and another
Each one of these windows is a separate login session
If you close one of these windows ...
it does not affect the others

Shell Scripts

Any of the commands you type at the command line ...
can be placed in a file
That file can then be run ...
and all the Unix commands in the file
Such a file is called a shell script
Let's say I wanted to create a script to go to my home directory ...
and then go to my tmp directory ...
and then list the contents of the directory
I can do this with the following shell script
```
cd		
cd tmp	
ls
```
If the name of the file is tmp.sh ...
I can run it like this
```
bash tmp.sh
```

Course Requirements

Shell Scripts for Homework Assignments

Starting with homework 3 ...
all homework assignments will require you to write shell scripts
This makes it easier for me to grade your homework
These script files must meet certain requirements ...
or you will lose points
I've created a document which lists the requirements ...
which you will find here
Let's look at them

Scripts for Class Exercises

Just as all homework assignments from now on ...
will require you to write shell scripts ...
so will all Class Exercises
This will help you get used to writing scripts
You do not have to follow all the requirements for homework scripts ...
when you write scripts for the Class Exercises
But there are a few thing you should not do ...
in shell scripts for class exercises
Do not do any of the following in a shell script
- Run man or info
- Run more or less
- Run nano, vi, vim, emacs or any text editors
All these commands ask the user for input
I am going to run these scripts to test them ...
and I will do this using other scripts that I will write
I want these scripts to run automatically ...
with little input from me
If you use any of the commands written above ...
the scripts will not run the way they should ...
and you will get no credit for the assignment ...
until you remove these commands

Review

Special Characters

Some characters have special meaning in Unix

They are

& : | * ? ' " [ ] ( ) $ < > { } # / \ ! ~

Whitespace characters are special too
They are
- Space
- Tab
- Newline
Space separates commands, options, and arguments on the command line
Tab has a special meaning we will discuss in another class
The Enter key creates a newline character on a PC ...
while on a Mac it is the Return key
When the shell sees a newline ...
it executes the commands on the command line

Quoting and Escaping

You can turn off the special meaning of a character ...
by preceding it with a backslash, \
You can also turn off special meanings by enclosing a string in quotes
You can use either single quotes, '
or double quotes, "

`cd` - Change Directory

To change your directory use cd
If you run cd without an argument ...
it will take you to your home directory
To go to the directory above your current directory ...
use ..
```
cd ..
```

`pwd` - Show Your Current Directory

pwd will print you your current location in the filesystem
pwd takes no arguments

`ls` - List the Contents of a Directory

ls lists the contents of a directory
To see the contents of directory work, run
```
ls work
```
When you use ls without an argument ...
it lists the contents of the current directory
For more information run ls with the -l (long) option
ls -a shows the "invisible" files ...
whose name begins with a .

`cat` - Print the Contents of a File

To show the contents of a file, use cat
Use cat -n to print a number for each line of the file

`rm` - Delete a File

To remove a file, use rm
To remove all files in a directory, use rm *
Be very careful when you use this construction
There is no file recovery mechanism in Unix
rm will not remove a directory unless you use special options

`mkdir` - Create a Directory

You create a directory using mkdir

`rmdir` - Delete a Directory

rmdir is used to remove a directory
rmdir will not work unless the directory is empty

`cp` - Copy Files

cp copies files or directories
To copy a directory and all its files and sub-directories ...
use cp with the -r option

`mv` - Move a File or Directory

Use the mv command to move a file or directory ...
from one place to another
To rename a file or directory, you also use mv

New Material

`echo` - Print Text to the Terminal

echo simply prints whatever comes after it to the screen
```
$ echo Hello world!
Hello world!
```
It's used a lot in shell scripts to prompt for input ...
or to let the user know what is happening

It is also useful for checking the value of a variable

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:::

When using echo with a variable ...
you must precede the variable name with a dollar sign, $
When echo is used with the -n option ...
it does not advance to the next line
This is useful in writing prompts inside shell scripts
To see this option at work let's compare two scripts

The first uses echo with no option

$ cat echo_no_option.sh 
#! /bin/bash
# demonstrates the use of echo without the -n option

echo Hello World!
echo About to leave

When I run it, I get

$ ./echo_no_option.sh 
Hello World!
About to leave

The second script has echo use the -n option

$ cat echo_with_n_option.sh 
#! /bin/bash
# demonstrates the use of echo with the -n option

echo -n Hello World!
echo About to leave

When I run it

$ ./echo_with_n_option.sh 
Hello World!About to leave

Notice that when I use the -n option in the second script ...
both messages are printed on the same line

`hostname` - Print the Name of Your Host Machine

Every machine on the network has a name
This name is known as the hostname
The hostname command prints the network name ...
of the machine you are using
```
$ hostname
itserver6
```
users3 is a virtual machine that calls itself itserver6
When used with the -i option ...
hostname will print the IP address of the host machine
```
$ hostname -i
192.168.107.136
```

Pagers - View a File One Screen at a Time

Pagers show the contents of a file ...
one screenful at a time
Unix provides two paging programs, more and less
less is an improved version of more
The name is an example of dry Unix humor
It's not more ...
it's less
Both allow you to advance to the next screen ...
by hitting the space bar
To move down one line hit Enter
To get help type h
less has more features
In less you can use the arrow keys to move up and down one line
When more gets to the end of a file ...
hitting the Space bar will bring you back to the command line
When you get to the end of a file in less ...
you must type q to quit

Pathname Completion

When typing a long file name it is easy to make a mistake
Unix helps with a feature called pathname completion
We'll talk more about pathnames in a future class
If you need to type the name of a file or directory ...
you can type in the first few characters ...
and then hit the Tab key ...
and Unix will try to fill in the rest
If nothing matches ...
you will hear a beep
You will also hear a beep if there is more than one match
In this case when you hit the Tab key again ...
you will get a list of possible matches

`grep` - Finding Strings inside Files

grep (get regular expression) is a utility which searches a text file ...
for lines containing a specific string ...
and prints the lines where it finds the string
A string is a collection of characters ...
that may, or may not, be a word
grep uses the following format
```
grep  [-OPTIONS]  STRING  FILE ...
```
The words in capital letters are things that you must supply
Anything enclosed in square brackets is optional
Here is an example of how grep can be used

Say we have a text file

$ cat  red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-15      Red Sox @  Rays         Loss 6-9
2011-07-16      Red Sox @  Rays         Win 9-5
2011-07-17      Red Sox @  Rays         Win 1-0
2011-07-18      Red Sox @  Orioles      Win 15-10
2011-07-19      Red Sox @  Orioles      Loss 2-6
2011-07-20      Red Sox @  Orioles      Win 4-0
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-25      Red Sox vs Royals       Loss 1-3
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

and we want to find all the games the Sox won

We can use grep

$ grep Win red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-16      Red Sox @  Rays         Win 9-5
2011-07-17      Red Sox @  Rays         Win 1-0
2011-07-18      Red Sox @  Orioles      Win 15-10
2011-07-20      Red Sox @  Orioles      Win 4-0
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

grep is case sensitive
so if we ran
```
$ grep win red_sox.txt
```
we would get no results
If you want grep to ignore case ,,,

you must run it with the -i option

$ grep -i win red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-16      Red Sox @  Rays         Win 9-5
2011-07-17      Red Sox @  Rays         Win 1-0
2011-07-18      Red Sox @  Orioles      Win 15-10
2011-07-20      Red Sox @  Orioles      Win 4-0
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

The first command failed ...
because I spelled "win" with a lowercase w
If your search string contains one of Unix's special characters
You must use a backslash to escape it ...
or use quotes
grep -r will search recursively through a directory ...
looking at all files in the directory ...
and all the subdirectories

grep -v returns all lines that do not match the search string

$ grep -v Win red_sox.txt 
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-15      Red Sox @  Rays         Loss 6-9
2011-07-19      Red Sox @  Orioles      Loss 2-6
2011-07-25      Red Sox vs Royals       Loss 1-3
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3

There are many more useful options for grep ...
which can be found on the man page
I use grep on an almost daily basis
For example, let's say I want to add a field to a database table
I keep all my SQL code in text files
I can use grep to find every file that references that table

`head` - View the Top of a File

Sometimes the first few lines of a file are all you need to see

head displays the first 10 lines of a file

$ head red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-15      Red Sox @  Rays         Loss 6-9

If you give head a number as an option ...

it will display that number of lines

$ head -5 red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4

`tail` - View the Bottom of a File

tail is like head except it prints the last 10 lines of a file

$ tail red_sox.txt 
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-25      Red Sox vs Royals       Loss 1-3
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

You can give tail a number as an option ...

to specify the number of lines printed

$ tail -4 red_sox.txt 
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

tail is especially useful when looking at log files
Log files are text files that record significant events
They are created automatically by programs that provide services ...
like a web server
The most recent entries are at the end of file ...
and those are usually what you want to see ...
when you are trying to solve a problem

`sort` - Print a File in Sorted Order

Unix has a number of utilities for manipulating text files
sort prints the contents of a file ...

with the lines sorted

$ cat  fruit.txt 
grapes
pears
oranges
cranberries
apples
melons
blueberries
	
$ sort  fruit.txt 
apples
blueberries
cranberries
grapes
melons
oranges
pears

sort looks at the beginning of each line of a file ...
and sorts the line based on these characters
sort does not change the file itself ...
it simply prints the sorted contents of the file to the screen
sort many useful options ...
which you can find in the man pages

sort -r (reverse) will sort the file in reverse alphabetical order

$ sort -r fruit.txt 
pears
oranges
melons
grapes
cranberries
blueberries
apples

sort -n (number) will sort a file by number ...

so a line starting with "2" will appear before a line starting with "11"

$ cat numbers.txt
11
1
17
2
3
15
4
5
14
6
13
7
8
9
10
12
16
18
19
20
	
$ sort numbers.txt
1
10
11
12
13
14
15
16
17
18
19
2
20
3
4
5
6
7
8
9
	
$ sort -n numbers.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

sort does not change the file on which it is run ...
it merely prints the contents of the file in sorted order

Class Exercise 5

Class Exercise 5 is the first exercise where you will have to write a shell script.

You will create a text file using nano, and type the Linux commands into it.

You must create this file in your ex5 directory.

Please note that the first few sections of Class Exercise 5 should not be included in your ex5.sh script.

Test your script to see that it works.

When your script works, come see me so I can test it with my test script.