IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 5

Microphone

Graded Quiz

You can connect to Gradescope to take weekly graded quiz today during the last 15 minutes of the class.

Once you start the quiz you have 15 minutes to finish it.

You can only take this quiz today.

There is not makeup for the weekly quiz because Gradescope does not permit it.

Readings

If you have the textbook you should read chapter 3, The Utilities.

Homework 3

I have posted homework 3.

As usual, it will be due this coming Sunday at 11:59 PM.

Late Penalty

You will lose 2 points for every day that your assignment is late.

Every Unix file has a time stamp that changes every time the file is modified

I look at this time stamp to determine whether a submission is late.

If your assignment is not working by the due date, you can continue to work on it but you will lose a few points.

If your assignment is working, do not go back and change it, because that will change the time stamp and result in a late penalty.

Do Not Email Me about Missing Assignments

If you get an email from me saying an assignment is missing do not email me about this.

I get far too many emails.

Instead of sending me an email, fix the problem.

If you do not know how to fix it, post a question on Piazza or contact the Class Assistant.

I collect homework assignments and and check Class Exercises several times during the week.

I will check or collect your assignment later in the week.

Questions

Are there any questions before I begin?

Course Work

Importance of Exams

I know that many of you find exams difficult
But the best way for me to know if you have really learned the material is by giving exams
It is too easy to cheat on Class Exercises and homework assignments
This is why the two exams account for 50% of your grade
Too many students do well on the graded quizzes, Class Exercises and homework assignment ...
only to do poorly on the exams
This lowers their grade considerably
The purpose of the Class Exercises and homework assignments is to give you hands on experience with the material ...
and to better prepare you for the exams

Preparing for Exams

Both the midterm and final exams follow the same format
60% of the points on the exams come from questions taken from the ungraded class quizzes
The remaining 40% of the points come from other questions created specially for the exam
These questions ask you to write short segments of Python code
Two things will help you prepare for exams
- Flash cards
- Class Exercises and homework assignments
Flash cards should be made for all questions from Ungraded Class Quizzes
You should review them frequently throughout the semester
Pay attention when you are working on the Class Exercises and homeworks
Work on these assignments is teaching much about the Linux command line
This will come in handy for answering the questions that do not come from the Ungraded Class Quizzes

Course Requirements

Shell Scripts for Homework Assignments

Starting with homework 3, you will need to write shell scripts for all homework assignments
This makes it easier for me to grade your homework
These script files must meet certain requirements or you will lose points
I've created a document which lists the requirements which you will find here
Let's look at them

Scripts for Class Exercises

Starting today, all Class Exercises will ask you to also write scripts
You do not have to follow all the requirements for homework scripts for Class Exercise scripts
But there are a few thing you should not do in these scripts
Do not do any of the following
- Run man or info
- Run more or less
- Run nano, vi, vim, emacs or any text editors
All these commands ask the user for input
I run scripts to check your Class Exercises
If you use any of the commands above the scripts will not run the way they should
You will get no credit for the assignment until you remove these commands

Tips and Examples

Do Not Fall Behind in This Course

Technical courses require that you master a great deal of detail
You will need to study for this course several times during the week
Studying once a week is usually not enough
The key to success in a technical course is steady, consistant, work
It should be spread out over several days
Some important concepts do not sink in immediately
It may take several days for something new to become clear
Material in this course often depends on what came before
So it is critical that you not fall behind
If you fall behind, it can be VERY hard to catch up

Read the Homework Assignments Carefully

Homework assignments may ask you to do many things
So you may need to read the assignment more than once
A single reading might not be enough to understand what I want you to do
Take the time to read these documents slowly
This will help you understand all the details
If you don't, you will make mistakes and lose points
Technical work is detailed work
So you must learn to read documents with a lot of detail
If you do not understand something post a question on Piazza
NEVER GUESS at the meaning of something I have written

Working with More than One Window

You can have more that one ssh window open at any one time
I normally run at least 4 ssh sessions on my Mac
Each one of these windows is a separate login session
If you close one of these windows it does not affect the others

Shell Scripts

The commands you type at the command line can be placed in a text file
That file can then be run and all the commands will be executed
Such a file is called a shell script
Here is a shell script that first goes to my home directory
Then it goes to my tmp directory
And lists its contents
```
cd
cd tmp
ls
```
If the name of the file is tmp.sh I can run it like this
```
bash tmp.sh
```

Review

Special Characters

There are some characters that have special meaning in Unix
They should not be used in file or directory names

They are

& : | * ? ' " [ ] ( ) $ < > { } # / \ ! ~

Three other characters, called whitespace, are also special
The whitespace characters are
- Space
- Tab
- Newline
Spaces are used to separate entries on the command line
They separate the command from its options and its arguments
Tab has a special meaning we will discuss later today
The newline character is what you get when you hit Enter on a PC ...
or Return on a Mac
When you are typing in a text editor, the newline moves you down to the next line
On the command line, hitting Enter or Return tells the shell you are done typing a command ...
and the shell should run the command you just entered

Quoting and Escaping

If you need to use a special character on the command line you must do one of two things
- Use quotes
- Escape the special character
Escaping means putting a backslash, \ , in front of a special character
This turn off the special meaning of the character
The backslash must come immediately in front of the special character
There are two types of quotes
Single quotes - ' '
Double quotes - " "
They have slightly different effcts ...
but you don't need to worry about the difference for now

`pwd` - Show Your Current Directory

pwd (print working directory) prints your current directory
```
pwd
```

`cd` - Change Directory

cd (change directory) changes your current directory
You use cd to move from one directory to another
```
cd DIRECTORY_NAME
```
If you use cd with an argument it moves you to that directory
If you use cd without an argument it takes you to your home directory
To go up one directory, use ..
```
cd ..
```
To go up more than one directory use multiple instances of ..
Each .. must be separated by a /
To go up two directories enter
```
cd ../..
```

`ls` - List the Contents of a Directory

ls shows you the files and directories inside a directory
You can use ls with or without an argument
If you give ls a directory name, it lists what is inside that directory
When you don't give ls an argument it lists the current directory
Like many Unix commands ls has many options
But two are used most often

The -l (el) option gives you a long listing

$ ls -l
total 20
-rw-rw-r--   1 ghoffman       103 Sep 11 14:34 basic.css
-rw-r--r--   1 ghoffman      3560 Aug 29 13:30 emacs_cheat_sheet.html
-rw-r--r--   1 ghoffman       701 Aug 29 13:30 index.html
drwxr-xr-x   6 ghoffman       512 Sep 15 14:11 it_244
-rw-r--r--   1 ghoffman      6831 Aug 29 13:30 tips.html
-rw-r--r--   1 ghoffman      6052 Aug 29 13:30 unix_cheat_sheet.html

The -a (for all) option also lists files whose names begin with . (dot)
These "invisible" files contain Unix configuration information

`cat` - Print the Contents of a File

cat (concatenate) prints a file
```
$ cat foo.txt 
foo
bar
blecth
```

When used with the -n option cat displays line numbers

$ cat -n numbered_lines.txt 
     1  Line 1
     2  Line 2
     3  Line 3
     4  Line 4
     5  Line 5

`rm` - Delete a File

rm (remove) deletes a file
```
rm FILENAME
```
There is no undelete feature in Unix
To remove all the files in a directory use
```
rm *
```

`mkdir` - Create a Directory

mkdir (make directory) creates a directory
```
mkdir DIRECTORY_NAME
```
The directory will be created in the current directory

`rmdir` - Delete a Directory

rmdir (remove directory) deletes a directory
```
rmdir DIRECTORY_NAME
```
rmdir will not work unless the directory is empty

`cp` - Copy Files

cp (copy) makes a copy of a file or a directory
```
cp  FILENAME  DIRECTORY_OR_NEW_FILENAME
```
cp takes two arguments
The first argument is the source
The file or directory to be copied
The second argument is the destination
Often the destination is a directory
If you are copying to the current directory the destination is a new filename
To copy a directory you must use the -r option

`mv` - Move a File or Directory

mv (move) is a command that does two different things
It can change the location of a file or directory
```
mv  FILENAME_OR_DIRECTORY_NAME  NEW_DIRECTORY
```

It can also change the name of a file or directory

mv  FILENAME_OR_DIRECTORY_NAME  NEW_FILENAME_OR_NEW_DIRECTORY_NAME

In either case, mv takes two arguments
The first argument is always the the name of the file or directory
When used to move, the the second argument is the new directory
When changing the name of a file or directory, the second is the new name

Attendance

New Material

`echo` - Print Text to the Terminal

echo simply prints whatever comes after it to the screen
```
$ echo Hello world!
Hello world!
```
It's used a lot in shell scripts to prompt for input ...
or to let the user know what is happening

It is also useful for seeing the value of a variable

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:...

When echo is used with the -n option it does not advance to the next line

`hostname` - Print the Name of Your Host Machine

Every machine on the network has a name
This name is known as the hostname
The hostname command prints the network name of the machine you are using
```
$ hostname
pe15
```
When used with the -i option hostname will print the IP address of the host machine
```
$ hostname -i
158.121.104.215
```
When used with the -d option hostname will show the domain name
```
$ hostname -d
cs.umb.edu
```
The -i option does not work on my Mac
Which is an example of how the Unix on Mac is slightly non-standard

Pagers - View a File One Screen at a Time

Pagers show the contents of a file one screenful at a time
Unix provides two paging programs, more and less
less is an improved version of more
The name is an example of dry Unix humor
It's not more, it's less
Both allow you to advance to the next screen by hitting the space bar
To move down one line hit Enter
To get help type h
less has more features
In less you can use the arrow keys to move up and down one line
less also has a search feature
To search for a string type / followed by the text you are looking for
To find the next occurrence of this string hit n
When more gets to the end of a file ...
hitting the Space bar will bring you back to the command line
When you get to the end of a file in less ...
you must type q to quit

Pathname Completion

When typing a long file name it is easy to make a mistake
To make things a little easier Unix provides a feature called pathname completion
We'll talk more about pathnames in a future class
If you need to enter the name of a file or directory on the command line ..
you can just type in the first few characters ...
and then hit the Tab key
Unix will try to fill in the rest
If nothing matches you will hear a beep
You will also hear a beep if there is more than one match
In this case when you hit the Tab key again ...
Unix will show you a list of possible matches
It will also restore your last command ...
so you can type more character to get the name you need

`grep` - Finding Strings inside Files

grep (get regular expression) searches a file
for lines containing a specific string
It prints the lines where it finds the string
A string is a collection of characters that may, or may not, be a word

grep uses the following format

grep  [-OPTIONS]  SEARCH_STRING  FILE ...

The words in capital letters, OPTION, SEARCH_STRING and FILE are placeholders
They stand in place of a value that you have to provide
Anything enclosed in square brackets is optional
Here is an example of how grep can be used

Say we have a text file

$ cat  red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-15      Red Sox @  Rays         Loss 6-9
2011-07-16      Red Sox @  Rays         Win 9-5
2011-07-17      Red Sox @  Rays         Win 1-0
2011-07-18      Red Sox @  Orioles      Win 15-10
2011-07-19      Red Sox @  Orioles      Loss 2-6
2011-07-20      Red Sox @  Orioles      Win 4-0
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-25      Red Sox vs Royals       Loss 1-3
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

And we want to find all the games the Sox won

We can use grep like this

$ grep Win red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-16      Red Sox @  Rays         Win 9-5
2011-07-17      Red Sox @  Rays         Win 1-0
2011-07-18      Red Sox @  Orioles      Win 15-10
2011-07-20      Red Sox @  Orioles      Win 4-0
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

grep is case sensitive
If we used "win" instead of "Win" we would get nothing
```
$ grep win red_sox.txt
```
Just like the Unix command line

If you want grep to ignore case you must run it with the -i option

$ grep -i win red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-16      Red Sox @  Rays         Win 9-5
2011-07-17      Red Sox @  Rays         Win 1-0
2011-07-18      Red Sox @  Orioles      Win 15-10
2011-07-20      Red Sox @  Orioles      Win 4-0
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

If your search string contains one of Unix's special characters ...
you need to use a backslash, \ ...
or use quotes
grep -r will search recursively through a directory
That means it looks through the directory and all the subdirectories

grep -v returns all lines that do not match the search string

$ grep -v Win red_sox.txt 
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-15      Red Sox @  Rays         Loss 6-9
2011-07-19      Red Sox @  Orioles      Loss 2-6
2011-07-25      Red Sox vs Royals       Loss 1-3
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3

There are many more useful options for grep
They can be found on the man page
I use grep on an almost daily basis
For example, let's say I want to add a field to a database table
I keep all my SQL code in text files
I can use grep to find every file that uses that particular table

`head` - View the Top of a File

Sometimes the first few lines of a file are all you need to see

head displays the first 10 lines of a file

$ head red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4
2011-07-07      Red Sox vs Orioles      Win 10-4
2011-07-08      Red Sox vs Orioles      Win 10-3
2011-07-09      Red Sox vs Orioles      Win 4-0
2011-07-10      Red Sox vs Orioles      Win 8-6
2011-07-15      Red Sox @  Rays         Loss 6-9

If you give head a number as an option it will display that number of lines

$ head -5 red_sox.txt 
2011-07-02      Red Sox @  Astros       Win 7-5
2011-07-03      Red Sox @  Astros       Win 2-1
2011-07-04      Red Sox vs Blue Jays    Loss 7-9
2011-07-05      Red Sox vs Blue Jays    Win 3-2
2011-07-06      Red Sox vs Blue Jays    Win 6-4

`tail` - View the Bottom of a File

tail is like head except it prints the last 10 lines of a file

$ tail red_sox.txt 
2011-07-22      Red Sox vs Mariners     Win 7-4
2011-07-23      Red Sox vs Mariners     Win 3-1
2011-07-24      Red Sox vs Mariners     Win 12-8
2011-07-25      Red Sox vs Royals       Loss 1-3
2011-07-26      Red Sox vs Royals       Win 13-9
2011-07-27      Red Sox vs Royals       Win 12-5
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

You can give tail a number as an option to specify the number of lines printed

$ tail -4 red_sox.txt 
2011-07-28      Red Sox vs Royals       Loss 3-4
2011-07-29      Red Sox @  White Sox    Loss 1-3
2011-07-30      Red Sox @  White Sox    Win 10-2
2011-07-31      Red Sox @  White Sox    Win 5-3

tail is especially useful when looking at log files
Log files are text files that record significant events
They are created automatically by programs that provide services ...
like a web server
The most recent entries are at the end of a file
Those are usually the lines you want to see when you are trying to solve a problem

`sort` - Print a File in Sorted Order

Unix has a number of utilities for manipulating text files

sort prints a file with the lines sorted

$ cat  fruit.txt 
grapes
pears
oranges
cranberries
apples
melons
blueberries

$ sort  fruit.txt 
apples
blueberries
cranberries
grapes
melons
oranges
pears

Note that sort does not change a file
It prints the lines of a file in a different order
sort prints the lines of a file in alphabetical order ...
unless you use one of its options

sort -r (reverse) will sort the file in reverse alphabetical order

$ sort -r fruit.txt 
pears
oranges
melons
grapes
cranberries
blueberries
apples

sort -n (number) will sort a file by number

So a line starting with "2" will appear before a line starting with "11"

$ cat numbers_jumbled.txt
11
1
17
2
3
15
4
5
14
6
13
7
8
9
10
12
16
18
19
20

$ sort numbers_jumbled.txt
1
10
11
12
13
14
15
16
17
18
19
2
20
3
4
5
6
7
8
9

$ sort -n numbers_jumbled.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

To sort a file in reverse numerical order use -nr

$ sort -nr numbers_jumbled.txt
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1

Class Exercise 5

Class Exercise 5 is the first exercise where you will have to write a shell script.

You will create a text file using nano, and type the Linux commands into it.

You must create this file in your ex5 directory.

Please note that the first few sections of Class Exercise 5 should not be included in your ex5.sh script.

Test your script to see that it works.

Course Work

Course Requirements

Tips and Examples

Review

New Material

Graded Quiz

Readings

Homework 3

Late Penalty

Do Not Email Me about Missing Assignments

Questions

Course Work

Importance of Exams

Preparing for Exams

Course Requirements

Shell Scripts for Homework Assignments

Scripts for Class Exercises

Tips and Examples

Do Not Fall Behind in This Course

Read the Homework Assignments Carefully

Working with More than One Window

Shell Scripts

Review

Special Characters

Quoting and Escaping

pwd - Show Your Current Directory

cd - Change Directory

ls - List the Contents of a Directory

cat - Print the Contents of a File

rm - Delete a File

mkdir - Create a Directory

rmdir - Delete a Directory

cp - Copy Files

mv - Move a File or Directory

Attendance

New Material

echo - Print Text to the Terminal

hostname - Print the Name of Your Host Machine

Pagers - View a File One Screen at a Time

Pathname Completion

grep - Finding Strings inside Files

head - View the Top of a File

tail - View the Bottom of a File

sort - Print a File in Sorted Order

Class Exercise 5

Class Exercise

Class Quiz

`pwd` - Show Your Current Directory

`cd` - Change Directory

`ls` - List the Contents of a Directory

`cat` - Print the Contents of a File

`rm` - Delete a File

`mkdir` - Create a Directory

`rmdir` - Delete a Directory

`cp` - Copy Files

`mv` - Move a File or Directory

`echo` - Print Text to the Terminal

`hostname` - Print the Name of Your Host Machine

`grep` - Finding Strings inside Files

`head` - View the Top of a File

`tail` - View the Bottom of a File

`sort` - Print a File in Sorted Order