IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 13, Tuesday

Microphone

Graded Quiz

You can connect to Gradescope to take weekly graded quiz today during the last 15 minutes of the class.

Once you start the quiz you have 15 minutes to finish it.

You can only take this quiz today.

There is not makeup for the weekly quiz because Gradescope does not permit it.

Homework 7

I have posted homework 7 here.

It is NOT due this coming Sunday.

Instead it is due Sunday, October 27th.

This is to give you time to study for the midterm.

And to give me time to score it.

Midterm

The Midterm exam for this course will be held on Tuesday, October 22nd.

It will be given in this room.

It will consist of 25 questions like those on the quizzes.

60% of the questions will come from the Graded Quizzes.

The other 40% of points will be questions that I create specially for this exam.

For these questions you will have to know

Absolute and relative pathnames
The PATH system variable
Access permissions
Redirection & and pipes
grep
Utilities

The last class before the exam, Thursday, October 17th, will be a review session.

You will only be responsible for the material in the Class Notes for that class on the exam.

You will find the Midterm review Class Notes here.

If for some reason you cannot take the exam on the date mentioned above you must contact me to make alternate arrangements.

The Midterm is given on paper.

I scan each exam paper and upload the scans to Gradescope.

I score the exam on Gradescope.

You will get an email from Gradescope with your score when I am done.

The Midterm is a closed book exam.

You are not allowed to any resource, other than what is in your head, while taking the exam.

Cheating on the exam will result in a score of 0 and will be reported to the Administration.

Remember your Oath of Honesty.

To prevent cheating, certain rules will be enforced during the exam.

Remember, the Midterm and Final determine 50% of your grade.

Today's New Material Not on Midterm

The new material in today's class will not appear on the Midterm.

You will only be responsible for the material in the Class Notes for the next class.

They will cover Class Notes 2 through 12.

Quiz

There will be no graded quiz next week.

Never Call `nano` in a Class Exercise Script

If your Class Exercise script contains a call to nano it screws up my testing scripts.

If your script contains a call to nano you will receive a 0 on the assignment.

Questions

Are there any questions before I begin?

Tips and Examples

Studying for the Midterm with Flashcards

60% of the point on the Midterm come from Class Quiz Questions
You can use the flashcards you create for these questions when studying for the Midterm
But not all Class Quiz Questions will appear on the Midterm
If the flashcard question covers a topic not in the Midterm Review ...
you do not have to study it for the exam
You should remove these flashcards from your collection

Using `chmod` Twice to Change Permissions

There are two different ways that you can specify the permissions ...
when using chmod
- Symbolically
- Numerically
Different characters are used with each format
But there is another difference between these two ...
that is not immediately obvious
There are 3 different permissions for 3 different groups of users
That makes 9 different permissions
Each of these permissions can be either on or off
When you use the numeric format you must specify all 9 permission all at once
I sometime see a student using chmod multiple times using the numeric format
This does not work
The results of the first call to chmod are overwritten by the second call

Here is an example

$ chmod 700 memo.txt 

$ ls -l memo.txt 
-rwx------ 1 ghoffman faculty 0 Mar  5 16:38 memo.txt

$ chmod 077 memo.txt 

$ ls -l memo.txt 
----rwxrwx 1 ghoffman faculty 0 Mar  5 16:38 memo.txt

As you can see the 2nd call to chmod removed the owner's permissions
So you only use 1 call to chmod when using the numeric format
And you must use the numeric format in this class

Pipes and Redirection

If Unix did not have pipes we could use redirection to get the same effect
Run the first command and redirect the output to a file
Run the second command taking input from the file just created
Then remove the file
Let's say I wanted to find all Sox games the won in red_sox.txt
But only in the games against the Orioles
I could do this with two commands
First I use grep to find all games the Sox won
But instead of having the output go to the screen I would send it to a file
```
$ grep Win red_sox.txt > output.txt
```
Now I run a second grep again taking input from output.txt
```
$ grep Orioles output.txt
```
I can do the same operation on a single line with a pipe
Redirection make pipes possible
In a pipe we redirect standard output from one command ...
into standard input of the next command

Using `ls` to Construct a Long Pathname

Constructing a long pathname can be very frustrating
Whenever I need to create a long pathname I build it slowly using ls
Let's say I am in the IT 244 test directory for Class Exercise scripts
```
$ pwd
/home/ghoffman/code/it244_code/testing_scripts_it244
```
And I want to run cheer in /home/ghoffman/course_files/it244_files
This directory is not in my PATH, so I must use a pathname
If I did not remember the absolute pathname ...
and wanted to construct a relative pathname ...
I would proceed step by step using ls ...
starting with ..

I would keep using .. until I got to my home directory

$ ls ..
examples_it244          hw_solutions_it244        student_code
exercise_scripts_it244  hw_testing_scripts_it244  testing_scripts_it244

$ ls ../..
it116_code  it117_code  it244_code

$ ls ../../..
bin               course_scoring_documents  html             it116_test  it244.d      stdwrk       tmp
bugs              dbeta                     hw1_filename.py  it117       it244_test   submitted    wrk
code              demos                     issues           it117.d     mail         test         xrchiv
course_files      dir1                      it116            it117_test  omar         testing
course_materials  FOO.TXT                   it116.d          it244       public_html  tests_taken

I would continue using ls until I got down to the correct directory

$ ls ../../../course_files/
it114_files  it116_files  it117_files  it244_files

$ ls ../../../course_files/it244_files/
_attendance_it115.txt.orig  echo_no_option.sh      for_2.sh              make_upper.py         select_4.sh
_attendance_it244.txt.orig  echo_with_n_option.sh  for_3.sh              match_three.sh        select.sh
big_job.sh                  empty.txt              forever.py            numbered_lines.txt    set.sh
bother10.sh                 executable_check.sh    forever.sh            numbers1.txt          shell_test_1.sh
bother.sh                   file_check.sh          for_test.sh           numbers2.txt          shell_test_2.sh
break.sh                    five_lines.txt         fruit.sh              numbers.txt           shift_1.sh
case_1.sh                   foo10.txt              fruit.txt             param_test_1.sh       shift_2.sh
case_2.sh                   foo1.txt               go_home_bad_2.sh      param_test_2.sh       shift_3.sh
case.sh                     foo2.txt               go_home_bad.sh        print_arg_numbers.sh  special_param_test_1.sh
cat_file.sh                 foo3.txt               hello_1.py            print_foo_bar.sh      special_param_test_2.sh
cat_foo.sh                  foo4.txt               hello_2.py            print_foo.sh          special_param_test_3.sh
cheer.sh                    foo5.txt               hello.sh              print_positionals.sh  test
comment_test.sh             foo6.txt               here_instructions.sh  read_1.sh             test.txt
continue.sh                 foo7.txt               here.sh               read_2.sh             update.sh
count_to_five.sh            foo8.txt               hi_bye.sh             read_only_dir         usage_1.sh
count_to_nine.sh            foo9.txt               if_4.sh               red_sox.txt           usage_2.sh
count_until.sh              foo_in_dir.sh          it_students           repeat.sh
dir1                        foo.sh                 lines.txt             select_1.sh
dir_permissions             foo.txt                log.txt               select_2.sh
dir_tree                    for_1.sh               make_foo.sh           select_3.sh

$ ls ../../../course_files/it244_files/cheer.sh /

Now I can use ↑ and remove ls from the command line to run the script
```
$  ../../../course_files/it244_files/cheer.sh 
Let's go Red Sox!
```

Review

Computers are Fast But People are Slow

Most of the time we spend in front of a machine ...
we are doing one of two things
Typing or thinking
Thinking requires nothing of the machine
Typing make take us a few minutes
But reading what we type takes the machine microseconds
So most of the time you and I are using a machine ...
the machine is not doing anything for us
The only time we make a significant demand on the machine ...
is when we run a program
Engineers realized that a user did not need exclusive access to the machine ...
while they were working at the terminal
The machine could do some work for one user for a few microseconds ...
then it could stop and save what it had done ...
before moving on to the next person
The amount of time given to one user was called a time slice
Machines are so fast that the users would never notice the difference
The computer would still be working with one person at a time
But it seemed to the user that many people were using the machine ...
at the same time
To make this work a new type of operating system was needed
A multiuser operating system

The Birth of Unix

People knew how efficient multiuser operating systems would be
So there was a big push to make this happen
This lead to the creation of a project called Multics
It was a huge project involving government, General Electric and Bell Labs
The project was so big and unwieldy that it was never finished
Bell Labs was working on Multics but eventually decided to pull out
Two very talented programmers there, Ken Thompson and Denis Ritchie ...
had worked on the project
They decided that Multics was trying to do too much
They simplified the specification inherited from Multics ...
and made some very clever design decisions
When they implemented this simpler specification ...
they created Unix
One of their key design concepts was data streams

Data Streams

Computers work with information
They take information in
And they send information out
We can think of these flows of information as data streams
When we run a command that produces some result ...
the characters we see are a stream of data sent to the screen
This is an output stream
When we type something into a word processor ...
data flows from the keyboard to RAM
This is an input stream

Standard Input, Standard Output and Standard Error

Every Unix process always has access to three different data streams
- Standard Input
- Standard Output
- Standard Error

Standard input is where the program normally gets its input
By default, standard input is the keyboard
Standard output is where the program normally sends its output
By default, standard output is the screen
Standard error is where the program normally sends error messages
By default, standard error is the same as standard output
The screen

Redirection

Redirection changes where standard input comes from ...
or where standard output goes
Redirection is one of the features that makes Unix so flexible
You can take input from something other than the keyboard
Like a file
You can send output to something other than the screen
Like another file
Redirection is what makes pipes possible
A pipe sends the standard output from one program ...
into the standard input of another

Redirecting Standard Output

To redirect standard output use the greater than symbol, > ...
followed by the filename
This tells Unix to to change standard output
It no longer goes to the screen
Instead it goes to the file or device specified
The format for output redirection is
```
COMMAND  [ARGUMENTS]  >  FILENAME
```
If the file does not already exist it will be created

Redirecting Standard Input

Redirecting standard input takes input from something other than the keyboard
To do this, we use the less than symbol, <
Here is the format
```
COMMAND  [ARGUMENTS] <  FILENAME
```

Redirecting Standard Output Can Destroy a File

Redirecting standard output to an existing file overwrites its contents
You will replace the original contents of the file ...
with the output of the command

Adding Output to an Existing File

What if you wanted output to be added to an existing file?
In other words, add to the text at the bottom of the file
Unix allows you to do this
It is called appending
The append symbol is two greater than symbols ...
with no space in between, >>
The format is
```
COMMAND  [ARGUMENTS] >>  FILENAME
```

/dev/null

Sometimes a program will do something useful ...
but produce output you don't want
For situations like this, Unix provides /dev/null
Any output you send to /dev/null will disappear
It will never appear on the screen
You can also redirect input to come from /dev/null
If you do, /dev/null will send an empty string
/dev/null is most useful when dealing with error messages
Error messages will normally be mixed up with normal output
That is often not what you want
Sending error messages to /dev/null prevents this from happening

Attendance

New Material

Running a Command in the Background

Normally, when you run a command you have to wait for it to finish
Such commands are said to be running in the foreground
When the command does not take long to finish this is not a problem
But some commands run for a long time
Compilers can run for a several minutes if the source code is long enough
And simulations can run for days
Unix gives you a way to get the command prompt back ...
while the command is running
You can run the command in the background
When you run a command in the background two things happen
- The background command is disconnected from the keyboard
- You get a new prompt
Normally after Bash runs a command ...
it goes to sleep until the command has finished ...
and wakes up when it gets the exit status
A background command is disconnected from the keyboard
So you cannot talk to it by typing
But it is not disconnected from the screen
This means any output from the background command ...
will be mixed up with normal output
The shell will tell you when the background command has finished
When you run a program, a process is created
A process is a running program
The process has access to system resources
Like memory and the filesystem
Unix, like most operating systems, is multitasking
This means you can have more than one process running at a time
To run a command in the background type an ampersand, & ...
at the end of the command line ...
then hit Enter
```
COMMAND  [ARGUMENTS]  &
```
For example
```
$ sleep 5 &
[1] 17895
$
```
sleep is a command that makes a program pause for a certain number of seconds
It is used in shell scripts when the script is waiting for something to happen

Jobs

When you type something at the command line and hit Enter ...
you have created a job
When a program runs, a single process is created for that program
But what about a pipeline?
A pipeline is a collection of commands joined by pipes
Each command will have its own process
But the collection of all the separate processes is a single job
Each process in a pipeline will have its own process ID
As the pipeline progresses, the currently running process will change
But the job number does not change
The job is the collection of all processes created at the command line
If you run a Bash script, that script may start other processes
All these processes are part of the same job
You can have multiple jobs running at the same time
But only one job can be in the foreground at any one time
What's so special about the foreground?
Only the foreground job can accept input from the keyboard
Every process has a process ID number
And every job has a job number
When you tell the shell to run a job in the background ...
it returns two numbers
```
$ sleep  5  &
[1] 7431
$
```
The job number is enclosed in brackets and comes first
The second number is the process identification number ...
of the first process in the job
The process identification number is also know as the PID
When the job finishes, the shell prints a message
```
[1]+  Done                  sleep 5
```
The message does not appear the moment the job finishes
If it did that the message would be mixed up with normal output
That would be very annoying and you might miss it
Instead, the shell waits for the next time you run a command
You will first see the output of the new command
Then a message that the background job has finished
If you put a job in the background you should redirect standard output
Otherwise the output from the background job will go to the screen
This can be very confusing
Be sure to redirect output from a background job to a file ...
or /dev/null

Moving a Job from the Foreground into the Background

When you run a command normally it is running in the foreground
There can only be one foreground job
But you can have many background jobs
What if you were running a foreground job that took too long to finish ...
and you wanted to get your prompt back?
Unix will let you suspend the job
A suspended job is not dead, merely sleeping
You can reactivate it later
To suspend a foreground job you must type the suspend key sequence
Control Z is the most common suspend key sequence
I like to call this Control Zombie
It is what our systems use
After you type Control Z the shell stops the current job
It also disconnects it from the keyboard
The job, still exists but it has stopped running
It is in a state of suspended animation
Now you can place it in the background using the bg command
bg stands for background
To illustrate this I'll use a special shell program I created, bother.sh

bother.sh prints a message to the screen every few seconds

$ ./bother.sh 
Excuse me
Excuse me
Excuse me
^Z 
[1]+  Stopped                 ./bother.sh

$ bg 1
[1]+ ./bother.sh &
$ Excuse me
Excuse me
Excuse me
jobs
[1]+  Running                ./bother.sh &

Once placed in the background the job resumes running
If more than one job is running you must give bg the job number
There are also default job numbers that you can use with bg

Default Job

If you have many background jobs running and run jobs ...

you will see something like this

$ jobs
[1]   Running                 ./bother10.sh > /dev/null &
[2]-  Running                 ./bother10.sh > /dev/null &
[3]+  Running                 ./bother10.sh > /dev/null &

The job listed with a + is the default job
If you run a command like bg which needs a job number ...
and you do not supply one ...
the job with the + will be used
The jobs with the - are next in line ...
to be the default

Aborting a Background Job

How do you stop a job that is running in the background?
There are two ways
If the job were running in the foreground ...
you could stop it by hitting Control C
That works with a foreground job ...
because it is connected to the keyboard
But a background job can't hear anything from the keyboard
The keyboard is disconnected from background jobs
But you can bring a job from the background into the foreground
You do this using the fg (foreground) command
Once you have the job in the foreground ...

you can abort it using Control C

$ ./bother.sh &
[1] 10575

$ Excuse me
ls
bother.sh  sleep_echo.sh

$ Excuse me
Excuse me
fg
./bother.sh
Excuse me
^C

$

When there is more than one job in the background
You have to use the job number with fg
But there is another way to stop a background job
You can terminate any job using the kill command
But to use kill you must tell it what to kill
The usual way to do this is with a process ID
```
kill  PROCESS_ID
```
You are given the job and the process number ...
when you start the background job

If you forget them you can always run ps (process status)

$ ./bother.sh &
[1] 12444

$ Excuse me
ps
PID   TTY          TIME CMD
12264 pts/2    00:00:00 bash
12444 pts/2    00:00:00 bother.sh
12447 pts/2    00:00:00 sleep
12448 pts/2    00:00:00 ps

$ Excuse me
Excuse me

Once you have this information you can run kill

$ Excuse me
Excuse me
kill 12444
$ 
[1]+  Terminated              ./bother.sh
$

You can also use the job number with kill
But you must precede a job number with a percent sign, %
```
kill %JOB_NUMBER
```

You can get the job number by using the jobs command

$ ./bother.sh &
[1] 12543
$ Excuse me
Excuse me
Excuse me
jobs
[1]+  Running                 ./bother.sh &
$ Excuse me
Excuse me
Excuse me
Excuse me
Excuse me
kill %1
$ 
[1]+  Terminated              ./bother.sh
$

Tips and Examples

Review

New Material

Graded Quiz

Homework 7

Midterm

Today's New Material Not on Midterm

Quiz

Never Call nano in a Class Exercise Script

Questions

Tips and Examples

Studying for the Midterm with Flashcards

Using chmod Twice to Change Permissions

Pipes and Redirection

Using ls to Construct a Long Pathname

Review

Computers are Fast But People are Slow

The Birth of Unix

Data Streams

Standard Input, Standard Output and Standard Error

Redirection

Redirecting Standard Output

Redirecting Standard Input

Redirecting Standard Output Can Destroy a File

Adding Output to an Existing File

/dev/null

Attendance

New Material

Running a Command in the Background

Jobs

Moving a Job from the Foreground into the Background

Default Job

Aborting a Background Job

Class Exercise

Class Quiz

Never Call `nano` in a Class Exercise Script

Using `chmod` Twice to Change Permissions

Using `ls` to Construct a Long Pathname