IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 15

Today's Topics

Reading Assignment

The reading assignment for next week is chapter 8 of Sobell, The Bourne Again Shell.

Homework 7

I have posted homework 7 here.

Announcements

There will be a graded quiz next week.

It will cover the material we will discuss today, and the material from Class 13.

Tips and Examples

Killing All Running Jobs

In a previous semester, someone asked whether it is possible to kill all currently running jobs
I did some googling and the answer is yes
You can use command substitution
Command substitution has the format
```
    $(COMMAND)
```
The shell creates a process to run the command inside the parentheses
Then it replaces the dollar sign and parenthetical expression ...
with the output of that command
When jobs is run with the -p option ...
it returns the process numbers of the currently running jobs
```
    $ jobs -p
    15521
    15523
    15525
```
If you use command substitution with this command

you can kill all running jobs

    $ jobs
    [1]   Running                 ./bother.sh > /dev/null &
    [2]-  Running                 ./bother.sh > /dev/null &
    [3]+  Running                 ./bother.sh > /dev/null &
    
    $ jobs -p
    15579
    15582
    15585
    
    $ kill $(jobs -p)
    
    $ jobs
    [1]   Terminated              ./bother.sh > /dev/null
    [2]-  Terminated              ./bother.sh > /dev/null
    [3]+  Terminated              ./bother.sh > /dev/null
    
    $ jobs
    
    $

Seeing All Running Processes

If you run ps you will see all running processes ...

even those running in the backgroud ..

    $ jobs
    [1]+  Running                 ./bother.sh > /dev/null &

    $ ps
      PID TTY          TIME CMD
    18389 pts/2    00:00:00 bash
    19223 pts/2    00:00:00 bother.sh
    19229 pts/2    00:00:00 sleep
    19230 pts/2    00:00:00 ps

But what if you had more than one terminal session going?
Without additional options, ps will not show you any process ..
running in that session ...

unless you run it with the -u option

    $ ps -u
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    ghoffman  18389  0.0  0.1  23440  4768 pts/2    Ss   13:53   0:00 -bash
    ghoffman  19151  0.0  0.1  23312  4528 pts/3    Ss+  14:55   0:00 -bash
    ghoffman  19223  0.0  0.0  12424  1412 pts/2    S    14:59   0:00 /bin/bash ./bother.sh
    ghoffman  19306  0.0  0.0  12424  1396 pts/3    S    15:02   0:00 /bin/bash ./bother.sh
    ghoffman  19309  0.0  0.0   7196   620 pts/3    S    15:02   0:00 sleep 5
    ghoffman  19310  0.0  0.0   7196   620 pts/2    S    15:02   0:00 sleep 5
    ghoffman  19311  0.0  0.0  18448  1324 pts/2    R+   15:02   0:00 ps -u

This option will also give you more information
When you log out of a terminal session ...
any processes that are running in the background will keep going
ps -u will show you these processes
Some of you may have process running bother.sh in the background
Check for these zombie processes using ps -u ...
and kill them with the kill command

Review

Running a Command in the Background

Normally, when you run a command ...
you have to wait for it to finish
Such commands are said to be running in the foreground
Unix gives you a way to get the command prompt back ...
after running a command that will take a long time to finish
You can run the command in the background
The background job loses it's connection to the keyboard ...
and the shell will give you a prompt
The shell will tell you when the background job has finished
Every time a program runs, a process is created
The process has access to system resources ...
like memory (RAM) and a connection to the filesystem
Unix, like most OSs, is a multitasking operating system
This means you can have more than one process running at a time
To run a command in the background ...
enter an ampersand, & , at the end of the command line ...
just before hitting Enter:
```
    $ sleep 5 &
    [1] 17895
    $
```

Jobs

Every time you type something at the command line ...
and hit Enter ...
you are creating a job
Every time a program runs ...
a process is created for that program
A pipeline is a collection of commands joined by pipes
Each command will generate its own process ...
but the collection of all the separate processes ...
is a single job
Each process in a pipeline will have its own process ID
So as the pipeline progresses, the currently running process will change ...
but the job number does not change
The job is the collection of all processes created at the command line
You can have multiple jobs running at the same time ...
but only one job can be in the foreground at any one time
Every process has a process ID number ...
and every job has a job number
When you tell the shell to run a job in the background ...
it returns two numbers:
```
    $ sleep 5 &
    [1] 7431
    $
```
The job number is enclosed in brackets and comes first
The second, larger, number is the process identification number ...
of the first process in the job
The process identification number is also known as the PID
When the job finishes, the shell prints out a message
```
    [1]+  Done                    sleep 5
```
The message does not appear the moment the job finishes
The shell waits for the next time you hit Enter ...
and it prints the message after the output from the command

Moving a Job from the Foreground into the Background

There can only be one foreground job ...
though you can have many background jobs
Unix will let you move a job from the foreground ...
to the background
To do this, you must first suspend the foreground job
A suspended job is not dead ...
it is in a state of suspended animation
You can reactivate it later
To suspend a foreground job you must use the suspend key sequence
On our systems you suspend a job by hitting Control Z
After you do this, the shell stops the current process
It also disconnects it from the keyboard
Once the job is suspended ...
you can place it in the background using the bg command
bg stands for background
Once placed in the background, the job resumes running
If more than one job is running ...
you must give bg the job number

Aborting a Background Job

There are two ways to abort a background job
You can bring a job from the background to the foreground ...
using the fg (foreground) command
Once you have the job in the foreground ...
you can abort it using Control C
When there is more than one job in the background ...
you must specify the job number when using fg
You can also terminate any job using the kill command
But to use kill you must tell it what to kill
The usual way to do this is to give kill a process ID
If you don"t remember the process ID ...
run ps (process status) to get the process ID (PID)
You can also use the job number with kill ...
but you must precede a job number with a percent sign, %
You can get the job number by using the jobs command

New Material

Pathname Expansion

What if you wanted to get a long listing ...
on all files in a directory ...
whose names started with "example"?
If would be painful to type all the names ...
one at a time ...
as arguments to ls
Fortunately, Unix provides a better way
This is a feature called pathname expansion
The it is also sometimes called globbing
Pathname expansion uses meta-characters
Meta-characters are sometimes called wildcards
They allow you to specify a pattern
When the shell sees such a pattern on the command line ...
it does something before executing the command
The shell replaces the pattern with a sorted list ...
of all pathnames that match the pattern ...
then it runs this altered command line
The pattern is called an ambiguous file reference
What happens if the shell finds no matching pathnames?
In that case, it passes the ambiguous file reference ...
to the program called on the command line
The shell lets the program try to make sense of the pattern
Pathname expansion is an operation ...
performed by the shell ...
before the program is called
You can use as many meta-characters as you want to form a pattern
Pathname expansion lets you specify a set of files ...
with a minimum amount of typing
It also comes in handy ...
when you can"t remember the exact pathname
Pathname expansion is different from pathname completion ...
which you get by hitting Tab
Pathname completion only gives you one pathname
Pathname expansion can create several pathnames ...
with one pattern
Pathname completion is an operation handled by tty
Pathname expansion is performed by the shell

The ? Meta-character

The question mark, ?, meta-character ...
stands for any one character
For a long listing of everything in my current directory
whose names begin with "dir" ...
followed by a single additional character ...

I could use

    $ ls -ld dir?
    drwxrwxrwx 2 it244gh libuuid 512 2011-09-30 15:26 dir1
    drwxr--r-- 2 it244gh libuuid 512 2011-09-30 15:26 dir2
    drwxrw---- 2 it244gh libuuid 512 2011-09-30 15:29 dir3
    drwxrw---- 2 it244gh libuuid 512 2011-09-30 15:29 dir4

Meta-characters work with any command
```
    $ echo dir?
    dir1 dir2 dir3 dir4
```
The ? meta-character does not match ...
a leading period in a filename
You must explicitly enter a leading period, . ...
when specifying an "invisible" file

The * Meta-character

An asterisk, * , will match any number of characters in a pathname
It will even match no characters
To find all the directories with names beginning with "dir" ...

we can use the * meta-character

    $ ls -ld dir*
    drwxr-xr-x 2 it244gh libuuid 512 2011-10-04 13:52 dir
    drwxrwxrwx 2 it244gh libuuid 512 2011-09-30 15:26 dir1
    drwxr-xr-x 2 it244gh libuuid 512 2011-10-04 13:53 dir10
    drwxr--r-- 2 it244gh libuuid 512 2011-09-30 15:26 dir2
    drwxrw---- 2 it244gh libuuid 512 2011-09-30 15:29 dir3
    drwxr-xr-x 2 it244gh libuuid 512 2011-10-02 17:07 dir4

Notice that * returns more names than ?
It returned dir, which has no additional characters after the string "dir"
And it returned dir10 ...
since it will accept any number of characters
Note also that dir10 appears before dir2 ...
since the list the shell creates is sorted alphabetically
* can be used with any command ...

even those that don"t normally deal with files

    $ echo dir*
    dir dir1 dir10 dir2 dir3 dir4

* cannot be used to match the initial period, . ...
in a hidden filename

But you can list all the hidden file in a directory using *

    $ echo .*
    . .. .addressbook .addressbook.lu .bash_history .cache .cshrc .login .msgsrc .pinerc .ssh

The [ and ] Meta-characters

The square brackets, [ and ], are also meta-characters
They work somewhat like the ?
They only match a single character in a pathname ...
but the pathname character must match one of the characters ...
within the brackets
If I wanted a long listing of directories named dir1 dir2 and dir3 ...
but wanted to omit dir4

I could use

    ls -ld dir[123]
    drwxrwxrwx 2 it244gh libuuid 512 2011-09-30 15:26 dir1
    drwxr--r-- 2 it244gh libuuid 512 2011-09-30 15:26 dir2
    drwxrw---- 2 it244gh libuuid 512 2011-09-30 15:29 dir3

No matter how many characters are within the brackets ...
the pattern can match only a single character
You can use the bracket meta-characters with any program
```
    $ echo dir[123]
    dir1 dir2 dir3
```
You can use a range to avoid explicitly listing all characters
A range is specified by listing the first and last characters ...
of a sequence ...
separated by a dash, -

The sequence is specified by alphabetical order

    ls -ld dir[1-3]
    drwxrwxrwx 2 it244gh libuuid 512 2011-09-30 15:26 dir1
    drwxr--r-- 2 it244gh libuuid 512 2011-09-30 15:26 dir2
    drwxrw---- 2 it244gh libuuid 512 2011-09-30 15:29 dir3

The square brackets provide another shortcut
If you insert an exclamation mark, ! , or a caret, ^ ...
immediately after the opening bracket ...
the shell will match any single character ...

NOT included within the brackets

    $ echo foo[!46]
    foo1 foo2 foo3 foo5 foo7 foo8 foo9

Built-ins

Not all commands can be found on disk ...
as executable files
Some are actually contained in the shell itself
Such commands are called built-ins
When you run a built-in ...
the shell does not have to create a new process
Instead, the shell calls a procedure in its own code ...
to perform the task ...
so no subprocess is created
This makes execution faster
If there is an executable file with the same name as a built-in ...
the shell will run the built-in instead of the file
echo is a built-in
There is also an executable version of echo on the disk
```
    $ which echo
    /bin/echo
```
If you want to run the disk version of echo ...
you have to specify the pathname
```
    $ /bin/echo foo
    foo
```
Most built-ins have no executable counterpart on disk
Running which on such a command will find nothing
```
    $ which bg
    $
```
The type command will confirm this
```
    $ type bg
    bg is a shell builtin
```

type is also a built-in

    $ type type
    type is a shell builtin

Help for Built-ins

If you run man on a built-in command ...
you will get nothing
```
$ man bg
No manual entry for bg
```
info also has no information about built-ins
But there is a command that you can use with built-ins ...
to learn something about the commands
The command is help ...
and it is itself a built-in
If you follow help with the name of a built-in ...

it will display some information on that command

$ help bg
bg: bg [job_spec ...]
    Move jobs to the background.
    
    Place the jobs identified by each JOB_SPEC in the background, as if they
    had been started with `&".  If JOB_SPEC is not present, the shell"s notion
    of the current job is used.
    
    Exit Status:
    Returns success unless job control is not enabled or an error occurs.

help does not work for normal commands

$ help ls
-bash: help: no help topics match `ls".  Try `help help" or `man -k ls" or `info ls".

If you run help with no argument ...
it will show all the built-in commands ...

for which it can provide information

$ help 
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
These shell commands are defined internally.  Type `help" to see this list.
Type `help name" to find out more about the function `name".
Use `info bash" to find out more about the shell in general.
Use `man -k" or `info" to find out more about commands not in this list.

A star (*) next to a name means that the command is disabled.

 job_spec [&]                                               history [-c] [-d offset] [n] or history -anrw [filename>
 (( expression ))                                           if COMMANDS; then COMMANDS; [ elif COMMANDS; then COMMA>
 . filename [arguments]                                     jobs [-lnprs] [jobspec ...] or jobs -x command [args]
 :                                                          kill [-s sigspec | -n signum | -sigspec] pid | jobspec >
 [ arg... ]                                                 let arg [arg ...]
 [[ expression ]]                                           local [option] name[=value] ...
 alias [-p] [name[=value] ... ]                             logout [n]
 bg [job_spec ...]                                          mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] >
...

Different Shell Versions

The shell we have been using is Bash
Bash stands for Bourne again shell
The original Bourne shell was written by Steve Bourne ...
at AT&T's Bell Laboratories
The original Bourne shell ...
also called the sh shell ...
is still with us
Many scripts are needed to set up and maintain Linux and Unix
Many of these scripts are quite old ...
and were written before the Bash shell
Those scripts are still in use
There are subtle differences between different shells
It's best to run a script in the shell for which it was written
Debian Linux and its derivatives use a striped down version of Bash ...
called Dash
Dash is much smaller than Bash
Dash is designed only to run scripts ...
and has fewer features
Its memory footprint is small ...
so it loads and executes scripts faster than Bash
System V Unix introduced the Korn shell, ksh ...
which was written by David Korn
It introduced aliases and command line editing
It also introduced other features that are now found in Bash
Another popular shell is the C shell ...
which is popular with C programmers ...
because it"s scripting language looks like C
There are two versions of this shell
- csh
- tcsh
But the difference between them is slight
Both the Korn and C shells cannot run bash scripts
Our version of Ubuntu stores different shell versions in /bin
We will only concern ourselves with bash ...
and to a lesser extent sh

POSIX

A standard exists for how shells should run on Unix
that specifies how they must work
It was created by Portable Application Standards Committee ...
of the IEEE (Institute of Electrical and Electronics Engineers )
It is called POSIX (Portable Operating System Interface) 1003.2
The GNU community is working on making Bash fully compliant with POSIX
Until then you can run bash with the --posix option
This will make bash run in a way more compatible with the POSIX standard

Ways a Shell Can Be Created

There are three ways a user can run a shell
- Login shell
- Interactive non-login shell
- Non-interactive shell
There are subtle differences between these three types of shells
We'll concentrate on login shells in this course
But you should be aware of the existence of the other shell types

Your Login Shell

When you first login to Unix, you are running a shell
This shell is your login shell
Which shell version you run is determined by the SHELL system variable
```
    $ echo $SHELL
    /bin/bash
```
In Ubuntu, the default shell version is Bash
When your login shell starts up ...
it runs the commands found in /etc/profile
This is a file customized by the system administrator ...
for all users
You can create your own customizations ...
in a startup file in you home directory
That file must have one of these names
- .bash_profile
- .bash_login
- .profile
If there is more than one of these files ...
in your home directory...
bash will each execute them ...
in the order given above
We will use .bash_profile

Interactive Non-login Shells

The shell is a program, just like cat or ls
You can run another shell ...
as a sub-shell of your current shell ...

by typing the name of the shell at the command line

    $ ps
      PID TTY          TIME CMD
    12778 pts/1    00:00:00 bash
    12969 pts/1    00:00:00 ps
    
    $ bash
    
    $ ps
      PID TTY          TIME CMD
    12778 pts/1    00:00:00 bash
    12970 pts/1    00:00:00 bash
    12973 pts/1    00:00:00 ps
    $

Notice that there are two bash processes
Your login bash shell is still running ...
but you are now runnin a second bash ...
which is running inside your login bash shell ...
as a sub-shell
This sub-shell is the second type of shell
It is not a login shell ...
because you ran this shell from the command line ...
when you were already logged in
It is an interactive non-login shell
When you run script to generate a typescript file ...
you are working inside an interactive non-login shell
It is interactive because you can type commands to it through the keyboard ...
but it is not the shell you got when you logged in
A non-login interactive shell is a shell that you create ...
without having to enter a password
The commands in the startup files named above
- .bash_profile
- .bash_login
- .profile
are NOT run before starting this kind of shell
Instead, the commands found in .bashrc are run ...
for a non-login interactive shell
You are not limited to running Bash

You can also run sh in a sub-shell

    $ ps
      PID TTY          TIME CMD
    19874 pts/27   00:00:00 bash
    20500 pts/27   00:00:00 ps
    
    $ sh
    
    $ ps
      PID TTY          TIME CMD
    19874 pts/27   00:00:00 bash
    20510 pts/27   00:00:00 sh
    20526 pts/27   00:00:00 ps

You leave a interactive login shell by typing exit

    $ ps
      PID TTY          TIME CMD
    19874 pts/27   00:00:00 bash
    20737 pts/27   00:00:00 ps
    
    $ sh
    
    $ ps
      PID TTY          TIME CMD
    19874 pts/27   00:00:00 bash
    20743 pts/27   00:00:00 sh
    20751 pts/27   00:00:00 ps
    
    $ exit
    exit
    
    $ ps
      PID TTY          TIME CMD
    19874 pts/27   00:00:00 bash
    20771 pts/27   00:00:00 ps

Non-interactive Shells

When you create a file of Linux commands ...
you have created a shell script
This is what you have been doing in the Class Exercises
A shell script contains Unix commands ...
which only a shell can understand ...
but your current shell goes to sleep ...
when you run a program ...
so your shell has to create a sub-shell to run the commands
Such a shell is called a non-interactive shell
There is no standard startup file for such a shell
You can create a startup file for non-interactive shells ...
if you put name of the file in the shell variable BASH_ENV

Creating Startup Files

A startup file contains Unix commands ...
that are run just before you get a prompt
Bash normally uses two startup files
- .bash_profile
- .bashrc
.bash_profile commands are run ...
before you get a prompt in a login shell
.bashrc commands are run ...
before you get a prompt in an interactive non-login shell
.bash_profile is where you define variables
We will not be talking much about .bashrc
Most Ubuntu installations only use .bashrc ...
when running a GUI
Every time you open a window in a Linux GUI ...
you are creating an interactive non-login shell ...
which can be customized in .bashrc