IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 20

Today's Topics

Homework 9

I have posted Homework 9 here.

As usual, it will be due next Sunday at 11:59 PM.

Tips and Examples

Seeing All Running Processes

By default ps only shows you your processes ...
running in your current login
If you have another login going simultaneously ...
ps will not show you them ...
unless you use the -U option ...
followed by a Unix username
So if you wanted to see all processes run by root ...
you would type
```
ps -U root
```
You can see all the processes running on a machine ...
if you run ps with the -aux options
The -U option allows you to see all processes running o

Review

Positional and Special Parameters

Positional and special parameters are variables set by Unix ...
that change each time you enter a command
We have already encountered the special parameter ?
It contains the status code returned by the last command
Positional parameters are used by shell scripts
Positional parameters allow a shell script ...
to get arguments from the command line
Whenever you enter a command ...
each word on the command line ...
is assigned to a positional parameter
The first token on the is assigned to positional parameter 0
This parameter contains the pathname of the script
Each succeeding token on the command line

is assigned to the next positional parameter

$ cat print_positionals.sh
#!/bin/bash
#
# Prints the value of the first four positional arguments

echo
echo 0: $0
echo 1: $1
echo 2: $2
echo 3: $3

$ ./print_positionals.sh foo bar bletch
0: /print_positionals.sh
1: foo
2: bar
3: bletch

Positional parameters are the usual way input is given to a script
Another special parameter is #
# contains the number of arguments ...

passed to a program from the command line

$ cat print_arg_numbers.sh 
#!/bin/bash
#
# Prints the number of arguments sent to this script

echo
echo This script received $# arguments

$ ./print_arg_numbers.sh foo bar bletch
This script received 3 arguments

Quoting and the Evaluation of Variables

Whenever the value of a variable contains spaces or tabs ...
you must quote the string or escape the whitespace character
There are three ways this
- Single quotes (' ')
- Double quotes (" ")
- Backslash (\)
Single quotes are the most restrictive
Everything surrounded by single quotes ...
appears in the variable exactly as you entered it
This means that special characters like $

do not have their special meaning inside single quotes

$ team="Red Sox'

echo $team
Red Sox

$ cheer='Go $team'

$ echo $cheer
Go $team

Double quotes also preserve spaces and tabs ...
in the strings they contain
But you can use a $ in front of a variable name ...
to get the value of a variable ...

inside double quotes

       
$ cheer="Go $team"

$ echo $cheer
Go Red Sox

Quotes affect everything they enclose
The backslash, \ , only effects the character immediately following it

Removing a Variable's Value

There are two ways of removing the value of a variable

You can use the unset command

$ echo $foo
FOO

$ unset foo

$ echo $foo

$

Notice that the variable name was not preceded by a $
That's because we are dealing with the variable itself ...
not its value
Or you can set the value of the variable to the empty string
```
$ echo $foo
FOO

$ foo=

$ echo $foo

$
```

Processes

A process is a running program
Unix is a multitasking operating system
Many processes can run at the same time
The shell runs in a process like any other program
Every time you run a program ...
a process is created ...
except when you run a built-in
Running a built-in command does not create a process ...
because it runs in the same process as your current shell
When you run a shell script ...
your current shell creates a sub-shell to run the script
This sub-shell runs in a new process
When each command in the script is run ...
a process is created for that command

Process Structure

Processes are created in a hierarchical fashion
When the machine is started, there is only one process
This process is either init or systemd ...
depending on what flavor of Linux you are using
This process then creates all the other processes ...
needed to provide the services required of the machine
These new processes are child process of first process
These child processes can create other processes
init or systemd has PID (Process ID) of 1 ...
and is the ancestor of every other processes ...
that ever runs on the machine

Process Identification

Each process has a unique Process ID (PID) number

ps -f displays a full listing of information about each process running for the current user

$ ps -f
UID        PID  PPID  C STIME TTY          TIME CMD
it244gh  26374 26373  0 13:41 pts/5    00:00:00 -bash
it244gh  27891 26374  0 13:57 pts/5    00:00:00 ps -f

The UID column shows the user's Unix username
The PID column is the process ID of the process
The PPID column is the process ID of the parent process ...
the process that created this process
The CMD column gives the command line that started the process

Executing a Command

When you run a command from the shell ...
the shell asks the operating system to create a process ...
to run the command
Then it sleeps ...
waiting for the child process to finish
When the child process finishes ...
it notifies its parent process of its success or failure ...
by returning an exit status ...
and then it dies
When the parent process receives the exit status code ...
it wakes up and runs again
When you run a command in the background ...
the shell creates a child process for the job ...
but does not go to sleep
Global variables are inherited by all child processes

New Material

History

If you hit ↑ at the command line ...
the shell will bring back your last command
When you do this, you are using the history mechanism
The history mechanism maintains a list of the commands you have run
These command line entries are called events
Each time you hit the up arrow ...
the history mechanism shows you a previous command line
By repeatedly hitting ↑ ...
you can go back in time to see previous commands
If you go too far, hit the down arrow key, ↓ ...
to go forward in time
The history list also serves as a record of what you have done
If a command does not work ...
you can use this history to see what you did wrong

To view the history list, use the history command

$ history
    2  exit
    3  cd
    4  cd it244/work
    5  pwd
    6  rm -rf *
    7  cd ~/it244/work
    8  pwd
    9  cp ~ghoffman/examples_it244/bother.sh .
   10  ls /home/ghoffman/examples_it244
   11  cp ~ghoffman/examples_it244/bother.sh .
   12  ./bother.sh
   13  ./bother.sh &
   14  jobs
   ...

If you run history without an argument ...
it will display all the events in this history list
By default, this list contains 500 values ...
which is probably more than you want to see
To show a smaller number of events ...

run history followed by a number

$ history 10
  498  ps
  499  exit
  500  exit
  501  history
  502  cd
  503  cd it244
  504  cd work
  505  ls
  506  history
  507  history 10

Notice that there is no - in front of the number ...
as there must be when using head or tail

Variables that Control the History Mechanism

There are three variables that Bash uses to manage the history mechanism

File	Contents
HISTFILE	The location of the file that records the command history. The default is ~/.bash_history
HISTSIZE	The maximum number of command lines saved in a list in RAM during a given session
HISTFILESIZE	The maximum number of command lines saved in the file specified by HISTFILE after you quit

All these variables are keyword variables
Notice that they are all capitalized

When Unix is set up, these variables are assigned values

$ echo HISTFILE: $HISTFILE; echo HISTSIZE: $HISTSIZE; echo HISTFILESIZE: $HISTFILESIZE
HISTFILE: /home/it244gh/.bash_history
HISTSIZE: 500
HISTFILESIZE: 500

You can change the values of these variables in your .bash_profile file
Your history list is kept in .bash_history in you home directory ...
unless you change HISTFILE

So the history mechanism uses two lists

List	Location	Size
File list	~/.bash_history	$HISTFILESIZE
Memory list	RAM	$HISTSIZE

~/.bash_history contains commands ...
from your last terminal session
Commands from your current terminal session ...
are stored in a separate list
When you first log in the two lists are identical ...
because the initial value of the list in memory ...
is taken from the contents of ~/.bash_history
As you enter new commands at the terminal ...
these commands are added to the end of the list in RAM ...
and older commands are removed ...
to keep the size of the RAM list limited to the value of $HISTSIZE
When you quit your terminal session the contents of RAM list ...
are added to ~/.bash_history
If .bash_history already has the maximum number of lines ...
events from the top of the list are deleted ...
to make room for the new entries

Using the History Mechanism

The ↑ and ↓ keys ...
are not the only way to use the history list
If you only had the arrow keys to get back an old command ...
it would be very annoying retreiving a very old event
But the history mechanism provides an easier way
If you know the event number ...

which you can get by running history

$ history 5
  515  cd ~ghoffman/examples_it244/
  516  pwd
  517  echo $PATH
  518  cd
  519  history 5

you can run the command again ...
by using an exclamation mark, ! ...

followed by the event number

$ !517
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Notice, that the history mechanism prints out the old command line ...
before running it
There must be no space between the ! and the number ...
or you will get an error message
```
$ ! 517
517: command not found
```
If you follow the ! with a letter ...
the last command line that began with that letter

will be run

$ !e
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Using `fc` to Edit and Run an Old Command

The utility fc (fix command) allows you to edit a previous command line ...
and then re-execute it
fc is a built-in ...
so it executes quickly
When run with no arguments fc will bring up an editor window ...
holding the last command line
You can then
- Modify the command in the editor window
- Save your changes
- Execute the modified command
Running fc with an event number ...
will put that command in the editor window
If you change your mind while in fc editor ...
you must delete all text from the window
If you don't fc will try to execute whatever you have left in the window
fc can also be used to view the history list

When run with the -l option, fc will list the last 16 command lines

$ fc -l
511  ls
512  cd
513  ls
514  history 5
515  cd ~ghoffman/examples_it244/
516  pwd
517  echo $PATH
518  cd
519  history 5
520  echo $PATH
521  ! 517
522  echo $PATH
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH
526  echo $PATH

You can also tell fc to list all command lines ...
starting with a certain event number
You do this by running fc -l ...

followed by a space and a number

$ fc -l 522
522  echo $PATH
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH
526  echo $PATH
527  fc -l

You can also have fc -l list a range of events

To do this, follow fc -l with two numbers

$ fc -l 522 525
522  echo $PATH
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH

If you run fc -l with two strings ...
it will list a range of command lines
The list will start with the last command line that matches the first string ...

and end with the last command that matches the last string

$ history 10
  521  ! 517
  522  echo $PATH
  523  traceroute -a standford.edu
  524  echo $PATH
  525  echo $PATH
  526  echo $PATH
  527  fc -l
  528  fc -l 521
  529  fc -l 522 525
  530  history 10
  
$ fc -l t f
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH
526  echo $PATH
527  fc -l
528  fc -l 521
529  fc -l 522 525

The Readline Library

The Readline library is a collection of procedures ...
written in the C programming language ...
which let you edit the command line
The Readline library was created by the GNU project
When you use Control key combinations on the command line ...
you are using the Readline library
Any program running under Bash and written in C ...
can use the Readline library
I use it when running MySQL commands ...
at the MySQL command line
There are two modes available in the Readline library
- emacs mode
- vim mode
users3 is configured to use emacs mode by default
That is the mode we have been using

Here are some of the more useful commands for the emacs version of the Readline library

Command	Meaning
Control A	Move to the beginning of the line
Control E	Move to the end of the line
Control U	Remove everything from the text entry point to the beginning of the line
Control K	Remove everything from the text entry point to the end of the line
←	Move the text entry point one character to the left
→	Move the text entry point one character to the right
↑	Recall the previous command line entry in the history list
↓	Recall the following command line entry in the history list

Readline Completion

The Readline library provides a completion mechanism
Type a few letters and hit Tab ...
and Readline completion will try to supply the rest
There are three forms of completion provided by the Readline library
They all use Tab to complete something ...
but they differ in what they complete

Pathname Completion

Pathname completion is where to start to type a pathname ...
and then hit Tab ...
to have the Readline fill in the rest
If you type the first few characters of a pathname ...
and then hit Tab ...
the Readline library will try to supply the rest
If there is only one pathname that matches ...
the Readline library will provide the the rest of the pathname
If there is more than one possible completion ...
you will hear a beep
You can then enter more characters before hitting Tab again ...
or you can hit Tab right after the first beep ...
and the Readline library will give you a list of possible completions
```
$ ls hw[Tab][Tab]
hw2/ hw4/ hw5/ hw6/
```
If the second Tab still gives you a beep ...
there are no possible completions

Command Completion

The Readline library will complete the name of a command for you
Begin typing a command ...
then hit Tab
If there is more than one possible completion ...
you will hear a beep
If you hit Tab a second time ...

you will see a list of possible completions

$ e[Tab][Tab]
e2freefrag           elfedit              esac
e2fsck               elif                 etags
e2image              else                 etags.emacs23
e2label              emacs                ethtool
e2undo               emacs23              eval
e4defrag             emacs23-x            ex
ebrowse              emacsclient          exec
ebrowse.emacs23      emacsclient.emacs23  exit
echo                 enable               expand
ed                   enc2xs               expiry
edit                 env                  export
editor               envsubst             expr
editres              eqn                  extcheck
egrep                erb                  
eject                erb1.8

Variable Completion

When you type a dollar sign, $ ...
followed immediately by some text ...
you are entering a variable name
The Readline library knows this
and will attempt to complete the name of the variable
```
$ bar=BLETCH

$ echo $b[Tab]ar 
BLETCH
```
If there is more than one possibility, you will hear a beep
If you then hit Tab another time

you will see a list of possible completions

$ foo1=FOO; foo2=BAR

$ echo $foo[Tab][Tab]
$foo1  $foo2

If no list appears after the second Tab ...
there are no possible variable name completions

Aliases

An alias is a string ...
that the shell replaces with some other string ...
when you use it on the command line
Usually, the value assigned to the alias is a command ...
or a part of a command
I often want to get a long listing for a directory ...
and typing ls -l is quite a few characters
So I have defined an alias, ll
```
alias ll="ls -l'
```
If I want a long listing I can simply type ll ...

instead of ls -l

$ ll
total 45
-rwxr-xr-x 1 ghoffman grad     38 Oct 11 20:05 border.sh
-rwxr-xr-x 1 ghoffman grad    135 Oct 16 08:35 bother.sh
-rwxr-xr-x 1 ghoffman grad     13 Oct 29 14:23 cheer.sh
-rwxr-xr-x 1 ghoffman grad    103 Oct  9 08:53 command_name.sh
-rwxr-xr-x 1 ghoffman grad     99 Oct 29 16:15 comment_test.sh
...

To define an alias, you use the alias command
alias uses the following format in Bash
```
alias  ALIAS_NAME=ALIAS_VALUE
```
there must be no spaces on either side of the equal sign, =, ...
when defining an alias in Bash
If the string assigned to the alias has spaces ...
it must be quoted
```
alias  la='ls -a'
```
If you run alias with no arguments

it will list all aliases currently defined

$ alias
alias bin="pu $bin'
alias binl="ls $bin'
alias ck755="ls -l *.sh | tr '\''-'\'' '\'' '\'' | grep '\''rwxr xr x'\'''
alias ckhb="head -1 *.sh | grep  /bin/bash'
alias cl="pu $cl'
alias clhws="pu $clhws'
alias clhwsl="ls $clhws'
alias cll="ls $cl'
alias clr="clear'
...

If you follow alias with the name of an alias ...
it will display the definition
```
$ alias ll
alias ll="ls -l'
```
In Bash, an alias cannot accept an argument ...
but it can in the TC shell
Although a alias cannot accept an argument in Bash ...
an argument can follow it
This is a subtle point
I can use my ll alias ...
and follow it with the name of a directory
So if I were to type the following at the command line
```
ll /home/ghoffman
```
the shell would substitute "ls -l" for ll ...
and the shell would theb execute the changed commmand line
```
ls -l /home/ghoffman
```
But what if I wanted to create an alias for a pipe using two commands ...
and I needed to pass an argument to the first command?
For example if I wanted to create an alias for something like this
```
ls -l DIRECTORY_NAME | head
```
In Bash, I cannot create an alias like this
```
alias llh="ls -l $1 | grep txt'
```
because an alias will not accept an argument
When Bash comes across an alias ...
it substitutes the value of the alias ...
for the name of the alias
So you can't give an argument to an alias in Bash ...
unless the argument comes after the alias
Instead of allowing aliases to accept arguments ..
Bash has functions
Functions in bash can consist of many commands ...
and you can use arguments with each of these commands
We'll discuss functions a little later in this class
The TC shell has no functions
You can't use the name of an alias ...
inside the value of an alias
In other words, an alias cannot call itself
If you defined an alias ...
and then used the name of the alias in the value of the alias ...
how would Bash know when to stop?
In other words if you tried to do something like this
```
alias foo="foo foo foo'
```
The alias would try to call itself ...
and that call to the alias would try to call itself ...
and you would have an infinite recursive loop
To keep this from happening ...

an alias will not work if it calls itself

$ alias foo="echo foo'

$ foo
foo

$ alias foo="foo  foo  foo'

$ foo
foo: command not found

Aliases are not global

They only work in the shell in which they are defined

$alias ll="ls -l'

$ bash

$ ll
ll: command not found

The alias command is a built-in
```
$ type alias
alias is a shell builtin
```
This makes sense since an alias only works in the shell ...
in which it is defined
If alias were not a built-in ...
it would be defined in the subshell that ran the alias command ....
not your current shell

Single Quotes Versus Double Quotes in Aliases

There are two types of quotes in bash
- Single quotes - ' '
- Double quotes - " "
Both single and double quotes ...
allow you to assign a variable a value that contains whitespace
```
$ name="Glenn Hoffman'

$ echo $name 
Glenn Hoffman
```
The whitespace characters are
- Spaces
- Tabs
- Newlines (carriage returns)
Single quotes turn off all special meanings of characters
```
$ echo 'My name is $name'
My name is $name
```
Variables are not evaluated ...
when they are enclosed in single quotes
But double quote allow you to use the $ in front of a variable

to get the value of the variable

$ echo "My name is $name"
My name is Glenn Hoffman

Usually, when defining aliases ...
you want to use single quotes
If you use single quotes when defining an alias ...
any variables in the alias value will be evaluated ...
when you use the alias ...
which is usually what you want
If you use double quotes when defining an alias ...
any variable in the alias will be evaluated ...
when it is defined ...
not when it is used
Consider the following
The PWD keyword variable is used by the shell ...

to keep track of your current directory

$ pwd
/home/it244gh

$ echo $PWD
/home/it244gh

This means that the value of PWD changes ...
as you move about the filesystem
Let's see what happens if we define an alias using PWD ...

inside double quotes

$ alias where="echo My current location is $PWD "

$ where
My current location is /home/it244gh 

$ cd /

$ pwd
/

$ where
My current location is /home/it244gh

The value of PWD was evaluated ...
when the alias was defined
If I now move to another directory ...
the value of PWD will be changed ...
but that will not affect the alias ...
since it got the value of PWD when it was defined ...
and it ignores the current value of the variable
To define this alias correctly ...

we must use single quotes

$ alias where='echo My current location is $PWD '

Now PWD will be evaluated when the alias is used

$ pwd
/home/it244gh

$ alias where='echo My current location is $PWD'

$ where
My current location is /home/it244gh

$ cd /

$ pwd
/

$ where
My current location is /

Here we get the value of PWD ...
when the alias is run
When the shell comes across an alias ...
it simply substitutes the value of the alias ...
for the name of the alias

Examples of Aliases

I have defined a number of aliases ...
to make my work easier

One alias makes it easier for me to list the "invisible files" in a directory

$ alias la="ls -a'

$ la
.   .addressbook   .bash_profile  .cshrc  .login   .pinerc
..  .bash_history  .cache     it244   .msgsrc  .ssh

Another alias makes it easier to see the most recently created files and directories

$ alias ltr
alias ltr="ls -ltr'

$ ltr ~/bin/shell
total 4
-rwxr-xr-x 1 ghoffman grad 107 Oct 13 09:42 border.sh
-rwxr-xr-x 1 ghoffman grad 339 Oct 13 09:42 hw_setup.sh
-rwxr-xr-x 1 ghoffman grad 306 Oct 14 00:18 hw_copy.sh
-rwxr-xr-x 1 ghoffman grad 156 Nov  5 17:09 step.sh

This alias sorts the directory listing ...
with the most recent entries at the bottom
This make is easier to see the most recently modified files
I often create aliases using a variable ...

to go to directories I visit often

$ lnwb=/home/ghoffman/public_html/it244

$ alias lnwb="cd $lnwb'

$ pwd
/home/ghoffman/bin

$ lnwb

$ pwd
/home/ghoffman/public_html/it244

I follow a standard procedure when defining such aliases
First I create a variable ...
holding the absolute address of the directory
then I create an alias to go to that directory ...
and the name of the alias is the same as the variable
There is no possibility of confusion ...
since you must put a $ ...
in front of a variable to get its value ...
but you don't do this with an alias

Functions

A function is a collection of shell commands that is given a name
Functions can accept arguments from the command line ...
using positional parameters
A function can be run anywhere in the filesystem ...
since it has no pathname
You simply type the name of the function to run it
Functions differ from shell scripts in a number of ways
- They are stored in memory (RAM), rather than in a file on disk
- The shell preprocesses the function so it can execute more quickly
- The shell executes the function in it's own process
For these reasons, functions run much faster than shell scripts
Functions should be used sparingly
Functions take up memory
All the functions you define ...
are loaded into the memory of your shell process
If you define too many functions ...
it will hurt the performance of the shell
Functions are local to the shell in which they are defined ...
so they do not work in sub-shells

Functions definitions have the following form

FUNCTION_NAME ()
{
    COMMAND
    COMMAND
    ...
}

Where COMMAND is anything you can enter at the command line

$ whoson ()
> {
>   date
>   finger | grep 'pts/'$1
> }

$ whoson 14
Sun Nov 10 20:18:33 EST 2013
thamerfa  Thamer AlTuwaiyan   pts/14      2  Nov 10 18:44 (c-174-63-86-44.hsd1.ma.comcast.net)

Once you type the final } the definition is complete ...
and you get a command prompt back

You can define a function on a single command line

$ echo3 () { echo $1; echo $1; echo $1;} 

$ echo3 foo
foo
foo
foo

But you must use a semicolon after each command
Including the last command
For clarity, you can precede the function name

with the keyword function

$ function cheer ()
> {
>   echo Go $1'!'
> }

$ cheer 'Red Sox'
Go Red Sox!

function is not a command

$ type function
function is a shell keyword

So it is optional when definition a function
You pass command line arguments to a function ...

using the positional parameters

$ print_args ()
> {
>   echo "arg1: $1"
>   echo "arg2: $2"
> }

$ print_args foo bar
arg1: foo
arg2: bar

To see the definition of a function, you can use the type command
I declare the function pu everywhere I work

I use this in place of cd

$ type pu
pu is a function
pu () 
{ 
    echo;
    pushd $1 > /dev/null;
    ls --color=auto
}

$ pwd
/home/it244gh

$ pu ~ghoffman   

assignments_submitted  it114  mail    public_html  vp
bin                    it244  Mail    scans
course_files           it341  News    test
html                   it441  nsmail  test_taken

$ pwd
/home/ghoffman

This function first prints a blank line ...
which makes things easier to read
Then it calls pushd
But pushd prints the directory stack ...
which I find distracting ...
so I send this output to /dev/null
Next I run ls ...
so I can see the contents of my new directory
I chose pu as the function name ...
because it is shorter than "push"
Whenever I use pu ...
the directory stack keep a record of my last directory
To return to this directory I use po ...
which is short for "pop"
But po does not require an argument ...
since it uses the directory stack to return to a previous directory ...

so po is an alias

$ alias po
alias po="popd > /dev/null; echo; ls'

This alias uses popd to return to the previous directory ...
redirecting the printing of the directory stack to /dev/null
Then it prints a blank line ...
and prints the contents of the directory

To remove a function use the unset command

$ cheer 'Red Sox'
Go Red Sox!

$ unset cheer

$ cheer
cheer: command not found

Functions, like aliases, only work ...
in the shell in which they are defined

Where to Define Variables, Aliases and Functions

Global variables are visible in all sub-shells
Global variables should be defined in .bash_profile ...
in your home directory
That will make them available when you login
since the commands in .bash_profile ...
are run after your password is accepted
Aliases and functions cannot be made global ...
which means that if you define them in .bash_profile ...
they will not be available in interactive sub-shells
But since few people use interactive sub-shells ...
they should probably be put in .bash_profile ...
to keep things simple
If you need your aliases and functions in interactive sub-shells ...
you should define them instead in .bashrc ...
and then add the following in your .bash_profile
```
source .bashrc
```
or they won't be defined in your login shell