IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 21

Today's Topics

Review

History
Using the History Mechanism
Using fc to Edit and Run an Old Command
The Readline Library
Readline Completion
Pathname Completion
Command Completion
Variable Completion
Aliases
Single Quotes Versus Double Quotes in Aliases
Functions

Homework 10

I have posted Homework 10 here.

As usual, it will be due next Sunday at 11:59 PM.

Quiz 7

I have posted the answers to Quiz 7 here.

Review

History

When you use the up arrow, ↑, to get a previous command ...
you are using the history mechanism
The history mechanism maintains a list of the command lines you type
These command line entries are called events

To view the history list, use the history command

    $ history
        2  exit
        3  cd
        4  cd it244/work
        5  pwd
        6  rm -rf *
        7  cd ~/it244/work
        8  pwd
        9  cp ~ghoffman/examples_it244/bother.sh .
       10  ls /home/ghoffman/examples_it244
       11  cp ~ghoffman/examples_it244/bother.sh .
       12  ./bother.sh
       13  ./bother.sh &
       14  jobs
       ...

If you run history without an argument ...
it will display all the events in this history list
By default, this list contains 500 values
To restrict how many lines are displayed ...

run history followed by a number

    $ history 10
      498  ps
      499  exit
      500  exit
      501  history
      502  cd
      503  cd it244
      504  cd work
      505  ls
      506  history
      507  history 10

Notice that there is no - in front of the number ...
as there must be when using head or tail

Using the History Mechanism

If you know the event number of a previous command ...
you can run it again by using an exclamation mark, ! ...

followed by the event number

    $ !517
    echo $PATH
    /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Notice, that the history mechanism prints out the old command line ...
before running it
There must be no space between the ! and the number ...
or you will get an error message
```
    $ ! 517
    517: command not found
```
If you follow the ! with a letter ...

the last command line that began with that letter will be run

    $ !e
    echo $PATH
    /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Using `fc` to Edit and Run an Old Command

The utility fc (fix command) allows you to edit a previous command line ...
and then run the new command
fc is a built-in, so it executes quickly
When run with no arguments fc will bring up an editor window ...
holding the last command line
You can then
- Modify the command in the editor window
- Save your changes
- Execute the modified command
Running fc with an event number ...
will put that event in the editor window
If you change your mind while in fc ...
you must delete all text from the window
If you don't fc will try to execute whatever you have left in the window
fc can also be used to view the history list
When run with the -l option, fc will list the last 16 command lines
You can also tell fc to list all command lines starting with a certain number

You do this by running fc -l followed by a space and a number

    $ fc -l 522
    522  echo $PATH
    523  traceroute -a standford.edu
    524  echo $PATH
    525  echo $PATH
    526  echo $PATH
    527  fc -l

You can also use fc -l to specify a range of events

To do this follow fc -l with two numbers

    $ fc -l 522 525
    522  echo $PATH
    523  traceroute -a standford.edu
    524  echo $PATH
    525  echo $PATH

The Readline Library

The readline library is a collection of procedures ...
which let you edit the command line
When you use Control key combinations on the command line ...
you are using the readline library
Any program running under Bash and written in C can use the readline library
There are two modes available in the readline library
- emacs mode
- vim mode
users3 is configured to use emacs mode by default
That is the mode we have been using

Here are some of the more useful commands for the emacs version of the readline library

Command Meaning

Control A Move to the beginning of the line

Control E Move to the end of the line

Control U Remove everything from the text entry point to the beginning of the line

Control K Remove everything from the text entry point to the end of the line

→ Move the text entry point one character to the right

← Move the text entry point one character to the left

↑ Recall the previous command line entry in the history list

↓ Recall the following command line entry in the history list

Command	Meaning
Control A	Move to the beginning of the line
Control E	Move to the end of the line
Control U	Remove everything from the text entry point to the beginning of the line
Control K	Remove everything from the text entry point to the end of the line
→	Move the text entry point one character to the right
←	Move the text entry point one character to the left
↑	Recall the previous command line entry in the history list
↓	Recall the following command line entry in the history list

Readline Completion

The readline library provides a completion mechanism
Type a few letters of something ...
and readline completion will try to supply the rest
There are three forms of completion provided by the readline library

Pathname Completion

The readline library provides pathname completion
You begin typing a pathname, then hit Tab
If there is only one pathname that matches ...
the readline library will provide the the rest of the pathname
If there is more than one possible completion ...
the readline library will beep
You can then enter more characters ...´
before hitting Tab again ...
or you can hit Tab right after the first beep ...
and the readline library will give you a list of possible completions
```
    $ ls hw[Tab][Tab]
    hw2/ hw4/ hw5/ hw6/
```
If the second Tab still gives you a beep ...
there are no possible completions

Command Completion

The readline library will complete the name of a command for you
Begin typing a command ...
then hit Tab ...
and the readline library will try to supply the rest of the command
If there is more than one possibility ...
you will hear a beep
If you hit Tab a second time ...
you will see a list of possible completions

Variable Completion

When you type a dollar sign, $ , followed immediately by some text ...
you are entering a variable name
The readline library knows this
and will attempt to complete the name of the variable
If there is more than one possibility, you will hear a beep
If you then hit Tab another time
you will see a list of possible completions
If no list appears after the second Tab ...
there are no possible variable name completions

Aliases

An alias alias is a string that the shell replaces with some other string
Usually, the value assigned to the alias ...
is a command or part of a command
To define an alias, you use the alias command
alias uses the following format in Bash
```
    alias ALIAS_NAME=ALIAS_VALUE
```
In Bash, there must be no spaces on either side of the equal sign, =
If the value assigned to the alias has spaces, it must be quoted
```
    alias  la="ls -a'
```

If you run alias with no arguments, it will list all aliases currently defined

    $ alias
    alias bin="pu $bin'
    alias binl="ls $bin'
    alias ck755="ls -l *.sh | tr '\''-'\'' '\'' '\'' | grep '\''rwxr xr x'\'''
    ...

If you follow alias with the name of an alias, it will display the definition
```
    $ alias ll
    alias ll="ls -l'
```
In Bash, an alias cannot accept an argument
Although a alias cannot accept an argument in Bash an argument can follow it
I can use my ll and follow it with the name of a directory
So you can't give an argument to an alias in Bash ...
unless you are giving it to the last command in the alias
Instead of allowing Bash to have aliases that accept an argument ...
Bash has functions
Aliases are not global
They only work in the shell in which they are defined

Single Quotes Versus Double Quotes in Aliases

There are two types of quotes in Bash
- Single quotes - ' '
- Double quotes - " "
You must use quotes, or a backslash, \ ...
when you give a variable a value that contains whitespace
The whitespace characters are
- Spaces
- Tabs
- Newlines (carriage returns)
Single quotes turn off all special meanings of characters
```
    $ echo 'My name is $name'
    My name is $name
```
Variables are not evaluated when they are enclosed in single quotes
But double quote allow you to use the dollars sign, $ ...

to get the value of a variable

    $ echo "My name is $name"
    My name is Glenn Hoffman

Usually you want to use single quotes ...
when defining alias
If you use single quotes any variables in the alias ...
will be evaluated when you use the alias
If you use double quotes, any variable in the alias ...
will be evaluated when it is defined
When the shell see as alias on the command line ...
it simply substitutes the value of the alias ...
for the name of the alias

Functions

A function is a collection of commands that is given a name
Functions can accept arguments from the command line
Functions can be run anywhere you happen to be in the filesystem ...
because function exist in memory ...
not on the disk
You simply type the name of the function to run it
Functions, unlike aliases, can have arguments
They use the same positional arguments that shell scripts use
The first positional argument is $1 ...
the second $2 ...
and so on
Functions differ from shell scripts in a number of ways
- They are stored in memory (RAM), rather than in a file on disk
- The shell preprocesses the function so it can execute more quickly
- The shell executes the function in it's own process
So functions run much faster than shell scripts
Functions are local to the shell in which they are defined
They do not work in subshells

Functions definitions have the following form

    FUNCTION_NAME ()
    {
        COMMANDS
    }

Once you type the final } the definition is complete

You can define a function on a single command line

    $ echo3 () { echo $1; echo $1; echo $1;} 
    
    $ echo3 foo
    foo
    foo
    foo

But you must use a semi-colon after each command ...
including the last command
For clarity you can precede the function name ...
with the keyword function
The keyword function is optional
You can pass command line arguments to a function

You do this using the positional parameters

    $ print_args ()
    > {
    >   echo "arg1: $1"
    >   echo "arg2: $2"
    > }
    
    $ print_args foo bar
    arg1: foo
    arg2: bar

Functions, like aliases, only work in the shell in which they are defined

New Material

Bash Features and Options

There are a number of features of the shell ...
which you can turn on and off
One example is the noclobber option
When this option is set ...
you cannot overwrite the contents of a file ...
with redirected output
To set a feature, use set -o ...
followed by a space and the feature name
```
    $ set -o noclobber
```

If I now try to redirect output to a file, the shell will prevent this

    $ echo "Go Red Sox" > output.txt 
    bash: output.txt: cannot overwrite existing file

To unset a feature use set +o ...
followed by a space and the feature name
```
    $ set +o noclobber
```

I can now overwrite a file with redirection

    $ cat output.txt 
    foo
    
    $ echo "Go Red Sox" > output.txt
    
    $ cat output.txt 
    Go Red Sox

You can find a list of shell features and options in Sobell
Shell features and options will not be on the final

Processing the Command Line

The shell can modify what you enter at the command line
It does this to provide features like aliases
To do this properly ...
the shell must modify the command line in a specific order
Otherwise, things could become terribly confused
There are 10 different ways in which the shell can modify the command line
The order in which the shell performs them is as follows
1. History Expansion
2. Alias Substitution
3. Brace Expansion
4. Title, ~, Expansion
5. Parameter and Variable Expansion
6. Arithmetic expansion
7. Command substitution
8. Word splitting
9. Pathname expansion
10. Process substitution

History Expansion

The first substitution Bash performs is history expansion
History expansion occurs when you use the exclamation mark, ! ...
in front of an event ID ...

to recall a previous command from the history list

    $ history 5
      540  cat output.txt 
      541  echo "Go Red Sox" > output.txt
      542  cat output.txt 
      543  echo foo
      544  history 5
      
    $ !543
    echo foo
    foo

Alias Substitution

After history expansion, Bash performs alias substitution

$ alias ll="ls -l'

$ ll
total 2
lrwxrwxrwx 1 it244gh man    34 Sep  6 21:09 it244 -> /courses/it244/f12/ghoffman/it244gh
drwxr-xr-x 2 it244gh ugrad 512 Oct 27 09:16 work

The shell evaluates an alias by substituting the value of an alias ...
for the name of the alias

Brace Expansion

After alias substitution, Bash performs brace expansion
Braces, { }, allow you to write several strings ...
in one operation
The braces contain strings of characters ...
separated by commas
The shell expands a brace by creating multiple strings ...
one for each string contained in the braces
If I wanted to create "foo" files numbered 1 to 5...

I could use braces expansion as follows

    $ touch foo{1,2,3,4,5}.txt
    
    $ ls
    foo1.txt  foo2.txt  foo3.txt  foo4.txt  foo5.txt

The shell expanded the braces ...
to create as many files ...
as there were strings inside the braces
The shell takes the string that appears before the braces ...
and sticks it in front of every string inside the braces ...
followed by the text the follows the braces ...
creating many new strings on the command line
The strings inside the braces can contain one or more characters

but each string must be separated from the others by a comma

    $ touch {a,ab,abc}.txt
    
    $ ls
    abc.txt  ab.txt  a.txt

There should not be any unquoted spaces or tabs ...
within the braces

If there is, the expansion will not work properly

$ touch {b , bc, b c d}.txt

$ ls -l
total 0
-rw-r--r-- 1 it244gh ugrad 0 Nov 14 10:37 ,
-rw-r--r-- 1 it244gh ugrad 0 Nov 14 10:37 b
-rw-r--r-- 1 it244gh ugrad 0 Nov 14 10:37 {b
-rw-r--r-- 1 it244gh ugrad 0 Nov 14 10:37 bc,
-rw-r--r-- 1 it244gh ugrad 0 Nov 14 10:37 c
-rw-r--r-- 1 it244gh ugrad 0 Nov 14 10:37 d}.txt

Tilde, ~, Expansion

After brace expansion, Bash performs tilde expansion
Whenever Bash sees a tilde, ~, by itself ...
it substitutes the absolute address of your home directory
```
    $ echo ~
    /home/it244gh
```
Whenever Bash sees a ~ followed by a Unix username ...
it substitutes the absolute address of the home directory of that account
```
    $ echo ~ghoffman
    /home/ghoffman
```
If there is no username matching the string following the ~
no expansion is performed
```
    $ echo ~xxx
    ~xxx
```
There are two other tilde expansions
- ~+
- ~-
When Bash sees ~+ ...

it substitutes the value of the current directory

    $ pwd
    /home/it244gh/work
    
    $ echo ~+
    /home/it244gh/work

When Bash sees ~-

it substitutes the value of the previous directory

    $ pwd
    /home/it244gh/work
    
    $ cd
    
    $ pwd
    /home/it244gh
    
    $ echo ~-
    /home/it244gh/work

Parameter and Variable Expansion

After tilde expansion, Bash performs parameter and variable expansion

    $ echo $SHELL
    /bin/bash
    
    $ echo $?
    0

Notice that this expansion comes after alias expansion ...
so you can use variables and parameters ...
when defining aliases

Arithmetic Expansion

After parameter and variable expansion, Bash performs arithmetic expansion
Unix treats everything on the command line as text ...
unless told otherwise
```
    $ echo 5 + 4
    5 + 4
```
Arithmetic expansion allows Bash to interpret characters as numbers ...
and to perform ordinary arithmetic upon them
But it does more than that
Whenever bash sees the $(( ...
it treats everything that follows as a number ...
or an arithmetic operator ...
until it sees ))
```
    $ echo $(( 5 + 4 ))
    9
```
It then evaluates the arithmetic expression inside the double parentheses ...
and subsitutes the resulting numeric value ...
for the entire $(( ... )) expression
The rules for evaluating arithmetic expressions
are the same as for the C programming language
They are mostly what you would expect

You can use variables within an arithmetic expression

    $ a=5
    
    $ b=3
    
    $ echo $a $b
    5 3
    
    $ echo $(( $a - $b ))
    2

Arithmetic expansion recognizes the following operators

Operator	Description
+	Addition
-	Subtraction
*	Multiplication
/	Divison
%	Remainder
**	Exponentiation
==	Equality
!=	Inequality
>	Greater than
>=	Greater than or equal
<	Less than
<=	Less than or equal
\|\|	Logical OR
&&	Logical AND
!	Logical N0T
=	Assignment
++	Increment
--	Decrement

Inside the arithmetic expression itself ...

you do not have to use a $ to get the value of a variable

    $ echo $a $b
    5 3
    
    $ echo $(( a * b ))
    15

Command Substitution

After arithmetic expansion, Bash performs command substitution
Command substitution uses the following format
```
    $(COMMANDS)
```
Where COMMANDS are any valid Unix commands
The commands inside the ( ) are run in a subshell ...
and the entire command substitution expresseion ...
$( ) and whatever is inside it ...
is replaced by the output of the commands

For example, if I wanted to set a variable to the current time and date, I could use

    $ today=$(date)
    
    $ echo $today
    Tue Oct 25 17:00:07 EDT 2011

There is an alternate format for command substitution

You can place the command within back tics, ` `

    $ ls -l `which bash`
    -rwxr-xr-x 1 root root 954896 2011-03-31 17:20 /bin/bash

Before running ls, Bash first runs the command
```
    which bash
```
And replaces the command with the value returned by which
ls can now take /bin/bash as its argument

Comand substitution can be used inside double quotes

    $ echo "Today is $(date +'%A, %B %d, %Y')"
    Today is Wednesday, November 13, 2013

The back tic is the character you get by holding down the Shift key ...
and pressing the same key you use for ~
The back tics do not work in the TC shell ...
and is easily mistaken for the single quote, ' ...
so I will not use it in this course

Word Splitting

After command substitution, Bash performs word splitting
When Bash gets a command line ...
it spits the text into tokens
Tokens are strings of characters ..
usually separated by whitespace
- Spaces
- Tabs
- Newlines (carriage returns)
I have found a small script which demonstrates word splitting ...

by showing all command arguments

$ args.sh foo bar bletch 
3 args: [foo] [bar] [bletch]

Word splitting does not work ...

when strings are quoted

$ args.sh 'foo bar bletch' 
1 args: [foo bar bletch]

$ args.sh "foo bar bletch"
1 args: [foo bar bletch]

But word splitting does work ...

with both command substituion

$ args.sh $(date)
6 args: [Wed] [Apr] [12] [16:19:17] [EDT] [2017]

and variable substitution

$ cheer="Let's go Red Sox!"

$ args.sh $cheer
4 args: [Let's] [go] [Red] [Sox!]

But there are some situations ...
where you want another character to separate tokens
You can do this using the Unix keyword variable IFS
IFS stands for Internal Field Separator
If you give IFS a value ...
say the colon : ...

Bash will be use this character to separate tokens

$ args.sh $PATH
1 args: [/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/oracle/12.1/client64/bin:/home/ghoffman/bin/shell:/home/ghoffman/bin/python/umb:/home/ghoffman/bin:/home/ghoffman/bin/shell:/home/ghoffman/bin/python/umb]

$ IFS=:

$ args.sh $PATH
15 args: [/usr/local/sbin] [/usr/local/bin] [/usr/sbin] [/usr/bin] [/sbin] [/bin] [/usr/games] [/usr/local/games] [/snap/bin] [/usr/lib/oracle/12.1/client64/bin] [/home/ghoffman/bin/shell] [/home/ghoffman/bin/python/umb] [/home/ghoffman/bin] [/home/ghoffman/bin/shell] [/home/ghoffman/bin/python/umb]

Word splitting will not be on the final

Pathname Expansion

After word splitting, Bash performs pathname expansion ...
also known as globbing
Pathname expansion is where you use meta-characters ...
to specify one or more pathnames
The metacharacters are used to create patterns ...
that are called ambiguous file references
The metacharacters are
- ?
- *
- [ ]

Here are some examples

    $ ls t*
    test1.txt  test2.txt  test3.txt
    
    $ echo t*
    test1.txt test2.txt test3.txt

Process Substitution

After pathname expansion, Bash performs process substitution
Process substitution allows you to create a file on the fly
A command is run in a sub-shell ...
and the lines generated by that command are treated as a file ...
which can be used by other Unix/Linux commands
Process substitution uses the following format
```
    <(COMMAND)
```
The output of the command that appears between the parenthesis ...
is placed in a Unix structure called a named pipe
Normal Unix pipes connect the output of one command ...
to the input of another command
Each command runs inside its own process ...
so a pipe allows one process to talk to another
This is called interprocess communications
Normal Unix pipes are also known as anonymous pipes ...
because they have no name
Anonymous pipes only last as long a it takes for the first command ...
to talk to the second command
Named pipes can last longer than anonymous pipes
They can actually be created and removed at the command line
When Unix performs process substitution it creates a process to run a command ...
and sends the output of that command to a named pipe
Then Unix redirects input ...
to come from the named pipe

We can use named pipes to compare the contents of two directories

    $ diff -y <(ls -1 tia777/ce) <(ls -1 jgreen/ce)
    ce1                             ce1
    ce10                            ce10
    ce11                                  <
    ce2                             ce2
    ce3                             ce3
    ce4                             ce4
    ce5                             ce5
    ce6                             ce6
    ce7                             ce7
    ce8                             ce8
    ce9                             ce9

Here we have two ls commands ...
each running in their own subshell ...
and each one sending output ...
to its own named pipe
We can then run diff to look for differences ...
in these two "files"
Process substitution will not be on the final