IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 20

Microphone

Questions

Are there any questions before I begin?

Homework 9

I have posted Homework 9 here.

As usual, it will be due next Sunday at 11:59 PM.

Quiz 7

I have posted the answers to Quiz 7 here.

Today's Class

Today we will cover a number of topics including the history mechanism, the readline library, aliases and functions.

Tips and Examples

Seeing All Running Processes

By default, ps only shows you the processes running in your current shell
It will not show processes running in another terminal session ...
connected to the same machine
To see process running on all your sssh sessions ...
run ps -u
If you follow this with a username it will show all processes for that user
So if you wanted to see all processes run by root you would type
```
ps -u root
```
ps -aux will show all running processes

Review

Positional and Special Parameters

Positional and special parameters are special variables set by Unix
All these variables have names which are a single character
Their values change each time you run a command
We can see their values ...
but cannot change them
We have already encountered the special parameter ?
It contains the status code returned by the last command
Positional parameters are used by shell scripts
Positional parameters let a shell script get the values it needs ...
from the command line arguments
Each string on the command line is assigned to a positional parameter
The first token on the command line is assigned to positional parameter 0
This parameter contains the pathname used to run the script
Each additional token on the command line ...

is assigned to the next positional parameter

$ cat print_positionals.sh
#!/bin/bash
#
# Prints the value of the first four positional arguments

echo
echo 0: $0
echo 1: $1
echo 2: $2
echo 3: $3

$ ./print_positionals.sh foo bar bletch
0: ./print_positionals.sh
1: foo
2: bar
3: bletch

Positional parameters are the usual way input is given to a script
Another special parameter is #

# contains the number of command line arguments

 
$ cat print_arg_numbers.sh 
#!/bin/bash
#
# Prints the number of arguments sent to this script

echo
echo This script received $# arguments

$ ./print_arg_numbers.sh foo bar bletch
This script received 3 arguments

Quoting and the Evaluation of Variables

You have to be careful when giving a variable a value containing whitespace
```
$ name=Glenn Hoffman
Hoffman: command not found
```
Linux will not allow a variable's value to contain one of the whitespace characters
- Space
- Tab
- Newline
There are three things you can do to handle whitespace
- Use single quotes (' ')
- Used ouble quotes (" ")
- Use a backslash (\)
Single quotes are the most restrictive
Ever character surrounded by single quotes appears in the value ...
exactly as you entered it
So special characters like $ lose their special meaning ...

inside single quotes

$ team="Red Sox"

echo $team
Red Sox

$ cheer='Go $team'

$ echo $cheer
Go $team

If you want special character to work, use double quotes
```
$ cheer="Go $team"

$ echo $cheer
Go Red Sox  
```
Quotes affect everything they enclose
The backslash, \ , only effects the character immediately following it

Processes

A process is a running program
Processes have resouces that they need to do thier work
Among these resources are
- An area in RAM
- Time on the CPU
- Connections to the 3 standard data streams
Unix is a multitasking operating system
That means many processes can run at the same time
The shell runs in a process like any other program
Every time you run a program, a process is created ...
except when you run a builtin ...
because builtins run in the same process as your current shell
To run a shell script a process is created
Into this process is loaded the text of the script ...
and the binary code for Bash
The picture in memory looks like this

Process Structure

Processes are created in a hierarchical fashion
When the machine is started, there is only one process
This process is either init or systemd
Which one depends on what flavor of Linux you are using
This process then creates all the other processes needed
Different services, such as a web server, each need their own process
These new processes are child process of the first process
These child processes can create other processes
init or systemd has PID (Process ID) of 1 ...
and is the ancestor of every other processes on the machine

Process Identification

Each process has a unique Process ID (PID) number
ps -f displays a full listing of information about each process ...

created for the current user

$ ps -f
UID        PID  PPID  C STIME TTY          TIME CMD
it244gh  26374 26373  0 13:41 pts/5    00:00:00 -bash
it244gh  27891 26374  0 13:57 pts/5    00:00:00 ps -f

The UID column shows the user's Unix username
The PID column is the process ID of the process
The PPID column is the process ID of the parent process
The process that created this process
The CMD column gives the command line that started the process

Executing a Command

The shell is a program that lets you run commands
Each time you run a command ...
except a builtin ...
the shell has to ask the kernel to create a process
Then the shell sleeps ...
waiting for the child process to finish
When the child process finishes ...
it returns an exit status to the shell
The shell wakes from it's sleep when it gets the exit status
It is now ready for your next command
When you run a command in the background ...
the shell does not go to sleep
So it is ready to run another command
Child process inherit all the global variables ...
of the parent process

Attendance

New Material

History

If you hit ↑, the shell will bring back your last command
When you do this, you are using Bash's history mechanism
The history mechanism maintains a list of the command lines you have run
These command line entries are called events
Keep hitting ↑ to go back in time to previous commands
If you go too far, hit ↓ to go forward in time
The history list also serves as a record of what you have done
If a command does not work ...
you can use the history mechanism to see what you did wrong ...
and fix it with command line editing

To view the history list, use the history command

$ history
    2  exit
    3  cd
    4  cd it244/work
    5  pwd
    6  rm -rf *
    7  cd ~/it244/work
    8  pwd
    9  cp ~ghoffman/examples_it244/bother.sh .
   10  ls /home/ghoffman/examples_it244
   11  cp ~ghoffman/examples_it244/bother.sh .
   12  ./bother.sh
   13  ./bother.sh &
   14  jobs
   ....

Running history without an argument shows the entire list
By default, the list contains 500 values ...
which is probably more than you want to see
To show a smaller number of events ...

run history followed by a number

$ history 10
  498  ps
  499  exit
  500  exit
  501  history
  502  cd
  503  cd it244
  504  cd work
  505  ls
  506  history
  507  history 10

Notice that there is no - in front of the argument to history ...
unlike with head or tail

Variables that Control the History Mechanism

The history mechanism uses two lists of previous command lines
- A list stored in RAM
- A list stored in a file
As you you work on the command line ...
your commands are stored in the RAM list
When you logout, the RAM list is copied to the file list
The next time you log in, the file list is copied to the RAM list
This way the history mechanism can keep track of commands ...
from previous logins

There are three variables that Bash uses to manage the history mechanism

File	Contents
HISTFILE	The pathname of the file that records the command history. The default is ~/.bash_history
HISTSIZE	The maximum number of command lines saved in the RAM list during a given session
HISTFILESIZE	The maximum number of command lines saved in the file specified by HISTFILE, after you quit

All these variables are keyword variables
Notice that they are all capitalized

When Unix is set up, these variables are assigned default values

$ echo HISTFILE: $HISTFILE; echo HISTSIZE: $HISTSIZE; echo HISTFILESIZE: $HISTFILESIZE
HISTFILE: /home/it244gh/.bash_history
HISTSIZE: 500
HISTFILESIZE: 500

You can change the values of these variables in your .bash_profile file
Your history file list is kept in .bash_history in you home directory ...
unless you change the value of HISTFILE
Remember, the history mechanism uses two lists

List Location Number of Entries

File list ~/.bash_history $HISTFILESIZE

Memory list RAM $HISTSIZE
~/.bash_history contains commands from previous sessions
Commands from your current session are stored in a separate list in RAM
When you first log in the two lists are identical
Because the RAM list is copied from ~/.bash_history
New commands are added to the end of the list in RAM ...
and older commands are removed
This keeps the size of the RAM list limited to the value of $HISTSIZE
The RAM list is copied to ~/.bash_history when you quit

List	Location	Number of Entries
File list	~/.bash_history	$HISTFILESIZE
Memory list	RAM	$HISTSIZE

Using the History Mechanism

You can use the history list without using the ↑ and ↓ keys
Using the arrow keys to get back a very old command ...
would be very annoying
You would have to hit ↑ many times ..
to get the right event
But the history mechanism provides an easier way ...
if you know the event number ...

which you can get by running history

$ history 5
  515  cd ~ghoffman/examples_it244/
  516  pwd
  517  echo $PATH
  518  cd
  519  history 5

You can run a previous command again ...
with an exclamation mark, ! ...

followed by the event number

$ !517
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

When you do this the history mechanism ...
prints out the old command line ...
before running it
There must be no space between the ! and the number
If there is you will get an error message
```
$ ! 517
517: command not found
```
You can follow ! with a letter ...

which will run the last command beginning with that letter

$ !e
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Using `fc` to Edit and Run an Old Command

The utility fc (fix command) allows you to edit a previous command line ...
and then run it
fc is a builtin so it executes quickly
When run with no arguments ...
fc will open up an editor window ...
holding the last command
You can then
- Modify the command in the editor window
- Save your changes
- The modified command will be run when you quit
You can also run fc with an event number
```
fc EVENT_NUMBER
```
Running fc this way will put that command in the editor window ...
where you can change it ...
and run it
fc can also be used to view the history list
When run with the -l option ...

fc will list the last 16 command lines

$ fc -l
511  ls
512  cd
513  ls
514  history 5
515  cd ~ghoffman/examples_it244/
516  pwd
517  echo $PATH
518  cd
519  history 5
520  echo $PATH
521  ! 517
522  echo $PATH
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH
526  echo $PATH

You can also tell fc to list all events ...
starting with a certain number
You do this by running fc -l ...

followed by a space and a number

$ fc -l 522
522  echo $PATH
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH
526  echo $PATH
527  fc -l

You can also have fc -l list a range of events ...

by following fc -l with two numbers

$ fc -l 522 525
522  echo $PATH
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH

If you run fc -l with two strings ...
it will list a range of command lines
The list will start ...
with the last command line that matches the first string ...

and end with the last command that matches the last string

$ history 10
  521  ! 517
  522  echo $PATH
  523  traceroute -a standford.edu
  524  echo $PATH
  525  echo $PATH
  526  echo $PATH
  527  fc -l
  528  fc -l 521
  529  fc -l 522 525
  530  history 10

$ fc -l t f
523  traceroute -a standford.edu
524  echo $PATH
525  echo $PATH
526  echo $PATH
527  fc -l
528  fc -l 521
529  fc -l 522 525

When would you use fc?
To modify the last command you would simple hit ↑ ...
and use command line editing
But what if you wanted to change a command you ran some time ago?
In that case you could use history to get the event number ...
then edit the command using fc and that number

The Readline Library

The Readline library is a collection of procedures ...
written in the C programming language
These procedures are compiled into a binnary file ...
that can be used by any program written in C
These procedures give the program that uses them ...
the ability to edit the command line
The Readline library was created by the GNU project
Whenever you do command line editing ...
you are using the Readline library
Earlier I said that the device driver tty ...
provided command line editing
It's true that it does ...
but it does this using the Readline library
Any program running under Bash and written in C ...
can use the Readline library
I use it when running SQLite commands ...
at the SQLite command line
SQLite can do this because it was written in C
There are two modes available in the Readline library
- emacs mode
- vi mode
The Readline Library uses emacs mode by default
That is the mode we have been using in this class
Here are some of the more useful commands ...

for the emacs version of the Readline library

Command	Meaning
Control A	Move to the beginning of the line
Control E	Move to the end of the line
Control U	Remove everything from the text entry point to the beginning of the line
Control K	Remove everything from the text entry point to the end of the line
←	Move the text entry point one character to the left
→	Move the text entry point one character to the right
↑	Recall the previous command line entry in the history list
↓	Recall the following command line entry in the history list

If you want to switch to vi mode run the following command
```
set -o vi
```
Add this command to your .bash_profile ...
and you will always run Bash in vi mode

Readline Completion

The Readline library provides a completion mechanism
Type a few letters and hit Tab ...
and Readline completion will try to supply the rest
There are three forms of completion provided by the Readline library
They all use Tab to complete something
But they differ in what they complete

Pathname Completion

You already use the first kind of completion ...
when hitting Tab to complete a pathname
Type the first few letters ...
then hit Tab ...
and the Readline Library will finish as much of the pathname ...
as it can
If it cannot provide the full pathname ...

hit Tab again to see the possible completions

$ ls 
hw1  hw2  hw3  hw4  foo.txt bar.txt bletch.txt		
$ ls hw[Tab][Tab]
hw1/  hw2/  hw3/  hw4/

Command Completion

The Readline library will also complete the name of a command
Begin typing the first few letters of the command ...
then hit Tab
If there is more than one possible completion ...
you will hear a beep
If you hit Tab a second time ...

you will see a list of possible completions

$ e[Tab][Tab]
e2freefrag           elfedit              esac
e2fsck               elif                 etags
e2image              else                 etags.emacs23
e2label              emacs                ethtool
e2undo               emacs23              eval
e4defrag             emacs23-x            ex
ebrowse              emacsclient          exec
ebrowse.emacs23      emacsclient.emacs23  exit
echo                 enable               expand
ed                   enc2xs               expiry
edit                 env                  export
editor               envsubst             expr
editres              eqn                  extcheck
egrep                erb                  
eject                erb1.8

I use this feature when I run my Bash script ...
to collect your homework scripts
The name of this script is assignment_collect.sh
I type
```
$ ass[Tab]
```
And Bash fills in the rest

Variable Completion

Bash will also complete variable names ...
when your are using their value
When you type a dollar sign, $ ...
Bash knows you will follow it with a variable name
Type a few characters after $ and hit Tab
The Readline library will try to complete the variable name
```
$ bar=BLETCH

$ echo $b[Tab]
BLETCH
```
If there is more than one possibility, you will hear a beep

Aliases

aliases give you a shortcut ...
for running a command
Instead of typing the command and its options
You type the name of the alias
And Bash replaces it with the command ...
and maybe some options and arguments
I often want to get a long listing for a directory ...
and ls -l is quite a few characters to type
So I have defined an alias, ll
```
alias ll='ls -l'
```

If I want a long listing I can simply type ll instead of ls -l

$ ll
total 45
-rwxr-xr-x 1 ghoffman grad     38 Oct 11 20:05 border.sh
-rwxr-xr-x 1 ghoffman grad    135 Oct 16 08:35 bother.sh
-rwxr-xr-x 1 ghoffman grad     13 Oct 29 14:23 cheer.sh
-rwxr-xr-x 1 ghoffman grad    103 Oct  9 08:53 command_name.sh
-rwxr-xr-x 1 ghoffman grad     99 Oct 29 16:15 comment_test.sh
...

To define an alias, you use the alias command ...
with the following format
```
alias  ALIAS_NAME=COMMAND
```
There cannot be any spaces on either side of the equal sign, =
Just like when defining variables
If the command string has spaces it must be quoted
```
alias  la='ls -a'
```
If you run alias with no arguments ...

it will list all aliases currently defined

$ alias
alias bin="pu $bin'
alias binl="ls $bin'
alias ck755="ls -l *.sh | tr '\''-'\'' '\'' '\'' | grep '\''rwxr xr x'\'''
alias ckhb="head -1 *.sh | grep  /bin/bash'
alias cl="pu $cl'
alias clhws="pu $clhws'
alias clhwsl="ls $clhws'
alias cll="ls $cl'
alias clr="clear'
...

To see the value of an alias you have already defined ...
type alias with the alias name
```
$ alias ll
alias ll='ls -l'
```
In Bash an alias cannot use positional parameters
But an alias can in the TC shell
Although an alias cannot accept command line arguments in Bash ...
an argument can follow the alias
This is a subtle point
I can use my ll alias and follow it with the name of a directory
So if I were to type the following on the command line
```
ll /home/ghoffman
```
Bash substitutes ls -l for ll ...
and the command becomes
```
ls -l /home/ghoffman
```
But what if I wanted to create an alias for a pipe using two commands ...
and I needed to pass an argument to the first command?
For example if I wanted to create an alias for something like this
```
ls -l DIRECTORY_NAME | head
```
In Bash, I cannot create an alias like this
```
alias llh="ls -l $1 | grep txt"
```
Because an alias cannot use positional parameters
Instead of allowing aliases to accept command line arguments ...
Bash has functions
Functions in Bash can consist of many commands ...
and you can use positional parameters with each of these commands
We'll discuss functions a little later in this class
An alias cannot call itself
And aliases are not global

They only work in the shell in which they are defined

$alias ll='ls -l'

$ bash

$ ll
ll: command not found

The alias command is a builtin
```
$ type alias
alias is a shell builtin
```
Because aliases are always local ...
the alias command will only work ...
in the shell in which it is defined

Single Quotes Versus Double Quotes in Aliases

There are two types of quotes in Bash
- Single quotes - ' '
- Double quotes - " "
Both single and double quotes allow you to use a string ...
that contains whitespace ...

as a value to a variable

$ name="Glenn Hoffman'

$ echo $name 
Glenn Hoffman

The whitespace characters are
- Space
- Tas
- Newline
Single quotes turn off all special meanings of characters
```
$ echo 'My name is $name'
My name is $name
```
Variables are not evaluated when they are enclosed in single quotes
But double quote allow you to use the value of a variable using $
```
$ echo "My name is $name"
My name is Glenn Hoffman
```
Usually, when defining aliases you want to use single quotes
These quotes prevent variables from being evaluated when the alias is defined
They force variables inside an alias to be evaluated when they are used
Which is usually what you want

Examples of Aliases

I have defined a number of aliases to make my work easier

One alias makes it easier for me to list the "invisible files" in a directory

$ alias la="ls -a'

$ la
.   .addressbook   .bash_profile  .cshrc  .login   .pinerc
..  .bash_history  .cache         it244   .msgsrc  .ssh

Another alias makes it easier to see the most recently created files and directories

$ alias ltr
alias ltr="ls -ltr'

$ ltr ~/bin/shell
total 4
-rwxr-xr-x 1 ghoffman grad 107 Oct 13 09:42 border.sh
-rwxr-xr-x 1 ghoffman grad 339 Oct 13 09:42 homework_setup.sh
-rwxr-xr-x 1 ghoffman grad 306 Oct 14 00:18 homework_copy.sh
-rwxr-xr-x 1 ghoffman grad 156 Nov  5 17:09 step.sh

This alias sorts the long directory listing ...
with the most recent entries at the bottom
This make is easier to see the most recently modified files

I have created global variables for all the directories I visit often

export lncd=/courses/it244/s19/ghoffman            
export lntst=/home/ghoffman/code/it244_code/testing_scripts_it244
export lnwb=/home/ghoffman/public_html/it244_html

Then I create two aliases using each of these variables

One to go to that directory

alias lncd='pu $lncd'
alias lntst='pu $lntst'
alias lnwb='pu $lnwb'

And the other to run ls on the directory

alias lncdl='ls $lncd'
alias lntstl='ls $lntst'
alias lnwbl='ls $lnwb'

pu is a function which I will describe below
I follow a standard procedure when defining such aliases
First I create a variable holding the absolute address of the directory
Then I create an alias to go to that directory
The name of this alias ...
is the same as the name of the variable
There is no possibility of confusion since you put $ in front of a variable
The aliases to list the contents of the directory use the variable name
Followed by l (el)
I also use an alias to log into pe15
Instead of typing
```
ssh ghoffman@pe15.cs.umb.edu
```
I type
```
pe15
```

Here is the definition of pe15

$ alias pe15
alias pe15='ssh ghoffman@pe15.cs.umb.edu'

All my aliases are defined in a startup file

Functions

A function is a collection of shell commands that is given a name ...
and does something useful
Functions can accept arguments from the command line ...
using positional parameters
A function can be run anywhere in the filesystem ...
since it has no pathname
You simply type the name of the function to run it
Functions differ from shell scripts in a number of ways
- They are stored in memory (RAM), rather than in a file on disk
- The shell executes the function in it's own process
For these reasons, functions run faster than shell scripts
All the functions you define are loaded into the memory ...
of your shell process
So functions are like shell scripts you define in RAM

Functions definitions have the following form

FUNCTION_NAME ()
{
    COMMAND
    COMMAND
    ...
}

Where COMMAND is anything you can enter at the command line

$ whoson ()
> {
>   date
>   finger | grep 'pts/'$1
> }

$ whoson 14
Sun Nov 10 20:18:33 EST 2013
thamerfa  Thamer AlTuwaiyan   pts/14      2  Nov 10 18:44 (c-174-63-86-44.hsd1.ma.comcast.net)

Once you type the final } the definition is complete ...
and you get a command prompt back

You can define a function on a single command line

$ echo3 () { echo $1; echo $1; echo $1;} 

$ echo3 foo
foo
foo
foo

But you must use a semicolon after each command ...
including the last command
For clarity, you can precede the function name ...

with the keyword function

$ function cheer ()
> {
>   echo Go $1'!'
> }

$ cheer 'Red Sox'
Go Red Sox!

function is not a command

$ type function
function is a shell keyword

So it is optional when defining a function
You can pass command line arguments to a function ...

using the positional parameters

$ print_args ()
> {
>   echo "arg1: $1"
>   echo "arg2: $2"
> }

$ print_args foo bar
arg1: foo
arg2: bar

To see the definition of a function, you can use the type command
I declare the function pu everywhere I work

I use this in place of cd

$ type pu
pu is a function
pu () 
{ 
    echo;
    pushd $1 > /dev/null;
    ls --color=auto
}

$ pwd
/home/it244gh

$ pu ~ghoffman

bin                            issues      omar
bugs                           it114       public_html
code                           it116       stdwrk
course_files                   it116_test  submitted
course_materials               it117       tests_taken
course_scoring_documents       it117_test  tmp
demos                          it244       wrk
flask                          it244_test  xrchiv
html                           mail
hw1_piazza_filename_create.py  messages

$ pwd
/home/ghoffman

This function first prints a blank line ...
which makes things easier to read
Then it calls pushd
But pushd prints the directory stack
Which I find distracting
So I send this output to /dev/null
Next I run ls so I can see the contents of my new directory
I chose pu as the function name because it is shorter than "push"
When I call pu it goes to a new directory ...
and puts that directory on top of the directory stack
To return to the previous directory I use po
Which is short for popd
popd takes the current directory off the stack ...
and returns to the previous directory
But popd does not require an argument

So po is an alias

$ alias po
alias po='popd > /dev/null; echo; ls'

This alias uses popd to return to the previous directory
It redirects the printing of the directory stack to /dev/null
Then it prints a blank line, and prints the contents of the directory

To remove a function use the unset command

$ print_args foo bar
arg1: foo
arg2: bar

[591] (pe15) ghoffman - ~
$ unset print_args

[592] (pe15) ghoffman - ~
$ print_args foo bar
print_args: command not found

Functions, unlike aliases, can be make global ...
by calling export with the -f option
```
export -f FUNCTION_NAME
```
Here is an example

First I define the function hw

$ type hw
hw is a function
hw () 
{ 
    echo Hello world!
}

When I call the function I get
```
$ hw
Hello world!
```
Then I export it
```
$ export -f hw
```

The I create a Bash subshell and call hw

$ bash # create a subshell
ghoffman@pe15:~$ hw # call the function
Hello world!

Where to Define Variables, Aliases and Functions

Alias are local
And cannot be used in subshells
Aliases should be defined in your .bash_profile
If I need to use an alias in a script
I could include the following a the top
```
source ~/.bash_profile
```
Since functions can be made global by using export -f ...
you can define and export them in .bash_profile