Review

• Running a Command in the Background
• Jobs
• Moving a Job from the Foreground into the Background
• Aborting a Background Job
• Pathname Expansion
• The ? Metacharacter
• The * Metacharacter
• The [ and ] Metacharacters
• Built-ins
• Creating Startup Files
• Redirecting Standard Error
• Shell Scripts
• Making a Shell Script Executable
• Specifying Which Version of the Shell Will Run a Script
• Comments in Shell Scripts
• Shell Variables
• Local Variables
• Global Variables
• Keyword Shell Variables
• User-created Variables
• Positional and Special Parameters
• Quoting and the Evaluation of Variables
• Processes
• Process Structure
• Process Identification
• Executing a Command
• The Readline Library
• Readline Completion
• Aliases
• Shell Modification of the Command Line
• Alias Substitution
• Tilde, ~, Expansion
• Parameter and Variable Expansion
• Pathname Expansion
• Brace Expansion
• Shell Script Control Structures
• The if ... then ... Construct
• test
• The test Operators
• Using [ ]
• The if ... then ... else ... Construct
• The if ... then ... elif ... Construct
• for ... in ... Loops

Final Exam

The final exam will be held on Tuesday, May 19th 3:00 PM - 6:00 PM.

The exam will be held in this room.

If for some reason you are not able to take the Final at the time it will be offered, you MUST send an email to me before the exam so we can make alternative arrangements.

60% of the points on this exam will consist of questions from the Weekly Graded Quizzes.

You do not need to study a Graded Quiz question if the topic is not mentioned in either the Midterm or Final review.

There is a link to the answers to the graded quizzes on the class web page.

The other questions I will make up specifically for this exam.

For these questions you will have to know

• Absolute and relative pathnames
• The PATH system variable
• Access permissions
• Redirection
• Metacharacters
• Utilities

The today's class will be a review session.

You will only be responsible for the material in today's class, and the review for the Midterm, which you will find here.

Although the time alloted for the exam is 3 hours, I would expect that most of you would not need that much time.

The final is a closed book exam.

To prevent cheating, certain rules will be enforced during the exam.

Remember, the Midterm and Final determine 50% of your grade.

Course Evaluation

At the end of each semester we offer you the opportunity to say what you think about this course.

What have I done right?

What have I done wrong?

What can I do better?

You are not asked for you name.

So the submissions are anonymous.

I will not see your responses until after I have submitted grades for this course.

We collect this feedback through Course Evaluations.

I will use what you say to make this course better.

To complete the course evaluation, use the following link .

After I finish speaking I will leave the class but stand outside the door.

I will ask all of you to fill out the Course Evaluation

No Class Next Week

A normal semester has 28 classes.

But this semester has 29.

I do not have materials for a 29th class.

Next Tuesday I will have Zoom Office Hours, from 9 AM to 5 PM, instead of a class.

Attendance

Review

Running a Command in the Background

• Normally, when you run a command you have to wait for it to finish
• Such commands are said to be running in the foreground
• Unix gives you a way to get the command prompt back right away
• You can run the command in the background
• The background job loses it's connection to the keyboard ...
• and the shell will give you a prompt immedidately
• The shell will tell you when the background job has finished
• To run a command in the background ...
• type an ampersand, & , at the end of the command line

  $ sleep 5 &
  [1] 17895
  $

• The number in square brackets is the job number
• The second number is the process ID
• When you run something in the backgroud ...
• it is disconnected from the keyboard
• But the output will still go to the screen ...
• unless you redirect standard output

Jobs

• Every time you type something on the comand line and hit Enter ...
• you are creating a job
• A job can be a single command ...
• a pipeline with many commands ...
• or a script
• Each command you type will generate its own process ...
• but the collection of processes is a single job
• When run a job in the background
• by typing an & at the end ...
• it returns two numbers

  $ sleep 5 &
  [1] 7431
  $

• The job number is enclosed in brackets and comes first
• The second, larger, number is a the PID ...
• the process identification number
• To see the status of all background jobs ...
• run the jobs command

  $ sleep 10 & sleep 10 & sleep 10 &
  [1] 13256
  [2] 13257
  [3] 13258
  
  $ jobs
  [1]   Running                 sleep 10 &
  [2]-  Running                 sleep 10 &
  [3]+  Running                 sleep 10 &

Moving a Job from the Foreground into the Background

• There can only be one foreground job
• But there can be many background jobs
• Unix will let you move a job from the foreground ...
• into the background
• To do this, you must first suspend the foreground job
• Do do this hit Control Z
• A suspended job is not dead
• It is in a state of suspended animation
• It's still in RAM
• But it is not getting any time cycles to run
• To make a suspended job run in the background use bg
• bg stands for background
• Once placed in the background ...
• the job resumes running
• If more than one job is suspended
• you must give bg the job number

Aborting a Background Job

• There are two ways to abort a background job
• You can bring a background job to the foreground ...
• using the fg (foreground) command
• Once you have the job in the foreground ...
• it is connected to the keyboard ...
• so you can abort it using Control C
• If there is only one background job
• Run fg without an argument
• If there is more than one background job ...
• fg needs the job number as an argument
• You can also terminate any job using the kill command
• But to use kill you must tell it what to kill
• The usual way to do this is to give kill a process ID
• You can also use the job number with kill
• But you must precede a job number with a percent sign, %
• You can get the job number by using the jobs command

Pathname Expansion

• Pathname expansion allows you to create pathnames using a pattern
• The pattern is a combination of ordinary characters ...
• and metacharacters
• When you use these patterns after a command ...
• Bash expands the them into pathnames
• Bash then runs this altered command line

The ? Metacharacter

• The ? metacharacter stands for a single instance ...
• of any character

The * Metacharacter

• * will match any number of characters in a pathname
• It will even match no characters

The [ and ] Metacharacters

• The square brackets, [ and ], are also metacharacters
• They work somewhat like ?
• They only match a single character in a pathname
• But that character must be one of the characters within the brackets
• Instead of writing out the individual characters ...
• you can use a range of characters
• Write one character, a dash (-) and another charachter ...
• to specify the range

  $ ls bar[1-5].txt
  bar1.txt bar2.txt bar3.txt bar4.txt bar5.txt

Built-ins

• Not all commands can be found on disk as executable files
• Some are actually written as part of the shell
• Such commands are called built-ins
• When you run a built-in ...
• the shell does not have to create a new process
• Because the command runs in the same process ...
• as the shell

Creating Startup Files

• A startup file contains Unix commands ...
• that are run just before you get a prompt
• The startup file normally used by Bash is .bash_profile
• This file must be placed in your home directory
• The commands in this file are run automatically
• after you enter your password ...
• but before you get a prompt
• They are used to customize your Unix environment

Redirecting Standard Error

• Standard error is the data stream ...
• where error messages are sent
• Redirecting standard error allows a command or script ...
• to separate its output ...
• from its error messages
• To redirect standard error, you place a 2 in front of the >

  $ ls xxxx
  ls: cannot access xxxx: No such file or directory
  
  $ ls xxxx 2> error.txt
  
  $ cat error.txt 
  ls: cannot access xxxx: No such file or directory

Shell Scripts

• A shell script is a text file containing Unix commands
• A shell script can use any shell feature ...
• that is available at the command line
  • Ambiguous file references using the metacharacters ?, * and [ ]
  • Redirection
  • Pipes
• But not those features which are provided by the Readline Library
  • Command line editing (arrow keys, control key combinations)
  • Pathname completion (hit Tab to get more of a filename)
  • The history mechanism (up arrow to recall previous command line)
• Unix also provides control structures
  • Conditionals
  • Loops

Making a Shell Script Executable

• Making a scrip executable means you can run the script ...
• without having to type bash ...
• then the name of the script
• To make a script executable ...
• you must give it both read and execute permission
• Normally you would give a shell script 755 permissions
• The owner can read, write and execute
• The group and everyone else can read and execute

Specifying Which Version of the Shell Will Run a Script

• A script is just a text file ...
• with Unix commands
• The CPU does not understand text ...
• so a Unix shell is needed
• When the shell runs a script it creates a new subshell
• Normally this subshell will be the same shell version ...
• as your current shell
• But sometimes you want a script to run in a different shell version
• This can be done with a hashbang line
• The hashbang line must be the first line of the script
• The first two characters on the line ...
• must be a hash symbol, # ...
• followed by an exclamation mark, !
• After these two characters, you write the absolute pathname ...
• of the shell which will run the script

Comments in Shell Scripts

• Scripts have to be read by the people
  • Who write the program
  • Who maintain the program
  • Who use the program
• To make it clear what is happening inside a program ...
• you need to add comments to the code
• Anything following a hash mark, # , is a comment
• Except for the hashbang line

Shell Variables

• A variable is a place in memory with a name ...
• that holds a value
• To use the value of a variable ...
• put a dollar sign, $, in front of its name
• Like this

  $ echo $SHELL
  /bin/bash

• Some variables are set and maintained by the shell itself
• They are called keyword shell variables
• Or just keyword variables
• Other variables are created by the user
• The environment in which a variable can be used ...
• is called the scope
• Shell variables have two scopes
  • Local
  • Global

Local Variables

• Local variables only exist in the shell in which they are defined
• To create a local variable, use the following format

  VARIABLE_NAME=VALUE

• There cannot be any spaces on either side of the equal sign
• Variables are local unless you explicitly make them global
• If the value assigned to a variable has spaces or tabs ...
• you must quote it ...
• or put a backslash in front of each whitespace character

Global Variables

• Global variables are defined in one shell ...
• but can be used in all sub-shells
• The env command, when used without an argument ...
• prints the values of all global variables
• To make a variable global use the export command

  export VARIABLE_NAME=VALUE

Keyword Shell Variables

• Keyword shell variables have special meaning to the shell
• By convention, the names of keyword variables are always capitalized
• Most keyword variables can be changed by the user
• This is normally done in the startup file .bash_profile

User-created Variables

• User-created variables are any variables you create
• By convention, the names of user-created variables are lower case
• User-created variables can be either local or global in scope

Positional and Special Parameters

• Positional and special parameters are variables set by Unix
• Positional parameters allow a shell script ...
• to get arguments from the command line
• Whenever you run a script each token on the command line ...
• is assigned to a positional parameter
• The first token is assigned to positional parameter 0
• This parameter contains pathname of the script
• Each succeeding token on the command line ...
• is assigned to the next positional parameter

  $ cat print_positionals.sh
  #!/bin/bash
  #
  # Prints the value of the first four positional arguments
  
  echo
  echo 0: $0
  echo 1: $1
  echo 2: $2
  echo 3: $3
  
  $ ./print_positionals.sh foo bar bletch
  0: ./print_positionals.sh
  1: foo
  2: bar
  3: bletch

• Positional parameters are the usual way input is given to a script
• The special parameter is # contains the number of arguments

  $ cat print_arg_numbers.sh 
  #!/bin/bash
  #
  # Prints the number of arguments sent to this script
  
  echo
  echo This script received $# arguments
  
  $ ./print_arg_numbers.sh foo bar bletch
  This script received 3 arguments

Quoting and the Evaluation of Variables

• Whenever the value of a variable contains spaces or tabs
• You must quote the string or escape the whitespace character
• There are three ways this
  • Single quotes (' ')
  • Double quotes (" ")
  • Backslash (\)
• Everything surrounded by single quotes ...
• appears in the value of the variable exactly as you entered it
• But inside double quotes you can use the value of a variable ...
• by typing $ in front of a variable name
• The backslash, \ , only effects the character immediately following it

Processes

• A process is a running program
• When you run a program ...
• the kernel sets aside some RAM for the process ...
• and loads the executable code into it
• Unix is a multitasking operating system ...
• so many processes can run at the same time
• The shell runs in a process like any other program
• Every time you run a program a process is created
• Except when you run a built-in

Process Structure

• Processes are created in a hierarchical fashion
• When the machine is started, there is only one process
• This process is called either init ...
• or systemd
• init or systemd then creates other processes ...
• which provide the services running on the machine
• These child processes can create other processes
• init or systemd have a PID (Process ID) of 1
• They are the ancestor of every other processes ...
• that runs on the machine

Process Identification

• Each process has a unique Process ID (PID) number
• ps -f displays a full listing of information ...
• about each process running for the current user

  $ ps -f
  UID        PID  PPID  C STIME TTY          TIME CMD
  it244gh  26374 26373  0 13:41 pts/5    00:00:00 -bash
  it244gh  27891 26374  0 13:57 pts/5    00:00:00 ps -f

• The UID column shows the user's Unix username
• The PID column is the process ID of the process
• The PPID column is the process ID of the parent process ...
• the process that created this process
• The CMD column gives the command line that started the process

Executing a Command

• When you type something on the command line
• The shell asks the kernel to create a process for the program
• Then it sleeps ...
• waiting for the child process to finish
• When it finishes, it sends back an exit status
• Then the shell wakes up and gives you a prompt

The Readline Library

• The readline library is a collection of procedures ...
• which let you edit the command line
• When you use Control key combinations on the command line ...
• you are using the readline library
• Any program written in C can use the readline library

Readline Completion

• The readline library provides a completion mechanism
• Type a few letters of something ...
• and readline completion will try to supply the rest
• There are three forms of completion provided by the readline library
  • Pathname completion
  • Command completion
  • Variable completion

Aliases

• An alias is a string that you type on the command line
• When you hit Enter the shell replaces it with a command
• To define an alias, you use the alias command
• alias uses the following format in Bash

  alias ALIAS_NAME=ALIAS_VALUE

• There must be no spaces on either side of the equal sign, =
• If the value assigned to the alias has spaces, it must be quoted
• If you follow alias with the name of an alias ..
• it will print the definition

  $ alias ll
  alias ll='ls -l'

• Running alias without an argument ...
• will print all current aliases
• Aliases are not global
• They only work in the shell in which they are defined

Shell Modification of the Command Line

• But before the shell executes your command ...
• it first looks to see if it needs to change ...
• something you typed on the command line
• The shell can rewrite the command line before executing it
• It does this to implement features of the shell
• There are 10 different ways in which the shell can modify the command line
• You are only responsible for the following

Alias Substitution

• When Bash performs alias substitution ...
• it replaces the name of the alias ...
• with the value of the alias

Tilde, ~, Expansion

• Bash replaces ~ with the absolute address of your home directory
• If the ~ is followed by a user name
• Bash substitutes the absolute address of the home director ...
• for that user

Parameter and Variable Expansion

• After tilde expansion, Bash performs parameter and variable expansion

  $ echo $SHELL
  /bin/bash

• When you type a $ followed by the name of a variable ...
• Bash replaces this with the value of the variable

Pathname Expansion

• Pathname expansion is where we use metacharacters ...
• to form a pattern ..
• which is replaced by one or more pathnames
• The metacharacters are
  • ?
  • *
  • [ ]

After alias substitution, Bash performs brace expansion

Braces, { }, allow you to create several strings all at once

Braces can appear with strings of characters in front or behind

The braces contain strings of characters separated by commas

The expansion creates multiple strings

One for each string contained within the braces

If I wanted to create 5 foo files I could use braces expansion as follows

	$ touch foo{1,2,3,4,5}.txt

	$ ls
	foo1.txt  foo2.txt  foo3.txt  foo4.txt  foo5.txt

The strings inside the braces can contain one or more characters

	$ touch {a,ab,abc}.txt

	$ ls
	abc.txt  ab.txt  a.txt

Shell Script Control Structures

• Control structures are Unix statements that change the order of execution ...
• of commands within a program or script
• There are two basic types of control structures
  • Loops
  • Conditionals

The if ... then ... Construct

• The first conditional is the if ... then statement
• It has the format

  if TEST_COMMAND
  then
      COMMAND_1
      COMMAND_2
      ...
  fi

• The TEST_COMMAND can be any Unix command
• then and fi mark off the code block
• These commands are only executed if TEST_COMMAND returns a status code of 0
• The most common command used with if is test

test

• The test command evaluates a logical expression ...
• and returns a status code of 0 if the expression is true

The test Operators

• test has a number of operators
• The operators test for different conditions
• Some operators work only on numbers

  Operator	Condition Tested
  -eq	Two numbers are equal
  -gt	The first number is greater than the second
  -lt	The first number is less than the second

• test uses the different operators when comparing strings

  Operator	Condition Tested
  =	When placed between strings, are the two strings the same
  !=	When placed between strings, are the two strings not the same

• Note that test uses symbols (=) when comparing strings
• But letters preceded by a dash (-eq) when comparing numbers

Using test in Scripts

• Bash provides a synonym for test
• A pair of square brackets, [ ] ...
• that contain a logical expression
• You can write either

  test $number1 -gt $number2

  or

  [ $number1 -gt $number2 ]

The if ... then ... else ... Construct

• The another conditional is the if ... then ... else ... construct
• Which has the following format

  if TEST_COMMAND 
  then
      COMMAND_1
      COMMAND_2
      ....
  else
      COMMAND_A
      COMMAND_B
      ...
  fi

• If TEST_COMMAND returns an exit status of 0 ...
• the commands between then and fi are run
• Otherwise the commands between else and fi are executed

The if ... then ... elif ... Construct

• The if ... then ... elif ... construct allows you to create nested if statements

  if TEST_COMMAND_1
  then
      COMMAND_1
      COMMAND_2
      ...
  elif TEST_COMMAND_2
  then
      COMMAND_A
      COMMAND_B
  ...
  ...
  [else
      COMMAND_N1
      COMMAND_N2
      ...]
  fi

• elif stands for "else if"
• Notice that elif must be followed by then
• The else section is optional

for ... in ... Loops

• The for ... in loop has this format

  for LOOP_VARIABLE in LIST_OF_VALUES
  do
      COMMAND_1
      COMMAND_2
      ...
  done

• The block of loop commands starts after do ..
• and ends just before done
• With this loop, Bash
  • Assigns the first value in the LIST_OF_VALUES to the variable specified by LOOP_VARIABLE
  • Executes the block of commands between do and done
  • Assigns the next value in the LIST_OF_VALUES to the LOOP_VARIABLE
  • Executes the commands between do and the done again
  • And so on until each value in LIST_OF_VALUES has been used