IT 244: Introduction to Linux/Unix
Class 19
Tips and Examples
Review
New Material
Microphone
Graded Quiz
You can connect to Gradescope to take weekly graded quiz
today during the last 15 minutes of the class.
Once you start the quiz you have 15 minutes to finish it.
You can only take this quiz today.
There is not makeup for the weekly quiz because Gradescope does not permit it.
Homework 9
I have posted Homework 9 here.
As usual, it will be due next Sunday at 11:59 PM.
Tips and Examples
Don't Use Google for Homework Assignments
- I often see Unix commands in homework assignments that I have not
talked about in class
- When this happens it usually means that a student has used Google ...
- to find a command required to complete a Step
- Doing this is shortsighted ...
- and often leads to errors that result in points lost
- Everything you need to complete a homework assignment is contained
in the Class Notes
- It may seem easier to complete your homework by Googling ...
- but you learn less when you do this
- The purpose of the homework assignments is to give you hands
on experience ...
- with the material in the Class Notes
- The act of trying something and having it work or not work ...
- is critical to the learning process
- Sometimes using a command properly ...
- requires you to think a bit about the material
- Doing this thinking is again a fundamental part of the learning
process
- When you Google for an answer you are avoiding the hard work
of learning
- You are trading a better score on an assignment ...
- for the knowledge that will enable you do to better on tests ...
- and in life
- And sometimes Google gives you the wrong answer
- For example, many of you gave the wrong answer to Step 10 of
homework 7
- That Step read
Using meta-characters, perform a long listing of the files foo21.txt, foo25.txt and foo29.txt.
This must be done using a single command.
- Many of you wrote the following
ls -l foo{21,25,29}
- But the { and }
characters are NOT metacharacters
- They are used in a feature of Unix called Brace Expansion
- All these answers were wrong
- Think
- Don't Google
Using Control Z with an Editor
fg
is used to bring a background job into the foreground
- But it will also bring a suspended job back to life
- This allows you to use Control Z to switch in and out of a text editor ...
- while working on a script
- You launch an editor to work on a script
- When you want to test your script you hit Control Z
- This suspends the editor and brings you back to the command line
- There you can run your script to test it
- When you want to return to the editor simply enter
fg
- You can also use this trick with
man
and info
Redirecting Output From Background Job
Questions
Are there any questions before I begin?
Review
Shell Variables
Creating a Shell Variable
- You create a variable using the following format
VARIABLE_NAME=VALUE
- Like this
$ username=ghoffman
$ echo $username
ghoffman
- There must be no spaces on either side of =
- If you include spaces you will get an error
$ username = ghoffman
username: command not found
- The value you assign to a variable must not contain a space
$ name=Glenn Hoffman
Hoffman: command not found
- There are two ways to deal with this
- Escape the
whitespace
characters with a \
- Quote the entire value
- The backslash character turns off the special meaning ...
- of the character that follows it
$ name=Glenn\ Hoffman
$ echo $name
Glenn Hoffman
- Of course you must put the backslash in front of every whitespace
character ...
- or you will get an error
$ team=Boston\ Red Sox
Command 'Sox' not found
- For this reason it is usually better to quote the entire value
- You can use either a single quote '
$ team='Boston Red Sox'
$ echo $team
Boston Red Sox
- Or a double quote "
$ team="Boston Red Sox"
$ echo $team
Boston Red Sox
- There is a difference between the two quotes
- But we will save that for another class
- Be sure to use the same quote at the end of the value ...
- as you used at the start
Scope
- Where a variable can be used is called the
scope
of the variable
- Shell variables have two scopes
- Local variables
only have meaning in the shell in which they are defined
- They have no meaning in any other subshell
- Global variables
are variables defined in one shell ...
- and have meaning in all sub-shells created from that shell
Local Variables
- By default every shell variable you create is a local variable
- You must do something special to make a variable global
- If I create the variable name ...
- I can use it in that shell
$ name='Glenn Hoffman'
$ echo $name
Glenn Hoffman
- But not in a subshell
$ bash
ghoffman@itserver6:~$ ps
PID TTY TIME CMD
29566 pts/16 00:00:00 bash
29792 pts/16 00:00:00 bash
29796 pts/16 00:00:00 ps
ghoffman@itserver6:~$ echo $name
$
- This means we cannot use a local variable defined in the login shell ...
- inside a shell script
Global Variables
Choosing the Scope of a Variable
Defining Global Variables
- You can define local variables inside a script
- And they will be available every time you run the script
- What about global variables?
- If a global variable is defined on the command line ...
- it will be defined during that login session
- But it will disappear as soon as you log out
- To make them permanent you need to define them in .bash_profile
- .bash_profile is a
startup file
- This file contains Linux commands that are run when you log in
- All startup files are contained in your home directory
Keyword Shell Variables
Important Keyword Shell Variables
- There are a number of keyword variables that affect your Unix session
- Some of the more important are
Variable | Value |
HOME |
The absolute pathname of your home directory |
PATH |
The list of directories the shell must search to find
the executable file to run a command
|
SHELL |
The absolute pathname of your default shell |
PWD |
The absolute pathname off your current directory |
PS1 |
Your command line prompt - what you see after entering each command |
PS2 |
The secondary prompt - what you see if you continue a command to a second line |
User-created Variables
env
- Show All Global Variables
The read
command
- Most scripts need input from the user
- One way to get this input is with the
read
command
read
is used to set the value of a local variable using the following format
read VARIABLE_NAME
- When Bash gets to this line in a script ...
- it stops and waits for the user to type something
- When the user hit Enter or Return ...
- Bash assigns whatever the user typed ...
- to the variable whose name appears after
read
- If you run
read
with the -p
option it will print a
prompt
read -p PROMPT VARIABLE_NAME
- Here is an example
#! /bin/bash
read -p "Please enter a value: " value
echo $value
- When I run this I get
$ ./read_2.sh
Please enter a value: hi there
hi there
- The text in blue is user input
Separating and Grouping Commands
| (pipe) and & (ampersand) as Command Separators
Continuing a Command onto the Next Line
Using Parentheses, ( ) , to Run a Group of Commands in a Subshell
The Directory Stack
- Bash provides a way for you to move back to previous directories
- The directory stack keeps tracks of each directory you enter
- But directory stack is only engaged ...
- when you use two special commands
- The stack operates on the principle of last in, first out
- You go back to a previous directory ...
- by removing the top directory from the stack ...
- and going to that directory
- You can keep doing this until you get where you want to be
- There are three commands that use the directory stack
dirs
- Displays the Directory Stack
pushd
- Pushes a Directory onto the Stack
- In programming, putting something onto a stack is called a push
pushd
changes your current directory just like cd
- But it also adds your new directory to the directory stack
- When used with an argument,
pushd
- Places the new directory on the stack
- Displays the current contents of the directory stack
- Moves to the new directory
popd
- Pops a Directory off the Stack
- In programming, removing a value from a stack is called a pop
popd
changes your directory to another directory
- But it also removes a directory from the stack
- When used without an argument,
popd
- Removes the top directory from the stack
- Prints the current stack
- Goes to the directory that is now on top of the stack
Attendance
New Material
Positional and Special Parameters
- Positional and special parameters are variables set by Unix
- The values of these variable change each time you enter a command
- We have already encountered the special parameter ?
- It contains the status code returned by the last command
$ ls bar.txt
bar.txt
$ echo $?
0
$ ls xxx
ls: cannot access xxx: No such file or directory
$ echo $?
2
- Positional parameters
are used by shell scripts to get command line arguments
- Each word on the command line is assigned to a positional parameter
- The first word is the pathname of the script
- It is assigned to positional parameter 0
- Each succeeding word on the command line is assigned to the next integer
$ cat print_positionals.sh
#!/bin/bash
#
# Prints the value of the first four positional arguments
echo 0: $0
echo 1: $1
echo 2: $2
echo 3: $3
$ ./print_positionals.sh foo bar bletch
0: ./print_positionals.sh
1: foo
2: bar
3: bletch
- Positional parameters are the usual way input is given to a script
- Another special parameter is #
- # contains the number of arguments passed to a program
from the command line
$ cat print_arg_numbers.sh
#!/bin/bash
#
# Prints the number of arguments sent to this script
echo This script received $# arguments
$ ./print_arg_numbers.sh foo bar bletch
This script received 3 arguments
- Notice that # counts the arguments to the script
- It does not count the name of the script itself
Quoting and the Evaluation of Variables
- Whenever the value of a variable contains spaces, you must quote the string ...
- or escape the whitespace character
- There are two kinds of quotes
- Single quotes, ' '
- Double quotes, " "
- Single quotes are the most restrictive
- Everything surrounded by single quotes ...
- appears in the value exactly as you typed it
- This means that special meaning of characters like $
are ignored
$ team="Red Sox"
echo $team
Red Sox
$ cheer='Go $team'
$ echo $cheer
Go $team
- Double quotes also preserve spaces in the strings they contain
- But you can use a $ in front of a variable name ...
- and the $ will get the value of the variable
$ cheer="Go $team"
$ echo $cheer
Go Red Sox
- Quotes affect everything they enclose
- The backslash, \, only effects the character immediately following it
$ foo=bar
$ echo $foo
bar
$ foo3=\$foo
$ echo $foo3
$foo
Processes
- A process is a running program
- Every process has resources that it needs to do its job
- Unix is a multitasking operating system
- This means many processes can run at the same time
- Every time you run a program ...
- a new process is created ...
- except when you run a built-in
- Running a built-in command does not create a process
- Because it runs inside the shell's process
- When you run a shell script ...
- bash creates a subshell for the script
- This subshell runs in a new process
System Calls
- When you run a command that is not a built-in
- The shell asks the kernel to create a process
- How does the shell make this request?
- It uses a
system call
- A system call allows a program written in language like C ...
- to run a procedure built into the
kernel
- When a program needs the kernel to do something ...
- it makes a system call
The History of Computer Languages
Creating a New Process
- So how does Bash ask the kernel to create a process to run a program?
- Bash, like most of Unix, is written in C
- Bash creates a process by making the system call
fork
- It does this with the following line of C code
pid = fork()
fork
causes the kernel to create a new process
- This new process is an exact duplicate of the your command line Bash
processs ...
- but with a different process ID
- The varialbe fork gets the value of
this new process ID
- The picture in RAM looks like this
- This original process is called the
parent process
...
- and the new process is called the
child process
- Notice that the process ID of the login Bash ...
- is the parent process ID of the new process
How the Shell Runs a Program
- After the C code in Bash calls
fork
we have two identical process
- They only differ in their process ID
- How can the new process run a different program?
- To understand how this happens we have to look at the Bash C code
- A new process is created with following C statement
pid = fork()
- The value of the variable pid is the
process ID, PID, of the new process
- The running code in both processes is Bash
- And they both have the same value for the variable
pid
- The next line of the Bash C code in both processes
- is an if statement that looks at the value of
pid
- If the value of this variable does not match the PID ...
- as it does in the original shell process ...
- the process makes a system call to
sleep
...
- and goes into a state of suspended animation
- If, however, the PID, matches the value of the variable
pid ...
- as it does in the new process ...
- it know it must run a new program ...
- not Bash
- To do this it makes system call to
exec
exec
replaces the Bash code in process memory ...
- with the binary code of the program ...
- that you entered at the command line
- The situation in RAM now looks like this
- When this child process code finishes running ...
- it sends an
exit status
to its parent process
- The parent process is your login Bash shell
- When it gets the status code from the child process ...
- the login shell process wakes up again ...
- and is ready for you next command
- This will not be on the Final exam
Process Structure
- Every process on a computer was created by a parent process ...
- with one exception
- How is the very first process created?
- Whenever a Unix system is turned on ...
- a special first process is created automatically
- This special process then creates other process
- These processes can create other processes ...
- and so on
- This creates a hierarchical structure much like the file system
- In the filesystem the special directory at the top called the root
- Every other file or directory is contained in this directory ...
- or one of its subdirectories
- Until recently the special first process was init
- But for various reasons this is no longer the case in Ubuntu
- Though it still may be true in other versions of Linux
- The first process on an Ubuntu system is now systemd
Booting a Unix Machine
- Whenever you turn on a Unix machine
- A spontaneous process is automatically created
- The code for this process is either
init
...
- or
systemd
- Either way it has a process ID of 1
- This process then creates all other processes ...
- which are needed to run the services the machine provides
- Unix can be run in a mode with only one user
- But this is only done under special circumstances
- When run in multiuser mode ...
- Unix will create processes that allow users to log in
- There are an a number of different programs that can be run in these process
- For example
- Our systems run
agetty
- Each one of these processes is connected to a physical or virtual terminal
- When the code detects an attempt to connect ...
- it displays a login prompt
- It then waits for the user to respond
- When that happens it runs
login
to verify the password
login
is located in the /bin directory
- If the password is accepted a system call is made to
exec()
...
- to run Bash ...
- and this process becomes your login shell
- Again, this will not be on the Final exam
Process Identification
- Each process has a unique Process ID (PID) number
- As long as the process runs, it has the same PID
- After a process terminates its PID can be assigned to a new process
ps -f
displays a full listing for each of the user's running processes
$ ps -f
UID PID PPID C STIME TTY TIME CMD
it244gh 26374 26373 0 13:41 pts/5 00:00:00 -bash
it244gh 27891 26374 0 13:57 pts/5 00:00:00 ps -f
- The UID column shows the Unix username of the account that started the process
- The PID column is the process ID of the process
- The PPID column is the process ID of the parent process
- That is the process that created this process
- The CMD column lists the command that is running in the process
- If I were to run
sleep
in the background ...
- and then run
ps -f
- I would see
$ sleep 10 & ps -f
[1] 27352
UID PID PPID C STIME TTY TIME CMD
ghoffman 27292 27287 0 15:12 pts/1 00:00:00 -bash
ghoffman 27352 27292 0 15:13 pts/1 00:00:00 sleep 10
ghoffman 27353 27292 0 15:13 pts/1 00:00:00 ps -f
- Notice that the parent process of both
sleep
and ps -f
is my login shell
pstree
will display a tree of all currently running processes
- If I run
pstree
on
pe15 I can see the process structure
systemd─┬─accounts-daemon─┬─{gdbus}
│ └─{gmain}
├─acpid
├─agetty
├─atd
├─automount───10*[{automount}]
├─bash
├─bother.sh───sleep
├─cron
├─cups-browsed─┬─{gdbus}
│ └─{gmain}
├─dbus-daemon
├─dhclient
├─irqbalance
├─2*[iscsid]
├─lvmetad
├─lxcfs───4*[{lxcfs}]
├─master─┬─pickup
│ └─qmgr
├─mdadm
├─ntpd
├─polkitd─┬─{gdbus}
│ └─{gmain}
├─rpc.statd
├─rpcbind
├─rsyslogd─┬─{in:imklog}
│ ├─{in:imuxsock}
│ └─{rs:main Q:Reg}
├─rwhod───rwhod
├─screen───bash
├─screen───bash───vim
├─snapd───7*[{snapd}]
├─sshd─┬─3*[sshd───sshd───bash───alpine]
│ ├─2*[sshd───sshd───bash]
│ ├─sshd───sshd───bash───pstree
│ └─sshd───sshd───bash───ssh
├─10*[systemd───(sd-pam)]
├─systemd-journal
├─systemd-logind
├─systemd-udevd
└─ypbind───2*[{ypbind}]
- Where you see a number followed by a * ...
- it means there are multiple versions of that program ...
- are running in different processes
- You can see this more clearly if you run
pstree
with the -p option
- Now
pstree
will show each process along with its process ID
systemd(1)─┬─accounts-daemon(2099)─┬─{gdbus}(2136)
│ └─{gmain}(2134)
├─acpid(2076)
├─agetty(2481)
├─atd(2103)
├─automount(42937)─┬─{automount}(42938)
│ ├─{automount}(42939)
│ ├─{automount}(42942)
│ ├─{automount}(42945)
│ ├─{automount}(42946)
│ ├─{automount}(42947)
│ ├─{automount}(42948)
│ ├─{automount}(42949)
│ ├─{automount}(42950)
│ └─{automount}(42951)
├─bash(27573)
├─bother.sh(16996)───sleep(47645)
├─cron(2087)
├─cups-browsed(40751)─┬─{gdbus}(40833)
│ └─{gmain}(40832)
├─dbus-daemon(2089)
├─dhclient(2246)
├─irqbalance(2465)
├─iscsid(2355)
├─iscsid(2356)
├─lvmetad(1199)
├─lxcfs(2101)─┬─{lxcfs}(2126)
│ ├─{lxcfs}(2127)
│ ├─{lxcfs}(3081)
│ └─{lxcfs}(29814)
├─master(21825)─┬─pickup(44495)
│ └─qmgr(31008)
├─mdadm(2148)
├─ntpd(2507)
├─polkitd(2164)─┬─{gdbus}(2180)
│ └─{gmain}(2178)
├─rpc.statd(2782)
├─rpcbind(2053)
├─rsyslogd(2071)─┬─{in:imklog}(2132)
│ ├─{in:imuxsock}(2131)
│ └─{rs:main Q:Reg}(2133)
├─rwhod(2449)───rwhod(2450)
├─screen(19504)───bash(19505)
├─screen(20288)───bash(20289)───vim(21881)
├─snapd(2056)─┬─{snapd}(2118)
│ ├─{snapd}(2119)
│ ├─{snapd}(2120)
│ ├─{snapd}(2121)
│ ├─{snapd}(2150)
│ ├─{snapd}(2151)
│ └─{snapd}(2152)
├─sshd(2341)─┬─sshd(3104)───sshd(3160)───bash(3161)───alpine(8255)
│ ├─sshd(4288)───sshd(4347)───bash(4348)───alpine(4354)
│ ├─sshd(15840)───sshd(15919)───bash(15926)
│ ├─sshd(40522)───sshd(40572)───bash(40588)
│ ├─sshd(41273)───sshd(41339)───bash(41346)───alpine(41362)
│ ├─sshd(46574)───sshd(46658)───bash(46665)───pstree(47648)
│ └─sshd(48106)───sshd(48199)───bash(48200)───ssh(48213)
├─systemd(2874)───(sd-pam)(2878)
├─systemd(10316)───(sd-pam)(10320)
├─systemd(15847)───(sd-pam)(15860)
├─systemd(48118)───(sd-pam)(48124)
├─systemd(46589)───(sd-pam)(46600)
├─systemd(41295)───(sd-pam)(41303)
├─systemd(27537)───(sd-pam)(27542)
├─systemd(17838)───(sd-pam)(17842)
├─systemd(14978)───(sd-pam)(14983)
├─systemd(20176)───(sd-pam)(20181)
├─systemd-journal(1164)
├─systemd-logind(2095)
├─systemd-udevd(1207)
└─ypbind(2476)─┬─{ypbind}(2477)
└─{ypbind}(2478)
- The part in red is my current login shell
- Its child process is running
pstree
- There are problems when you run this command in an ssh client ...
- running on a Windows machine
- The vertical lines do not appear properly
Class Exercise
Class Quiz