IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 18

Today's Topics

Homework 9

I have posted Homework 9 here.

As usual, it will be due next Sunday at 11:59 PM.

Quiz 6

I have posted the answers to Quiz 6 here.

Review

Tips and Examples

Using Control Z with an Editor

fg is used to bring a background job into the foreground
But it will also bring a suspended job back to life
This allows you to use Control Z to switch in and out of a text editor ...
while working on a script
You launch an editor to work on a script
When you want to test your script you hit Control Z
This suspends the editor ...
and brings you back to the command line ...
where you can test your script
When you want to return to the editor ...
simply run fg
You can also use this trick with man ...
and info

A Strange Bug Involving ;

Review

Separating and Grouping Commands

You can enter many commands on a single command line ...

if you separate them with a semicolon, ;

$ echo Here are the contents of my home directory ; ls ; echo
Here are the contents of my home directory
bin   course_files  it116       it244       mail         submitted    tmp
code  html          it116_test  it244_test  public_html  tests_taken  xrchiv

When you hit Enter each command is executed ...
in the order it was typed at the command line

| (pipe) and & (ampersand) as Command Separators

The pipe, | , and ampersand, & , characters are also command separators
When we separate commands with the pipe character, | ...
each command takes its input from the previous command
Each command is a separate process
We use an ampersand, & , after a command ...
to run that command in the background
When we do this, two things happen
- The command is disconnected from the keyboard
- The command will run at the same time
  as the next command you enter
But the ampersand is also a command separator

So we can use it to run many commands at the same time

$ ./bother.sh > /dev/null & ./bother.sh  > /dev/null & ./bother.sh  > /dev/null & jobs
[1] 1794
[2] 1795
[3] 1796
[1]   Running                 ./bother.sh > /dev/null &
[2]-  Running                 ./bother.sh > /dev/null &
[3]+  Running                 ./bother.sh > /dev/null &

Continuing a Command onto the Next Line

A command line entry can be as long as you like
If you reach the end of your session window while typing your command ...
your text will wrap to the next line
But sometimes it helps to break a command line into multiple lines
You can do this by typing a backslash, \

followed immediately by the Enter key

$ echo A man \
> A plan \
> A canal \
> Panama
A man A plan A canal Panama

The backslash turns off the special meaning of newline
Normally, when the shell sees a newline ...
is runs whatever you have entered at the command line
Backslash only escapes the character immediately following it
After hitting \ and newline ...
the shell responds with the greater than symbol, >
This is the secondary prompt
The shell is telling you it expects more input
The normal prompt is your primary prompt
You get the primary prompt when the shell is waiting for a command
You get the secondary prompt when the shell is waiting ...
for more of your command line

Using Parentheses, ( ) , to Run a Group of Commands in a Subshell

A group of commands within, a longer command line ...
can be given a subshell of their own ...
in which to run
You can do this by putting the commands within parentheses
```
( cd ~/bar ; tar-xvf - )
```
The shell creates a subshell and runs the commands in that subshell

The Directory Stack

Bash provides a way for you to move back to previous directories
The directory stack keeps tracks of each directory you enter ...
as long as you use commands that use the stack
The stack operates on the principle of last in, first out
You can go back to a previous directory ...
by removing the top directory from the stack ...
and going to that directory
You can keep doing this ...
until you get where you want to be
There are three commands that use the directory stack
- dirs
- pushd
- popd

`dirs` - Displays the Directory Stack

A stack is a LIFO data structure ...
which stands for Last In First Out
A stack allows you to go back in time to previous values ...
of some important variable
dirs displays the current contents of the directory stack
If the stack is empty, dirs displays the current directory
```
$pwd
~/it244/hw5 
   
$ dirs
~/it244/hw5
```

`pushd` - Pushes a Directory onto the Stack

In programming, putting something onto a stack is called a push
pushd changes your current directory ...
just like cd ...
but it also adds your new directory to the directory stack
When used with an argument, pushd
- Places the new directory on the stack
- Displays the current contents of the directory stack
- Moves to the new directory

`popd` - Pops a Directory off the Stack

In programming, removing a value from a stack is called a pop
popd changes your directory to another directory
But it also removes a directory from the stack
When used without an argument, popd
- Removes the top directory from the stack
- Prints the current stack
- Goes to the directory that is now on top of the stack

Shell Variables

A variable is a place in memory with a name ...
that holds a value
Shell variables are variables that can be used in the shell
To get the value of a variable ...

put a dollar sign, $, in front of its name

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Some variables are set and maintained by the shell itself
They are called keyword shell variables ...
or just keyword variables
Some of these variable are defined in /etc/profile
Many of these keyword variables can be changed by the user
Other variables are created by the user
They are called are called user-created variables
The environment in which a variable can be used is called the scope
Shell variables have two scopes
- Local
- Global

Local Variables

Local variables only exist in the shell in which they are defined
To create a local variable, use the following format
```
VARIABLE_NAME=VALUE
```
There are no spaces on either side of the equal sign ...
when setting Bash variables
Here is an example
```
$ city=Boston

$ echo $city
Boston
```
Variables are local unless you explicitly make them global
If the value assigned to a variable has spaces or tabs ...

you must quote it

$ hello="Hello there"

$ echo $hello
Hello there

If you run a shell script, the local variables will not be visible ...
because the script is running in a subshell ...
and the local variables are not defined there

Global Variables

Global variables are defined in one shell ...
and keep their values in all subshells created from that shell

Global variables are defined in Bash using the export command

$ echo $foo
FOO

$ export foo=BLETCH

$ echo $foo
BLETCH

$ ./print_foo.sh
foo = BLETCH

Usually, global variables are declared in a startup file ...
like .bash_profile
The env command, when used without an argument ...

displays the values of all global variables

$ env
TERM=xterm-color
SHELL=/bin/bash
SSH_CLIENT=66.92.76.9 53785 22
OLDPWD=/home/it244gh
SSH_TTY=/dev/pts/8
USER=it244gh
...

Keyword Shell Variables

Keyword shell variables have special meaning to the shell
They have short, mnemonic names
By convention, the names of keyword variables are always capitalized
Most keyword variables can be changed by the user
This is normally done in the startup file .bash_profile

Important Keyword Shell Variables

There are a number of keyword variables that affect your Unix session

Some of the more important are

Variable Value

HOME The absolute pathname of your home directory

PATH The list of directories the shell will search when looking for the executable file associated with a command you entered at the command line

SHELL The absolute pathname of your default shell

PS1 Your command line prompt - what you see after entering each command

PS2 The secondary prompt - what you see if you continue a command to a second line

Variable	Value
HOME	The absolute pathname of your home directory
PATH	The list of directories the shell will search when looking for the executable file associated with a command you entered at the command line
SHELL	The absolute pathname of your default shell
PS1	Your command line prompt - what you see after entering each command
PS2	The secondary prompt - what you see if you continue a command to a second line

User-created Variables

User-created variables are any variables you create
By convention, the names of user-created variables are lower case
```
$ foo=FOO

$ echo $foo
FOO
```
User-created variables can be either local or global in scope

New Material

Positional and Special Parameters

Positional and special parameters are variables set by Unix ...
that change each time you enter a command
We have already encountered the special parameter ?

It contains the status code returned by the last command

$ ls bar.txt 
bar.txt

$ echo $?
0

$ ls xxx
ls: cannot access xxx: No such file or directory

$ echo $?
2

Positional parameters are used by shell scripts ...
to get command line arguments
Positional parameters allow the shell script to get arguments from the command line
each word on the command line ...
is assigned to a positional parameter
The first word is the pathname of the script ...
and is assigned to positional parameter 0
Each succeeding word on the command line ...

is assigned to the next integer

$ cat print_positionals.sh
#!/bin/bash
#
# Prints the value of the first four positional arguments

echo
echo 0: $0
echo 1: $1
echo 2: $2
echo 3: $3

$ ./print_positionals.sh foo bar bletch
0: ./print_positionals.sh
1: foo
2: bar
3: bletch

Positional parameters are the usual way input is given to a script
Another special parameter is #
# contains the number of arguments ...

passed to a program from the command line

$ cat print_arg_numbers.sh 
#!/bin/bash
#
# Prints the number of arguments sent to this script

echo
echo This script received $# arguments

$ ./print_arg_numbers.sh foo bar bletch
This script received 3 arguments

Notice that # counts the arguments to the script ...
not the name of the script itself

Quoting and the Evaluation of Variables

Whenever the value of a variable contains spaces or tabs ...
you must quote the string ...
or escape the whitespace character
There are three ways this
- Single quotes, ' '
- Double quotes, " "
- Backslash, \
Single quotes are the most restrictive
Everything surrounded by single quotes ...
appears in the variable ...
exactly as you typed it
This means that special meaning of characters ...
like $ before a variable name ...

are ignored

$ team="Red Sox'

echo $team
Red Sox

$ cheer='Go $team'

$ echo $cheer
Go $team

Double quotes also preserve spaces and tabs ...
in the strings they contain
But you can use a $ in front of a variable name ...
to get the value of a variable ...

inside double quotes

       
$ cheer="Go $team"

$ echo $cheer
Go Red Sox

Quotes affect everything they enclose
The backslash, \, only effects the character immediately following it
```
$ foo=bar

$ echo $foo
bar

$ foo3=\$foo

$ echo $foo3
$foo
```

Removing a Variable's Value

There are two ways of removing the value of a variable

You can use the unset command

$ echo $foo
FOO

$ unset foo

$ echo $foo

$

Notice that the variable name was not preceded by a $
That's because we are dealing with the variable itself ...
not its value
The other way of removing a variable's value
is to set the value of the variable to the empty string
```
$ echo $foo
FOO

$ foo=

$ echo $foo

$
```

Variable Attributes

Variables can have attributes ...
such as being read only or global
We have already seen one way to set the attribute of a variable
If you precede the name of a variable with export ...
it makes the variable global
```
$ export foo=FOO
```
You can make a variable read only ...

by using the readonly command

$ echo $foo
FOO

$ readonly foo

$ foo=bar
-bash: foo: readonly variable

You must set the value of a variable ...
before you make it read only
Once you make a variable read only ...
it cannot be changed
There are many more variable attributes
They can be set using the declare command ...
which is a built-in

Here are some of the options declare

Option Meaning

a Declares a variable to be an array

f Declares a variable to be a function name

i Declares a variable to be an integer

r Makes a variable read only

x Makes a variable global

Option	Meaning
a	Declares a variable to be an array
f	Declares a variable to be a function name
i	Declares a variable to be an integer
r	Makes a variable read only
x	Makes a variable global

Let's look at some examples

$ foo=bar

$ echo $foo
bar

$ foo=bletch

$ echo $foo
bletch

$ declare -r foo

$ foo=bling
-bash: foo: readonly variable

The readonly command is also a built-tin ...
and is a synonym for declare -r
export is a built-in too ...
and it is a synonym for declare -x

Processes

A process is a running program
Every process has resources ...
that it needs to do its job
Unix is a multitasking operating system ...
so many processes can run at the same time
The shell runs in a process like any other command
Every time you run a program ...
except a built-in ...
a new process is created
Running a built-in command does not create a process ...
because the built-in is part of the shell ..
and the shell already has a process
When a shell script is run ...
your current shell creates a subshell to run the script
This subshell runs in a new process

System Calls

When you run a command that is not a built-in in ...
your shell asks the kernel to create a process ...
to run that command
How does the shell make this request?
It does it through a system call
Built-ins are procedures contained in the binary instructions for the shell ...
that you can run at the command line
In a similar way, the kernel contains procedures inside its binary instructions ...
that other programs can use
When a program needs to have the kernel do something for it ...
it runs one of these procedures inside the shell
The mechanism by which a program runs one of these procedures ...
is called a system call
Certain programming languages ...
like C ...
have features that allow you to make a system call ...
directly inside a program written in the language

How the Shell Runs a Program

To create a process to run a command that you have entered ...
your shell calls the kernel procedure fork()
This procedure makes new process ...
that is a duplicate of the process running your shell
The only difference between the two processes ...
is that the new process knows it was created ...
the original process
The original shell process is called the parent process
and the new process is called the child process
Every process has to have binary code to make it run
The new process has the same binary code as the shell ...
but it knows that it is a child process ...
and must run the program that was entered at the command line
It does this by making a system call to another kernel procedure ...
called exec()
This procedure replaces the binary code of the process ...
with the binary code of the command the user entered ...
and runs that code
When the code finishes running ...
it sends an to its parent process
The parent process was your shell
While the command you entered is running ...
you shell has nothing to do ...
so it makes a system call to the kernel procedure sleep
This causes your shell process to go into a state of suspended animation
The shell process still exists
It does not lose memory in RAM ...
but it is not using any process time
The shell process continues sleeping ...
until it gets the exit status from the child process ...
running the program you entered
Then the shell process wakes up again ...
and it ready for you next command

Process Structure

Since every process on the computer ...
was created by a parent process ...
how is the very first process created?
Whenever a Unix system is turned on ...
a special first process is created automatically
This special process then creates other process ...
which can create other process ...
and so on
This creates a hierarchical structure ...
much like the file system
In the hierarchical filesystem this is special directory at the top ...
called the root ..
and every other file or directory is contained in this directory ...
or one of its subdirectories
Until recently the special first process was init
But for various reasons this is no longer the case in Ubuntu ...
though it still may be true in other versions of Linux
The spontaneous process that is created whenever you power on an Ubuntu system ...
is now systemd

Booting a Unix Machine

Whenever you turn on a Unix machine ...
a spontaneous process is automatically created
The code for this process is either init ...
or systemd
and has process id 1
This process then creates all other processes ...
needed to run the services the machine provides
Unix can be run is a mode with only one user ...
but this is only done under special circumstances
When run in multiuser mode ...
Unix will create processes that allow users to log in
There are an a number of different programs that can be run in these process ...
such as
- getty
- mingetty
- agetty
Our systems run agetty
Each one of these processes is connected to a physical or virtual terminal
When the code detects an attempt to connect ...
it displays a login prompt ...
and waits for the user to respond
When that happens it runs login ...
which is located in the /bindirectory ...
to verify the password
If the password is accepted exec() is called ...
on the code for the default shell ...
and the process becomes your login shell

Process Identification

Each process has a unique Process ID (PID) number
As long as the process runs, it has the same PID
After a process terminates ...
its PID can be assigned to a new process
ps -f displays a full listing of information about each process running ...

for the user

$ ps -f
UID        PID  PPID  C STIME TTY          TIME CMD
it244gh  26374 26373  0 13:41 pts/5    00:00:00 -bash
it244gh  27891 26374  0 13:57 pts/5    00:00:00 ps -f

The UID column shows the Unix username of the account that started the process
The PID column is the process ID of the process
The PPID column is the process ID of the parent process ...
the process that created this process
The CMD column lists the command that is running in the process
If I were to run sleep in the background ...

and then run ps -f, I would see

$ sleep 10 & ps -f
[1] 27352
UID        PID  PPID  C STIME TTY          TIME CMD
ghoffman  27292 27287  0 15:12 pts/1    00:00:00 -bash
ghoffman  27352 27292  0 15:13 pts/1    00:00:00 sleep 10
ghoffman  27353 27292  0 15:13 pts/1    00:00:00 ps -f

Notice that the parent process of both sleep and ps -f ...
is my login shell
pstree will display a tree of all currently running processes

If I run pstree on users3 I can see the process structure

systemd─┬─accounts-daemon─┬─{gdbus}
        │                 └─{gmain}
        ├─acpid
        ├─agetty
        ├─atd
        ├─automount───10*[{automount}]
        ├─bash
        ├─bother.sh───sleep
        ├─cron
        ├─cups-browsed─┬─{gdbus}
        │              └─{gmain}
        ├─dbus-daemon
        ├─dhclient
        ├─irqbalance
        ├─2*[iscsid]
        ├─lvmetad
        ├─lxcfs───4*[{lxcfs}]
        ├─master─┬─pickup
        │        └─qmgr
        ├─mdadm
        ├─ntpd
        ├─polkitd─┬─{gdbus}
        │         └─{gmain}
        ├─rpc.statd
        ├─rpcbind
        ├─rsyslogd─┬─{in:imklog}
        │          ├─{in:imuxsock}
        │          └─{rs:main Q:Reg}
        ├─rwhod───rwhod
        ├─screen───bash
        ├─screen───bash───vim
        ├─snapd───7*[{snapd}]
        ├─sshd─┬─3*[sshd───sshd───bash───alpine]
        │      ├─2*[sshd───sshd───bash]
        │      ├─sshd───sshd───bash───pstree
        │      └─sshd───sshd───bash───ssh
        ├─10*[systemd───(sd-pam)]
        ├─systemd-journal
        ├─systemd-logind
        ├─systemd-udevd
        └─ypbind───2*[{ypbind}]

Where you see a number followed by a * it means there are multiple versions of that software running in different processes
You can see this more clearly if you run pstree with the -p option ...

which will show each process along with its process ID

systemd(1)─┬─accounts-daemon(2099)─┬─{gdbus}(2136)
           │                       └─{gmain}(2134)
           ├─acpid(2076)
           ├─agetty(2481)
           ├─atd(2103)
           ├─automount(42937)─┬─{automount}(42938)
           │                  ├─{automount}(42939)
           │                  ├─{automount}(42942)
           │                  ├─{automount}(42945)
           │                  ├─{automount}(42946)
           │                  ├─{automount}(42947)
           │                  ├─{automount}(42948)
           │                  ├─{automount}(42949)
           │                  ├─{automount}(42950)
           │                  └─{automount}(42951)
           ├─bash(27573)
           ├─bother.sh(16996)───sleep(47645)
           ├─cron(2087)
           ├─cups-browsed(40751)─┬─{gdbus}(40833)
           │                     └─{gmain}(40832)
           ├─dbus-daemon(2089)
           ├─dhclient(2246)
           ├─irqbalance(2465)
           ├─iscsid(2355)
           ├─iscsid(2356)
           ├─lvmetad(1199)
           ├─lxcfs(2101)─┬─{lxcfs}(2126)
           │             ├─{lxcfs}(2127)
           │             ├─{lxcfs}(3081)
           │             └─{lxcfs}(29814)
           ├─master(21825)─┬─pickup(44495)
           │               └─qmgr(31008)
           ├─mdadm(2148)
           ├─ntpd(2507)
           ├─polkitd(2164)─┬─{gdbus}(2180)
           │               └─{gmain}(2178)
           ├─rpc.statd(2782)
           ├─rpcbind(2053)
           ├─rsyslogd(2071)─┬─{in:imklog}(2132)
           │                ├─{in:imuxsock}(2131)
           │                └─{rs:main Q:Reg}(2133)
           ├─rwhod(2449)───rwhod(2450)
           ├─screen(19504)───bash(19505)
           ├─screen(20288)───bash(20289)───vim(21881)
           ├─snapd(2056)─┬─{snapd}(2118)
           │             ├─{snapd}(2119)
           │             ├─{snapd}(2120)
           │             ├─{snapd}(2121)
           │             ├─{snapd}(2150)
           │             ├─{snapd}(2151)
           │             └─{snapd}(2152)
           ├─sshd(2341)─┬─sshd(3104)───sshd(3160)───bash(3161)───alpine(8255)
           │            ├─sshd(4288)───sshd(4347)───bash(4348)───alpine(4354)
           │            ├─sshd(15840)───sshd(15919)───bash(15926)
           │            ├─sshd(40522)───sshd(40572)───bash(40588)
           │            ├─sshd(41273)───sshd(41339)───bash(41346)───alpine(41362)
           │            ├─sshd(46574)───sshd(46658)───bash(46665)───pstree(47648)
           │            └─sshd(48106)───sshd(48199)───bash(48200)───ssh(48213)
           ├─systemd(2874)───(sd-pam)(2878)
           ├─systemd(10316)───(sd-pam)(10320)
           ├─systemd(15847)───(sd-pam)(15860)
           ├─systemd(48118)───(sd-pam)(48124)
           ├─systemd(46589)───(sd-pam)(46600)
           ├─systemd(41295)───(sd-pam)(41303)
           ├─systemd(27537)───(sd-pam)(27542)
           ├─systemd(17838)───(sd-pam)(17842)
           ├─systemd(14978)───(sd-pam)(14983)
           ├─systemd(20176)───(sd-pam)(20181)
           ├─systemd-journal(1164)
           ├─systemd-logind(2095)
           ├─systemd-udevd(1207)
           └─ypbind(2476)─┬─{ypbind}(2477)
                          └─{ypbind}(2478)

The part in red is my current login shell ...
and its child process running pstreet
For some reason, the connecting lines in the output of this command ...
do not appear properly when running an ssh client on Windows