IT 116: Introduction to Scripting

IT 116: Introduction to Scripting
Class 21

Today's Topics

Homework 10

I have posted homework 10 here.

It is due this coming Sunday at 11:59 PM.

Quiz 7

You will find the answers to Quiz 7 here.

Let's review them.

Tips and Examples

Hashbang Problem with Windows Text Files

Some time ago while testing the Class Exercises for last week I came upon the following error
```
./ex20.py /usr/bin/python3^M bad interpreter: No such file or directory
```
You will get this error if you create the script on a Windows machine
The problem is that Unix and Windows have different ways of marking the end of a line
Unix uses the single newline character which in Python is written \n
Windows uses two characters to indicate the end of a line, a carriage return and a line feed
In Python this combination is written \r\n
It is the carriage return character that Unix prints as ^M above
Unix reads this extra carriage return character as part of the filename for the Python interpreter
There is no file with this filename, so Unix give you the error above
There is a simple fix for this problem
Unix provides the utility dos2unix which converts Windows text files into Unix text files
The command takes one argument, the file you are trying to convert
```
dos2unix FILENAME
```

Review

Errors

It is easy to get an error when writing scripts
But to fix errors it is important to recognize the different error types
There are three classes of errors
- Syntax errors
- Logic errors
- Runtime errors

Syntax Errors

Syntax errors occur when you write a statement that violates the rules of the programming language
```
>>> for = 5
  File "<stdin>", line 1
    for = 5
        ^
SyntaxError: invalid syntax
```
Python does not let you use a keyword as a variable name

A common type of syntax error is when you misspell the name of a function or variable

>>> name = "Glenn"
>>> print(nme)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'nme' is not defined

Another common syntax error is leaving out a )

$ cat get_number_1.py 
#! /usr/bin/python3

number = int(input("Number: ")

$ ./get_number_1.py 
  File "./get_number_1.py", line 6

                                  ^
SyntaxError: unexpected EOF while parsing

Indentation problems are also syntax errors

>>> for i in range(5):
...     num = i * 2
...   print(num)
  File "<stdin>", line 3
    print(num)
             ^
IndentationError: unindent does not match any outer indentation level

Syntax errors are the most common errors encountered when writing scripts

Logic Errors

A logic error occurs when the code does not give the correct results

Here is an example

average = num_1 + num_2 / 2
print ('Average:', average)

The numerator should be sum of both num_1 and num_2
But here it is only num_2
You get no error message with a logic error
The code works and unless you check the results you will not know there is a problem
This is why it is always important run tests on the scripts you write

Runtime Errors

Runtime errors occur only when the the program is run
They occur when a value makes a statement fail
If the program is run again with a different value no runtime error would occur

A common way to get a runtime error is to open a file that does not exist

$ ./file_open.py 
Filename: xxxxxx
Traceback (most recent call last):
  File "./file_open.py", line 6, in <module>
    file     = open(filename, "r")
FileNotFoundError: [Errno 2] No such file or directory: 'xxxxxx'

If you try to open a file for which you have no permission you will get another type of runtime error

$ ./file_open.py 
Filename: unreadable.txt
Traceback (most recent call last):
  File "./file_open.py", line 6, in <module>
    file     = open(filename, "r")
PermissionError: [Errno 13] Permission denied: 'unreadable.txt'

Giving a conversion function a value it can't handle is another runtime error

$ ./int_request.py 
Integer: five
Traceback (most recent call last):
  File "./int_request.py", line 5, in <module>
    number = int(input("Integer: "))
ValueError: invalid literal for int() with base 10: 'five'

Or you can give a function an argument with the wrong type

>>> round("five")
Traceback (most recent call last):
  File "<stdin>, line 1, in <module>
TypeError: type str doesn't define __round__ method

Or dividing by zero

$ ./divide_two_numbers.py 
Numerator: 5
Denominator: 0
Traceback (most recent call last):
  File "./divide_two_numbers.py", line 7, in <module>
    result      = numerator / denominator
ZeroDivisionError: division by zero

The problems that cause runtime errors are called exceptions

Exception Objects

Python has a mechanism that deals with runtime errors
It gathers information about what happened and where and uses it to create an error message
It does this by creating an exception object
When this happens we say that the interpreter has raised an exception

Catching Exceptions

The exception mechanism is more that just a way to create helpful error messages
Python allows you to see exception objects in your code while your script is running
Your code can then do something to prevent the script from aborting

You do this with a try/except statement which has the following form

try:
    STATEMENT
    STATEMENT
    ...
except:
    STATEMENT
    STATEMENT
    ...

The statements in the try code block execute normally unless a runtime error occurs
If a runtime error occurs, it stops executing the statements in the try block
Instead, it jumps to the except code block and executes those statements
When your code does this, it is said to catch an exception
Whenever you use open to create a file you should put it inside a try/except statement

Like this

$ cat open_file.py
#! /usr/bin/python3

# demonstrates using a try/except statement 
# to catch exceptions encountered while
# trying to open a file

filename = input("Filename: ")
try:
    file = open(filename, "r")
    for line in file:
        print(line.rstrip())
except:
    print("Could not open file", filename)

$ ./open_file.py
Filename: xxxx
Could not open file xxxx

Notice that the interpreter never ran the for loop

Exceptions and Data Validation

Whenever we get data from the user we need to check it
This is called data validation
We can use input to ask the user for a number
But input returns a string so we need to use a conversion function
But int and float can cause a runtime error
So we need to call them inside a try/except statement
We should put this inside a while loop
That way the it will keep running until we get valid input

Here is a function that does that

def get_integer():
    done = False
    while not done:
        number = input("Integer: ")
        try:
            number = int(number)
            done = True
        except:
            print(number, 'cannot be converted into an integer')
    return number

Improving the Validation Function

The above function works
But we can make it shorter
Taking working code and making it better in some way is called refactoring
We don't have to use a flag in the function header of the while loop
We can use the boolean literal True instead
Then we can use a return statement to jump out of the loop when a value is OK

Here is the refactored code

def get_integer():
    while True:
        number = input("Integer: ")
        try:
            number = int(number)
            return number
        except:
            print(number, 'cannot be converted into an integer')

New Material

The Code in the `try` Clause

In the scripts that processed temperature data the filename was written into the code
```
file = open('temps.txt', 'r')
```
A more realistic script would ask the user for the filename

But if the user enters the wrong filename you will get a runtime error

$ ./temps_average_2.py 
Filename: xxx
Traceback (most recent call last):
  File "./temps_average_2.py", line 7, in <module>
    file = open(filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'xxx'

To fix this, we might be tempted to put the call to open in the try block

filename = input("Filename: ")
try:
    file = open(filename, 'r')
except:
    print('Cannot open', filename)

count = 0
total = 0
for line in file:
    count += 1
    date, temp = line.split()
    temp   = int(temp)
    total += temp
    
average = round(total/count, 2)
print('Average:', average)

This fixes one problem but creates another

$ ./temps_average_3.py 
Filename: xxx
Cannot open xxx
Traceback (most recent call last):
  File "./temps_average_3.py", line 15, in <module>
    for line in file:
NameError: name 'file' is not defined

We depend on the open statement working for the rest of the script to work

To fix this, we can put every statement that needs the file object inside the try block

filename = input("Filename: ")
try:
    file = open(filename, 'r')
    count = 0
    total = 0
    for line in file:
        count += 1
        date, temp = line.split()
        temp   = int(temp)
        total += temp

    average = round(total/count, 2)
    print('Average:', average)
except:
    print('Cannot open', filename)

This version better handles a bad filename

$ ./temps_average_4.py
Filename: xxx
Cannot open xxx

Different Exception Types

When a runtime error occurs the Python interpreter creates an exception object
When this happens the interpreter is said to raise an exception
The exception mechanism has three purposes
- To allow the interpreter to handle the runtime error
- To give the programmer some idea of what went wrong
- To give the programmer the chance to do something about it
There is more than one way to cause a runtime error
So different exception objects are created for different types of exceptions

Common Exception Types

Let's look at some of the more common types of exception objects

When you try to open a file that does not exists you get a FileNotFoundError exception object

>>> open("xxx.txt","r")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'xxx.txt'

If you try to open a file for which you do not have permission you will get a PermissionError object

file = open("test.txt","r")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [Errno 13] Permission denied: 'test.txt'

If a conversion function gets the wrong type of argument a ValueError object is created

>>> number = float(input("Number: "))
Number: five
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: 'five'
>>> number = float(input("Number: "))
Number: 5..4
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not convert string to float: '5..4'

If you import a module that doesn't exist you get an ImportError object

>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'foo'

If you try to divide a number by zero the interpreter will create a ZeroDivisionError exception object

>>> print(5/0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

Handling More Than One Type of Exception

Here is a function which which asks for a filename and returns a file object

def open_file_reading():
    while True:
        filename = input("Filename: ")
        try:
            file = open(filename, "r")
            return file
        except:
            print("Cannot open", filename)

This function keep looping until I give it a good filename

$ ./file_open_2.py
Filename: xxx
Cannot open xxx
Filename:

But there are two runtime errors that this function could encounter
- File not found
- Permission error
It would be nice if the function could tell the difference
It could then give a more specific error message
I have shown you the simplest version of the try/except statement
```
try:
    STATEMENT
    ...
except:
    STATEMENT
    ...
```
But there is another form
There can be different except clauses for each exception object

The format is

try:
    STATEMENT
    ...
except EXCEPTION_TYPE_1:
    STATEMENT
    ...
except EXCEPTION_TYPE_2:
    STATEMENT
    ...
...

You can catch as many exception types as you want

We can use this format to write an improved version of the function

def open_file_reading():
    while True:
        filename = input("Filename: ")
        try:
            file = open(filename, "r")
            return file
        except FileNotFoundError:
            print("Cannot find", filename)
        except PermissionError:
            print("You do not have permission to read", filename)

Now we get different error messages for different exceptions

$ ./file_open_3.py 
Filename: xxx
Cannot find xxx
Filename: unreadable.txt
You do not have permission to read unreadable.txt
Filename:

Using the Data in Exception Objects

Look at the last error message above

You do not have permission to read unreadable.txt

This error message was written inside the function

But if we had not caught the exception we would have seen the following

>>> open('xxx', 'r')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'xxx'

All the information in the error message is contained in the exception object
We can get this using a slightly different form of the except header
In this variant we create a variable that points to the exception object
```
except EXCEPTION_TYPE as VARIABLE:
```
A commonly used variable name for the exception is err
This is only a convention
You can use any valid variable name to reference the exception object
Once you have the variable you can use it to access any of the data inside the object
If you print the variable it will give you the error message

Here is a new version of the function that uses this feature

def open_file_reading():
    while True:
        filename = input("Filename: ")
        try:
            file = open(filename, "r")
            return file
        except FileNotFoundError as err:
            print(err)
        except PermissionError as err:
            print(err)

Here is what happens when we use it

$ ./file_open_4.py
Filename: xxx
[Errno 2] No such file or directory: 'xxx'
Filename: unreadable.txt
[Errno 13] Permission denied: 'unreadable.txt'
Filename:

The statements inside an except block are called an exception handler

The `else` Clause

A try/except statement can also have an else clause
The statements in this block will be run if try block code does not cause a runtime error

As I said above, the following code will cause an error if the open command fails

filename = input("Filename: ")
try:
    file = open(filename, 'r')
except:
    print('Cannot open', filename)

count = 0
total = 0
for line in file:
    count += 1
    date, temp = line.split()
    temp   = int(temp)
    total += temp

If no file object is created, the for loop cannot work
I suggested putting the loop and the two statement above it in the try code block

Like this

try:
    file = open(filename, 'r')
    count = 0
    total = 0
    for line in file:
        count += 1
        date, temp = line.split()
        temp   = int(temp)
        total += temp

        average = round(total/count, 2)
        print('Average:', average)
except:
    print('Cannot open', filename)

But a better way to handle this problem is to put the code in an else clause

try:
    file = open(filename, 'r')
except:
    print('Cannot open', filename)
else:
    count = 0
    total = 0
    for line in file:
        count += 1
        date, temp = line.split()
        temp   = int(temp)
        total += temp

    average = round(total/count, 2)
    print('Average:', average)

Why is it better?
The try code block is meant to include code that could cause a runtime error
We could put other code here, but that is not the purpose of the try clause
So reading from the top of the script we first encounter a short try clause
Then a short except clause
With this out of the way we can get to the meat of the script in the else clause

The `finally` Clause

There is another type of clause that can occur a try/except statement
It has the format
```
finally:
    STATEMENT
    ....
```
The statements in a finally clause will always be executed
It does not matter whether a runtime error occurs
We will not have to use this feature of Python
But in more advanced scripts it can be useful
The finally clause allows you to clean up any lose ends no matter what happens when the script is run

Talking About Ethics and Technology

You can use this class as a GED Natural Science/Math/Technology course
You need three such courses to graduate
The rules which decide whether a class can be used to fulfill this requirement have changed
They now require that technology courses address ethical issues
Part of each class from now on will deal with this subject
I encourage you to think about these issues
And discuss them in class

The Spiderman Principle

Many tools can be used for either good or evil
A hammer can pound a nail
It can also be used to batter a head
A knife can cut food into bite sized pieces
Or it can be used to stab and kill
The same is true of many skills
Doctors know medicines that can heal and extend life
But some, if given in the wrong dose, can kill
A baseball pitcher can through a ball at 90 miles per hour
If thrown at a head it can strike a person dead
In general the more powerful the skill, the more important it is that it be used for good
This is why Peter Parker learned the Spiderman Principle
With great power comes great responsibility

Professional Responsibility

When we hire a plumber we expect him to fix the problem, not make it worse
When we go to a doctor we expect him or her to help us
We don't expect the doctor to call for unnecessary tests to pad the bill
Or to do us harm
We live in a world great complexity
We depend on many services
- Doctors
- Dentists
- Pharmacists
- Accountants
And on many technologies
- Cars
- Plumbing
- Heating Systems
- Computers
All of these rely on highly trained professionals
Like all of you hope to become some day
We trust these people because we believe they have a sense of what they should and shouldn't do
That they have a sense of professional responsibility

Codes of Conduct

Professional organizations know the success of their members depends on their reputations
To increase public confidence they create create codes of ethics or of professional conduct
Perhaps the first example of this was the Hippocratic Oath taken by doctors in ancient Greece
It is the earliest statement of its kind in the Western world
It established many principles of medical ethics that are still followed today

It reads in part

I swear by Apollo Physician ... and by all the gods and goddesses 
...
I will use treatment to help the sick according to my ability and judgment, 
but never with a view to injury and wrong-doing.

These days most countries have medical societies whose members must follow a code of conduct
These codes extend the oath substantially and are updated regularly
In this country the code does not have the force of law
But violations of the code can be used in court in malpractice suits

Responsibility of IT Professionals

Unlike doctors, IT professionals do not have to be licensed
But that does not mean that we do not have professional responsibilities
In this course you will learn skills that most people do not have
This will give you access to information
And in today's world information is power
Recent events have shown that abuse of information can sway elections
Or lead Britain to the economic self-destruction known as Brexit
People with IT skills steal databases with millions of user records
Others use various scams to steal identities and commit fraud
IT people do not have to swear an oath
Yet some people with IT skills have done more damage that any doctor every has

The ACM Code of Ethics and Professional Conduct

Unlike doctors there is no professional organization that sets standards for people in IT
But the Association for Computing Machinery, ACM, has a Code of Ethics and Professional Conduct which you will find here
The code list 7 general ethical principles
1. Contribute to society and to human well-being, acknowledging that all people are stakeholders in computing
2. Avoid harm
3. Be honest and trustworthy
4. Be fair and take action not to discriminate
5. Respect the work required to produce new ideas, inventions, creative works, and computing artifacts
6. Respect privacy
7. Honor confidentiality
In the coming classes we will be discussing these principles