IT 116: Introduction to Scripting

IT 116: Introduction to Scripting
Class 20

Today's Topics

Review

Running a Script Using python3
Running a Script Without python3
Why Make a Script Executable?
Records
Reading Records from a File
Finding the Average From a File of Records
Closing a File Object
Looping Through a File More Than Once

Reading Assignment

You should read Chapter 7, Lists and Tuples, from our textbook, Starting Out with Python, before next week's class.

Homework 9

I have posted homework 9 here.

It is due this coming Sunday at 11:59 PM.

You must make the script for this assignment executable or you will loose points

Review

Running a Script Using `python3`

To write a program in the computer language C you create a text file
This file contains C statements and called the source code
You cannot run this file
In order to create a working program this file must be run through a compiler
The compiler creates a binary file
The binary file consists of instructions written in machine language
Machine language is binary code the CPU understands
The CPU does not understand C or Python
Our Python scripts are text files written in Python programming language
The computer does not understand this text so we cannot run Python scripts directly
Only the Python interpreter understands Python scripts
The Python interpreter was written in a language like C and compiled into an executable file
When we type python3 at the command line we are running this binary file
To run the script we give this command the filename of the Python script
python3 then executes all the Python statements in the file

So if I have the Python script hello_1.py

# prints a friendly message

print("Hello world!")

I can run it on Unix command line like this
```
$ python3 hello_1.py 
Hello world!
```

Running a Script Without `python3`

A script that is made executable can be run without first typing python3
To make a file executable we need to do two things
- Give it read and execute permissions
- Add the hashbang line
We can set the permission using the Unix chmod command
```
chmod  755  FILENAME
```
The hashbang line must be the very first line in the script
The first two characters must be #!
This is followed by the absolute pathname of the Python interpreter
The hashbang line on our Unix machines is
```
#! /usr/bin/python3
```

Why Make a Script Executable?

We make script executable so we can rum them from any directory
You do this by putting the script in a directory listed in your Unix PATH variable
When you type the name of your script, Unix searches the directories listed in PATH
If it find your script in a directory listed in PATH it will run it

Records

For any sporting event involving teams we have at least 5 pieces of information
- Date
- Home team
- Opposing team
- Home team score
- Opposing team score
The collection of data about a specific event is called a record
The individual pieces of data within a record are called fields
A text file containing data usually has one record per line

Here is a file with two fields

Date
Temperature

2017-06-01 67
2017-06-02 71
2017-06-03 69
...

Reading Records from a File

The characters used to separate one field from the next are called delimiters
The two most common delimiters are space and comma
We can break up a line into fields using the string method split and multiple assignment
```
>>> line = '2017-06-01 67'
date, temp = line.split()
```

Finding the Average From a File of Records

We can use split to compute the average temperature from a file that looks like this
```
2017-06-01 67
2017-06-02 71
2017-06-03 69
2017-06-04 88
2017-06-05 74
...
```

Here is the relevant part of the code

count = 0
total = 0
for line in file:
    count += 1
    date, temp = line.split()
    temp   = int(temp)
    total += temp
 average = roung(total/count)

Notice that I had to convert temp from a string into an integer

Closing a File Object

Creating a file object on a file limits what other program can do with that file
That's because the operating system want to prevent bad things from happening
If you have a file open for reading, other script can do that too
Reading a file does not change it, so many programs can read the file at the same time
But if you have a file open for reading no one else can opening it for writing
You don't want some other program changing a file while you are reading it
Similarly if you open a file for writing, no one else can open it for reading
It is good practice to dispose of a file object when you no longer need it
That makes the file available to others
You get rid of a file object by using the close method on the variable that points to the object
```
file.close()
```
The interpreter will close the file object for you when the script ends
But it is good practice for your script to close a file object when you are done with it

Looping Through a File More Than Once

There are some things that you cannot do by looping through a file once
For example, if we wanted to count the number of days with above average temperature
You have to loop through the file once to get the average
Then loop through it again to count the days above that average
When we loop through a text file we can only go in one direction
Once we have reached the end of a file, there is no way to rewind to the beginning
Instead we have to create a new file object
Before we do that, it is good practice to close the first file object we created

New Material

More Data Processing in a Loop

In the last class we found the maximum and minimum temperature from a file that looked like this
```
2017-06-01 67
2017-06-02 71
2017-06-03 69
...
```
But we can do more while we are looping through the file
We can also get the dates when the maximum and minimum occurred

We use an approach similar to the one we used for the maximum

set a variable to a value that will be replaced
for each value
    if the value is greater than the variable
        give the variable the new value

We will need 2 new variables
```
max_date = ''
min_date = ''
```
Why did we initialize the variables with the empty string?
Because we do not have to compare the date on a line with the values of these variables

We change these values when either the maximum or minimum value is replaced

if temp > max:
    max      = temp
    max_date = date
if temp < min:
    min      = temp
    min_date = date

Here is the code with the new lines in red

#! /usr/bin/python3

file = open('temps.txt', 'r')

max      = -100
min      = 200
max_date = ''
min_date = ''
for line in file:
    date, temp = line.split()
    temp  = int(temp)
    if temp > max:
        max      = temp
        max_date = date
    if temp < min:
        min      = temp
        min_date = date

print('Maximum:', max_date, max)
print('Minimum:', min_date, min)

When we run it we get

$ ./temps_max_min_dates.py
Maximum: 2017-06-24 89
Minimum: 2017-06-26 66

Errors

It is easy to get an error when writing scripts
But to fix errors it is important to recognize the different types
There are three classes of errors
- Syntax errors
- Logic errors
- Runtime errors

Syntax Errors

Syntax errors occur when you write a statement that violates the rules of the programming language
```
>>> for = 5
  File "<stdin>", line 1
    for = 5
        ^
SyntaxError: invalid syntax
```
Python does not let you use a keyword as a variable name
So this is not a legal Python statement

A common type of syntax error is when you misspell the name of a function or variable

>>> name = "Glenn"
>>> print(nme)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'nme' is not defined

This is also a syntax error

Another common syntax error is leaving out a )

$ cat get_number_1.py 
#! /usr/bin/python3

number = int(input("Number: ")

$ ./get_number_1.py 
  File "./get_number_1.py", line 6
    
                                  ^
SyntaxError: unexpected EOF while parsing

Indentation problems are also syntax errors

>>> for i in range(5):
...     num = i * 2
...   print(num)
  File "<stdin>", line 3
    print(num)
             ^
IndentationError: unindent does not match any outer indentation level

Syntax errors are the most common errors encountered when writing scripts
But they are the easiest to fix
All you have to do is change some characters
As you write more code, you will find it easier to spot and fix syntax errors

Logic Errors

A logic error occurs when the code does not give the correct results

Here is an example

average = num_1 + num_2 / 2
print ('Average:', average)

Can you spot the problem here?
When you have a syntax error the Python interpreter will tell you
The error message it gives specifies the kind of error and prints the line
But you get no warning with a logic error
The code works and unless you check the results you will not know there is a problem
This is why it is always important run tests on the scripts you write
Even if you know there is an error it can be hard to find
The problem could be anywhere in the code
It is hard to spot your own errors
That is why smart people ask others to proof read what they write

Runtime Errors

Runtime errors occur only when the the program is run
They occur when a value makes that statement fail
If the program is run again with a different value no runtime error would occur

A common way to get a runtime error is to open a file that does not exist

$ ./file_open.py 
Filename: xxxxxx
Traceback (most recent call last):
  File "./file_open.py", line 6, in <module>
    file     = open(filename, "r")
FileNotFoundError: [Errno 2] No such file or directory: 'xxxxxx'

If you try to open a file for which you have no permission you will get another type of runtime error

$ ./file_open.py 
Filename: unreadable.txt
Traceback (most recent call last):
  File "./file_open.py", line 6, in <module>
    file     = open(filename, "r")
PermissionError: [Errno 13] Permission denied: 'unreadable.txt'

Giving a conversion function a value it can't handle is another runtime error

$ ./int_request.py 
Integer: five
Traceback (most recent call last):
  File "./int_request.py", line 5, in <module>
    number = int(input("Integer: "))
ValueError: invalid literal for int() with base 10: 'five'

Or you can give a function an argument with the wrong type

>>> round("five")
Traceback (most recent call last):
  File "<stdin>, line 1, in <module>
TypeError: type str doesn't define __round__ method

Or dividing by zero

$ ./divide_two_numbers.py 
Numerator: 5
Denominator: 0
Traceback (most recent call last):
  File "./divide_two_numbers.py", line 7, in <module>
    result      = numerator / denominator
ZeroDivisionError: division by zero

The problems that cause runtime errors are called exceptions

Exceptions Objects

Errors like the ones above are frustrating
But at least the error message gives you some idea what went wrong
Decades ago my CS classes required me to write programs in the computer language C
In C runtime error would produce the following output
```
Segmentation fault: Core dumped
```
Thanks, that was really helpful
Many computer languages have a better way of dealing with this problem
The have a mechanism built into the language that deals with runtime errors
It gathers information about what happened and where
And uses that information to create an error message
The error message describes the problem and where it occurred
It does this by creating an exception object
When this happens we say that the interpreter has raised an exception

Catching Exceptions

But the exception mechanism is more that just a way to create helpful error messages
Python allows you to see exception objects in your code while your script is running
Your code can then do something to prevent the script from aborting
And sending you back to the command line

You do this with a try/except statement which has the following form

try:
    STATEMENT
    STATEMENT
    ...
except:
    STATEMENT
    STATEMENT
    ...

The statements in the try code block execute normally unless a runtime error occurs
If a runtime error occurs, it stops executing the statements in the try block
Instead, it jumps to the except code block and executes those statements
When your code does this, it is said to catch an exception
Whenever you use open to create a file you should put it inside a try/except statement

Here is an example

$ cat open_file.py
#! /usr/bin/python3

# demonstrates using a try/except statement 
# to catch exceptions encountered while
# trying to open a file

filename = input("Filename: ")
try:
    file = open(filename, "r")
    for line in file:
        print(line.rstrip())
except:
    print("Could not open file", filename)

$ ./open_file.py
Filename: xxxx
Could not open file xxxx

Notice that the interpreter never ran the for loop
When the exception occurred it stopped running the try code

Exceptions and Data Validation

We can use the input function to ask the user for an integer

But if the user enters a decimal we will get an exception

>>>  number = int(input("Integer: "))
Integer: 5.0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '5.0'

Whenever we get data from the user we need to check it
This is called data validation and it is best done with a function that uses a while loop

A few weeks ago I showed you an algorithm to validate data

ask the user for a value
while the value is not valid:
    print an error message
    ask the user for a value
return the value

We need a different algorithm here
I can't ask for input outside the loop and convert it
Doing so might cause runtime error
The conversion function must be run inside a try/except statement
And the try/except statement needs to be inside the loop
But what boolean expression should the while loop use?
One way to deal with this is to set a flag to False
The while loop will keep running as long as the flag is False
If the conversion does not cause an exception the flag is set to True
This will exit the loop and we can return the value

Here is the algorithm

set a flag to false
while the flag is not true
    ask the user  for a value
    try:
        convert the value to an integer
        set the flag to true
    except:
        print an error message
return the value

Here is a function that implements this algorithm

def get_integer():
    done = False
    while not done:
        number = input("Integer: ")
        try:
            number = int(number)
            done = True
        except:
            print(number, 'cannot be converted into an integer')
    return number

When we run the code we get

$ ./get_integer.py
Integer: 2.0
2.0 cannot be converted into an integer
Integer: 2
2

Improving the Validation Function

The above function works
But we can make it shorter
And shorter code is usually better code
Why?
Because every line you write could cause an error
Taking working code and making it better in some way is called refactoring
The shorter your code, the less chance there is for a problem
We can use a trick to make sure the while loop runs when we don't yet have a value
We can write
```
while True
```
But how do we ever get out of the loop?
Instead giving a flag a different value, we use a return statement
If the conversion does not cause a problem the return statement will be run
And a return statement always causes the function to stop
That's how we exit the loop

Here is the refactored code

def get_integer():
    while True:
        number = input("Integer: ")
        try:
            number = int(number)
            return number
        except:
            print(number, 'cannot be converted into an integer')