IT 116: Introduction to Scripting

IT 116: Introduction to Scripting
Class 24

Today's Topics

Homework 11

I have posted homework 11 here.

This is the last homework assignment.

Tips and Examples

Don't Use Indexes to Print a List

The simplest way to print a list is to use a for loop

>>> for team in teams:
...     print(team)
... 
Red Sox
Orioles
Blue Jays
Rays
Yankees

Of course you could also do this using indexes

>>> for index in range(len(teams)):
...     print(teams[index])
... 
Red Sox
Orioles
Blue Jays
Rays
Yankees

But that would be stupid
It's much more work
Which means a greater chance of making a mistake
You should only use indexing in a for loop when you want to change the elements in a list

Review

Empty Lists

We create an empty list using a list literal with nothing between the square brackets
```
>>> new_list = []
>>> new_list
[]
```
The empty list serves the same purpose for lists that zero does for numbers

Slices

You can use one list to create another
A slice is new, smaller list created from another list

To create a slice you use the following format

LIST_VARIABLE[FIRST_INDEX:ONE_MORE_THAN_LAST_INDEX]

The trick to using list is remembering how to get the second index
The second index is not the index of the last element
It's the index of the element after the last element you want

If I had a list of day names

>>> days = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']

I could get a list of weekday names using a slice

>>> weekdays = days[1:6]
>>> weekdays
['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday']

Leaving Out Indexes in a Slice

Python allows you to create a slice using only one index

If you leave out the first index, Python assumes the value is 0

>>> days
['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']
>>> days[:4]
['Sunday', 'Monday', 'Tuesday', 'Wednesday']
>>> digits
[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
>>> digits[:5]
[1, 2, 3, 4, 5]

If you leave out the second index Python uses the length of the list

So the the slice will end with the last element

>>> days[4:]
['Thursday', 'Friday', 'Saturday']
>>> digits[5:]
[6, 7, 8, 9, 0]

You can even leave out both indexes as long as you keep the :

In this case Python will return a slice that consists of the entire list

>>> days[:]
['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']
>>> digits[:]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]

This is a great way to make a copy of a list

>>> daynames = days[:]
>>> daynames
['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']

Skipping Elements in a Slice

You can also skip elements when creating a slice
It works just like the 3rd argument to the range function

To use this feature you add a second : followed by a number

LIST_VARIABLE[FIRST_INDEX:ONE_MORE_THAN_LAST_INDEX:STEP_VALUE]

The STEP_VALUE works just like the range function
It is what we add to the index of the previous element to get the next one
It is the difference between the index of any two elements in the original list

We can use this feature to create a slice of all the even digits

>>> digits = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
>>> digits[1:10:2]
[2, 4, 6, 8, 0]

Actually, I could leave out the second index
```
digits[1::2]
[2, 4, 6, 8, 0]
```

The `in` Operator

Operators are special characters that perform some action
The only operator we have seen so far have been symbols
- +
- -
- *
- /
- //
- **
- %
But operators can also be words
in is an operator that works on objects that are a collection of values
We use it like this
```
VALUE in LIST
```
The in operator returns True if the LIST contains VALUE

Otherwise it returns False

>>> digits
[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
>>> 1 in digits
True
>>> 11 in digits
False

List Methods

Lists are objects
Objects usually have functions that work on the values inside the object
These functions are called methods

Here are some of the more useful list methods

Method	Description
append(item)	Adds item to the end of the list
pop(index)	Removes the element at a given index from the list and returns the value. If called with no argument, it returns the last element and deletes it from the list.
sort()	Sorts the items in the list so they appear in ascending order (from the lowest value to the highest value)
reverse()	Reverses the order of the items in the list

The append Method

The append method adds a value to the end of a list
It takes one argument, the value you want to add
Is it probably the most frequently used list method
You can use the append method to create a list from scratch
First you create an empty list
```
>>> teams = []
>>> teams
[]
```
Then we can call the append method on this list using dot notation
```
>>> teams.append('Red Sox')
>>> teams
['Red Sox']
```

We can call this method as many times as we like

>>> teams.append('Orioles')
>>> teams.append('Blue Jays')
>>> teams.append('Rays')
>>> teams.append('Yankees')

Each new value is added to the end of the list

>>> teams
['Red Sox', 'Orioles', 'Blue Jays', 'Rays', 'Yankees']

The pop Method

The pop method does two things
- Removes an element from a list
- Returns the value of that element

It's argument is the index of the value to be removed

>>> numbs = [1, 2, 3, 4, 5]
>>> n = numbs.pop(0)
>>> n
1
>>> numbs
[2, 3, 4, 5]

If you leave out the index argument, pop removes the last element
```
>>> n = numbs.pop()
>>> n
5
>>> numbs
[2, 3, 4]
```

The sort Method

A list is mutable so we can add elements or remove them
But we can also change the order of the elements

This can be done with the sort method which takes no argument

>>> l2 = [9, 1, 0, 2, 8, 6, 7, 4, 5, 3]
>>> l2
[9, 1, 0, 2, 8, 6, 7, 4, 5, 3]
>>>l2.sort()
>>> l2
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

The reverse Method

The reverse method reverses the order of a list

>>> l3 = [5, 4, 3, 2, 1]
>>> l3.reverse()
>>> l3
[1, 2, 3, 4, 5]
>>> l3.reverse()
>>> l3
[5, 4, 3, 2, 1]

If you have a list of unsorted numbers and you want to sort them in descending order you first need to use sort
```
>>> l4 = [2, 5, 9, 1, 8, 6, 3, 7, 4]
>>> l4.sort()
>>> l4
[1, 2, 3, 4, 5, 6, 7, 8, 9]
```

Then use reverse

>>> l4.reverse()
>>> l4
[9, 8, 7, 6, 5, 4, 3, 2, 1]

The `del` Statement

List methods aren't the only things that work on lists
You can delete elements from a list using the del statement
The format of a del statement looks like this
```
del LIST_VARIABLE[INDEX]
```
After the keyword del comes the list variable and the index inside square brackets

>>> l5 = [1, 2, 3, 4, 5, 6]
>>> del l5[2]
>>> l5
[1, 2, 4, 5, 6]

The min and max Functions

Two built-in functions can also be used with lists
They do not change the list but they return information about it
min returns the smallest value in a list
```
>>> l6
[1, 4, 5, 6]
>>> min(l6)
1
```
max returns the largest
```
max(l6)
6
```

New Material

Time for Computer Operations

Computers are very fast
Much faster than we are
But different computer operations take different amounts of time
There are three time scales for accessing data
- Getting data from a user
- Getting data from a file
- Getting data from memory
Computers are fast and we are slow
Think about using a program like Word
Even if you were the worlds fastest typist you would be glacially slow to a computer
This is why you want to minimize it
Getting data from a disk is better
It is much faster than waiting for characters from the keyboard
But there is still a time penalty
With a hard disk you have to wait for the sector to spin under the head
A file has many sectors so this can add up
Accessing data from memory is the fastest
There is no human typist or mechanical device to slow things down

Reading a File into a List

When we read in numbers from a file to find the average we used a loop like this

file = open('numbs.txt', 'r')
total = 0
count = 0
for line in file:
    number = int(line)
    total += number
    count += 1

average = round(total/count)

To count the numbers in the file above average, we needed to loop through the file a 2nd time

file.close()
file = open('numbs.txt', 'r')
above_average = 0
for line in file:
    number = int(line)
    if number > average:
        above_average += 1

We had to loop through the file twice
And create a new file object each time
This is very inefficient
And it is slow as I explained above
It would be much better to read the file into a list
Once this is done all work would be done on the list
The list lives in memory so data access is quick

The easiest way to create a list from a file is

create an empty list
for each line in the file:
    append the line to the empty list

Using this algorithm we can write the following

file = open('numbs.txt', 'r')
numbers = []
for line in file:
    numbers.append(int(line))

total = 0
for num in numbers:
    total += num

average = round(total/len(numbers))

above_average = 0
for num in numbers:
    if num > average:
        above_average += 1

print(above_average)

Notice that I did not have to count the number of entries to compute the average
I could use the length of the list instead
Actually this code is not efficient
It uses 3 for loops
1. Read in the file to create the list
2. Loop through the list to create the total
3. Loop through the list to get the entries above average

We can combined the first two loops into a single for loop

file = open('numbs.txt', 'r')
total = 0
numbers = []
for line in file:
    num = int(line)
    numbers.append(num)
    total += num

average = round(total/len(numbers))

above_average = 0
for num in numbers:
    if num > average:
        above_average += 1

print(above_average)

Advantages of Using Lists with Files

Reading a file into a list has a number of advantages
We do not have to count the number of entries in the file
We can get this value using len on the list object
Remember the work required to get the maximum value from a file?

We used the following algorithm

set a variable to a very low value
for each value in the file:
    if the value is greater than the variable:
        set the variable to this new value

Once we have read the values from a file into a list we can use the max function
We can also use min to get the minimum value
We can sort the file values in ascending order using the list sort method
We can sort the values in descending order using sort and reverse
When processing a file, you should always use a for loop to create a list

Problems with Copying Objects

To make a copy of a variable holding a number we use an assignment statement
```
>>> number_1 = 5
>>> number_2 = number_1
>>> number_2
5
```
If you then change number_1 the value of number_2 remains unchanged
```
>>> number_1 = 6
>>> number_1
6
>>> number_2
5
```
This is not true for objects, such as lists
Let's say we create a list object and store it's location in the variable list_1
```
>>> list_1 = [1,2,3,4,5,6,7]
>>> list_1
[1, 2, 3, 4, 5, 6, 7, 8]
```
What happens if we copy this list variable?
You can create a new variable list_2 by assigning it the value of list_1
```
>>> list_2 = list_1
>>> list_2
[1, 2, 3, 4, 5, 6, 7, 8]
```

But when you change something in list_1 the same change appears in list_2

>>> list_1[6] = 8
>>> list_1
[1, 2, 3, 4, 5, 6, 8]
>>> list_2
[1, 2, 3, 4, 5, 6, 8]

Why?
To understand we need to look at what is happening in memory
When we first created list_1 we have this

When we copy list_1 to list_2 we have this

list_1 and list_2 point to the same object
When we make a change using list_1, it's the same as using list_2 to make the change
```
>>>  list_1[6] = 8
```

Using an assignment statement to create a new list variable does not work
It merely creates a new variable referring to the same list
This is true for all objects
Creating two variables that point to the same object is a very bad idea

Copying Lists

To copy a list you need to create a new list
This new list must have the same elements of the original list
And the elements must be in the same order

To copy a list we can use concatenation and the empty list

>>> list_1
[1, 2, 3, 4, 5, 6, 7]
>>> list_2 = [] + list_1 
>>> list_2
[1, 2, 3, 4, 5, 6, 7]

Here is the picture in memory

Now when you change one list the other remains unchanged

>>> list_1[6] = 8
>>> list_1
[1, 2, 3, 4, 5, 6, 8]
>>> list_2
[1, 2, 3, 4, 5, 6, 7]

In memory, it looks like this

You can also copy a list using a slice

>>> list_1 = [1,2,3,4,5,6,7]
>>> list_2 = list_1[0:len(list_1)]
>>> list_1[6] = 9
>>> list_1
[1, 2, 3, 4, 5, 6, 9]
>>>  list_2
[1, 2, 3, 4, 5, 6, 7]

There is a shorter way to do the same thing
When you omit the first index in a slice, Python assumes you mean 0
And when you omit the second index, Python assumes you mean the length of the list

so we can write

>>> list_1 = [1,2,3,4,5,6,7]
>>> list_2 = list_1[:]
>>> list_1[6] = 10
>>> list_1
[1, 2, 3, 4, 5, 6, 10]
>>> list_2
[1, 2, 3, 4, 5, 6, 7]

Functions That Return Lists

Functions can return objects as well as values
So we can create a function that reads in a file of numbers and returns a list

Here is the algorithm

use the filename to create a file object
create an empty list
for each line in the file:
    convert the line into a number
    add the number to the empty list
return a variable pointing to the list

This gives us the following code

#! /usr/bin/python3
# reads a text file containing integers
# and prints it

# reads a text file of integers
# and stores them in a list which is returned
def read_integers_into_list(filename):
    file = open(filename, "r")
    new_list = []
    for line in file:
        number = int(line)
        new_list.append(number)
    file.close()
    return new_list

number_list = read_integers_into_list("temperatures.txt")
print("List:", number_list)

When we run the script we get

$ ./integers_read.py 
List: [76, 67, 74, 76, 84, 69, 67, 76, 66, 71]

Functions That Return Objects

The function above returns a pointer to a list it creates
But where is the list?
When a function runs it gets it's own chunk of memory
All the function's variables live inside this memory
The variables disappear when the function ends
This brings up two questions
- Why doesn't the list disappear when the function ends?
- What's the point of returning the list variable?
The list remains after the function ends because it was not created in the function's memory space
Objects are created in the memory space for the script
Not the memory space for the function
Script memory space does not change when the function ends
When a function returns a variable it returns the value of that variable
The value of new_list is the memory address of the list
And that memory address is outside the function's memory space

Functions That Work with Lists

Functions can work on the values contained in a list
When we use such a function we give it a list variable as an argument
The function uses the memory address of the list to access the list

Here is an algorithm for a function that returns the average of a list of numbers

set an accumulator to zero
loop through the list using the list address:
    add each number to the accumulator
return the accumulator divided by the length of the list

Here is the function

def average_list(list):
    total = 0
    for index in range(len(list)):
        total += list[index]
    return total/len(list)

We can use a similar technique to create a function that count the entries above a certain number
This function will need two parameters
- A pointer to the list
- A value

The function looks like this

def entries_above(list, value):
    number_above = 0
    for index in range(len(list)):
        if list[index] > value:
            number_above += 1
    return number_above

Using these functions we can create the following script

#! /usr/bin/python3
# this script reads in daily temperatures from a file and 
# calculates the numbers of days with above average temperatures

# reads integers into a list
def read_integers_into_list(filename):
    file = open(filename, "r")
    new_list = []
    for line in file:
        number = int(line)
        new_list.append(number)
    file.close()
    return new_list

# returns the average of list of numbers
def average_list(list):
    total = 0
    for index in range(len(list)):
        total += list[index]
    return total/len(list)

# returns the number of entries in a list
# above a certain value
def entries_above(list, value):
    number_above = 0
    for index in range(len(list)):
        if list[index] > value:
            number_above += 1
    return number_above

temps = read_integers_into_list("temperatures.txt")
print("Temperature list:", temps)
average = average_list(temps)
print("Average:", average)
print("Days above average:", entries_above(temps, average))

Running this script we get

$ ./above_average_2.py 
Temperature list: [76, 67, 74, 76, 84, 69, 67, 76, 66, 71]
Average: 72.6
Days above average: 5

Functions That Change Lists

Lists are mutable
They can be changed
And we can write a function to make those changes

Here is a function that doubles all the elements in a list of numbers

>>> def double_list(list):
...    for index in range(len(list)):
...       list[index] =  2 * list[index]
... 
>>> numbers = [1,2,3,4,5]
>>> double_list(numbers)
>>> numbers
[2, 4, 6, 8, 10]

Notice that the function did not return a value
It doesn't have to
The code that called the function already has a variable pointing the the list

split Returns a List

Remember the string method split?

We used it when we were reading a file with two values on a line

for line in file:
    date, temp = line.split()
    print(date, temp)

This trick works because split returns a list

>>> line = '2017-06-01   67'
>>> fields = line.split()
>>> fields
['2017-06-01', '67']
>>> type(fields)
<class 'list'>

We can use split to read a file of dates and temperatures

And convert them into two lists

dates = []
temps = []
for line in file:
    date, temp = line.split()
    temp   = int(temp)
    dates.append(date)
    temps.append(temp)

But now we have the data for an entry contained in two lists
How can we do something that requires values from both lists?
Like finding the date with the highest temperature?
The values for each entry have the same index
As you can see from the diagram below

So we can use the following algorithm

set the variable highest_index to 0
set the variable max_temp to -500
for the index of each entry in the two lists:
    get the temperature for the entry
    if the temperature is greater than max_temp:
        set max_temp to the temperature
        set highest_index to the current index
    return the date at highest_index

Here is the function

def higest_temp_date(dates, temps):
    highest_index = 0
    max_temp      = -500
    for index in range(len(temps)):
        temp = temps[index]
        if temp > max_temp:
            max_temp = temp
            highest_index = index
    return dates[highest_index]

Intellectual Property

The fifth general ethical principles of the ACM is

Respect the work required to produce new ideas, inventions, 
creative works, and computing artifacts.

In other words, don't steal intellectual property
Intellectual property is code or documents created by a company
These assets were created at great expense and help the organization create it's product
Unscrupulous third parties would likely pay a great deal for them
IT professionals often have access to significant intellectual property
Theft of intellectual property is a crime
It is vigorously prosecuted
In some cases the intellectual product is the product
Like music and movies

Privacy

The sixth general ethical principles of the ACM is simply
```
Respect privacy
```
Technology has seeped into every part of our lives
We live with it so much that it can severely compromise our privacy
My laptop has a camera which I block with a piece of paper
Technicians have planted malware on the machines of young women that allow them to () turn on laptop cameras
Everything you say to Alexa is recorded on servers owned by Amazon
Some people use company emails to talk about intensely private matters
Working in IT you might have access to these emails

Today's Topics

Tips and Examples

Review

New Material

Homework 11

Tips and Examples

Don't Use Indexes to Print a List

Review

Empty Lists

Slices

Leaving Out Indexes in a Slice

Skipping Elements in a Slice

The in Operator

List Methods

The append Method

The pop Method

The sort Method

The reverse Method

The del Statement

The min and max Functions

New Material

Time for Computer Operations

Reading a File into a List

Advantages of Using Lists with Files

Problems with Copying Objects

Copying Lists

Functions That Return Lists

Functions That Return Objects

Functions That Work with Lists

Functions That Change Lists

split Returns a List

Intellectual Property

Privacy

Attendance

The `in` Operator

The `del` Statement