IT 117: Intermediate Scripting

IT 117: Intermediate Scripting
Class 7

Microphone

Questions

Are there any questions before I begin?

Announcement

The UMB IT Club and Boston Linux User Group will hold a Linux InstallFest on Wednesday, September 24th, from 4 to 8 in the University Hall Room 3-3540.

If you have a machine on which you would like to install Linux and would like some help in doing this, bring it the InstallFest.

Volunteers from the Boston Linux User Group with be on hand to help with the installation.

They will also help you install the Window Subsystem for Linux (WSL) on your machine or install Linux as a dual boot.

You can also bring your questions about Linux or Unix to the InstallFest.

The Boston Linux and Unix User Group counts among its members some of the most knowledgeable Linux and Unix people in the Boston area.

Graded Quiz

You can connect to Gradescope to take weekly graded quiz today during the last 15 minutes of the class.

Once you start the quiz you have 15 minutes to finish it.

You can only take this quiz today.

There is no makeup for the weekly.

Solution to Homework 2

I have posted a solution to homework 2 here.

Let's take a look.

Homework 4

I have posted homework 4 here.

It is due this coming Sunday at 11:59 PM.

Course Requirements

Collecting and Testing Assignments

All homework assignments and Class Exercises are due on the Sunday after they are assigned at 11:59 PM
I do not have the time to score this work until the following weekend
In the week after an assignment is due, I run scripts to collect homework assignments
I also run script to test Class Exercises
I do this three times a week on Monday, Wednesday and Friday
I run these scripts for the last time the following weekend
Any assignments not submitted by then receive a 0

Notifying You About Missing Work

If I cannot collect your homework assignment I will send you an email
If I cannot find your Class Exercise I will send you an email
If there is a problem with your Class Exercise I will send you an email describing the problem
This is only done in the week between the due date and the following weekend
Once I score an assignment I will send no more messages about missing work

Late Work

Once I have scored an assignment the grade book is closed
You cannot submit an "improved" copy of the script
You cannot submit a script that you did not get in on time
I simply do not have the time to rescore scripts

Tips and Examples

Help with Assignments

I will help you if you can't get your script to work
But you need to copy your code to pe15.cs.umb.edu
One of the best ways to find bugs is to insert print statements
To do this I need to make a copy of your code
I can only do this if you have copied it to pe15.cs.umb.edu

No Appointment Needed For Office Hours

You do not need an appointment for Office Hours
You just show up in my office ...
or connect to the Zoom session
You can use this time to get help with your scripts ...
or talk about other issues
And don't forget the weekend Zoom sessions ...
which I advertise with an email to all my students

Line Numbers In IDLE

Line numbers are helpful when debugging code ...
because error messages usually specify the line number ...
of the statement that caused the error
You can make the IDLE editor display line numbers
Go to the "Options" menu ...
and drag down to "Show Line Numbers"

Printing a Dictionary in Reverse Order

Let's say we created a dictionary by starting with an empty dictionary ...
and added entries one by one using literals ...

like this

>>> students = {}
>>> students["09329034"] = ("Jane", "Adams","jadamsjadams@yahoo.com")
>>> students["09929034"] = ("Alexander", "Smith","bigboy@hotmail.com")
>>> students["05431692"] = ("Christopher", "Cannon","chris@nomail.com")
>>> students["07511379"] = ("Joseph", "Malloney","jmaljoem@yahoo.com")
>>> students["04175276"] = ("James", "Reynolds","jrjim.reynolds@gmail.com")

If we print the dictionary in a for loop ...
the entries will be printed ...

in the order in which they were created

>>> for id in students:
...     print(id, students[id])
... 
09329034 ('Jane', 'Adams', 'jadamsjadams@yahoo.com')
09929034 ('Alexander', 'Smith', 'bigboy@hotmail.com')
05431692 ('Christopher', 'Cannon', 'chris@nomail.com')
07511379 ('Joseph', 'Malloney', 'jmaljoem@yahoo.com)
04175276 ('James', 'Reynolds', 'jrjim.reynolds@gmail.com')

But we can printed them sorted by their key ...
if we use the built-in function sorted
sorted takes as an argument anything that can be used in a for loop ...
and turns it into a sorted list
If we run sorted on a list of numbers
```
>>> nums = [5,8,2,6,1,9,3]
```
It will return a sorted list of the entries
```
>>> sorted(nums)
[1, 2, 3, 5, 6, 8, 9]
```
If I run it on a dictionary ...

it creates a sorted list of the dictionary keys

>>> sorted(students)
['04175276', '05431692', '07511379', '09329034', '09929034']

We can use sorted in a for loop over a dictionary

to print the entries sorted by key

>>> for id in sorted(students):
...     print(id, students[id])
... 
04175276 ('James', 'Reynolds', 'jrjim.reynolds@gmail.com')
05431692 ('Christopher', 'Cannon', 'chris@nomail.com')
07511379 ('Joseph', 'Malloney', 'jmaljoem@yahoo.co')
09329034 ('Jane', 'Adams', 'jadamsjadams@yahoo.com')
09929034 ('Alexander', 'Smith', 'bigboy@hotmail.com')

This listing show the entries in alphabetical order by student id
What about sorting the keys in reverse order?
This can be done, but it involves an extra step

We can run sorted on a dictionary to create a sorted list of keys

>>> sorted_ids = sorted(students)
>>> sorted_ids
['04175276', '05431692', '07511379', '09329034', '09929034']

Now we can use the list method reverse to reverse the sorted list

>>> sorted_ids.reverse()
>>> sorted_ids
['09929034', '09329034', '07511379', '05431692', '04175276']

And then use this list to print the dictionary in reverse sorted order

>>> for id in sorted_ids:
...     print(id, students[id])
... 
09929034 ('Alexander', 'Smith', 'bigboy@hotmail.com')
09329034 ('Jane', 'Adams', 'jadamsjadams@yahoo.com')
07511379 ('Joseph', 'Malloney', 'jmaljoem@yahoo.com')
05431692 ('Christopher', 'Cannon', 'chris@nomail.com')
04175276 ('James', 'Reynolds', 'jrjim.reynolds@gmail.com')

Review

Lists versus Dictionaries

Both lists and dictionaries are objects that hold multiple values
In a list you access a value by its index
In a dictionary, you access a value by its key
Here is one way to think of the difference
A list is a collection of values
But a dictionary is a collection of variables
A variable is a place in memory with a name and a value

The `in` And `not in` Operators

We can learn if a value is contained in a list by using the in operator

>>> numbers
[1, 2, 3, 4, 5]
>>> 5 in numbers
True
>>> 6 in numbers
False

Similarly, the not in operator tells us whether a value is not found in a list
```
>>> 5 not in numbers
False
>>> 6 not in numbers
True
```
The two words not and in together make a single operator
These two operators also work with dictionaries but they only look at keys ...

not values

>>> words_integers
{'three': 3, 'five': 5, 'two': 2, 'one': 1, 'four': 4}
>>> "three" in words_integers
True
>>> 3 in words_integers
False

Deleting Elements from A Dictionary

To delete an entry in a dictionary you use a del statement

The general format of such a statement is

del DICTIONARY_NAME[KEY]

where DICTIONARY_NAME is a variable that points to the dictionary object

>>> words_integers
{'three': 3, 'five': 5, 'two': 2, 'one': 1, 'four': 4}
>>> del words_integers["five"]
>>> words_integers
{'three': 3, 'two': 2, 'one': 1, 'four': 4}

If you use a key that does not exist, a KeyError exception is raised

del words_integers["six"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'six'

Getting the Number of Elements in a Dictionary

The len function can be used with a dictionary

It returns the number of entries in the dictionary

>>> email_addresses
{'Chris': 'chrisk@yahoo.com', 'Alan': 'alanh@gmail.com'}
>>> len(email_addresses)
2
>>> words_integers
{'five': 5, 'one': 1, 'six': 6, 'two': 2, 'four': 4, 'three': 3}
>>> len(words_integers)
6

Attendance

New Material

Values in a Boolean Context

Scripting languages try to make it easier to write code
One example of this is how they can turn numbers and strings ...
into boolean values
They can do this without your having to call the conversion function bool ...
in certain situations
The if and while statements both need boolean expressions
These expressions often use relational operators ...
and logical operators
- and
- or
- not
So you might find the following boolean expression in these constructs
```
value == 0
answer == ""
num != 0
```
But scripting languages give you another choice
You can use a string or a number in place of a boolean expression
If the string is the empty string ...

Python will treat that value as false

>>> value = ""
>>> if value:
...     print("True")
... else:
...     print("False")
... 
False

If the value of the string is anything else ...

Python will treat it as true

>>> value = "x"
>>> if value:
...     print("True")
... else:
...     print("False")
... 
True

Python will do something similar with numbers

To show how this works, let's create a test function

>>> def true_or_false(value):
...     if value:
...         print("True")
...     else:
...         print("False")
...

Any expression whose value is zero will be treated as false
```
>>> true_or_false(0)
False
>>> true_or_false(0.0)
False
```
Any expression whose value is not zero will be thought of as true
```
>>> true_or_false(1)
True
>>> true_or_false(4)
True
```

This even works for negative values

>>> true_or_false(-1)
True
>>> true_or_false(-100)
True

This even works with objects

Any variable that points to an object will be treated as true

>>> file = open("students.txt", "r")
>>> file
<_io.TextIOWrapper name="students.txt" mode="r" encoding="UTF-8">
>>> true_or_false(file)
True

Can an object variable ever be false?
The value of an object variable is a location in memory
Python has a special value called None
An object variable whose value is None points to nothing
It's like zero for objects
So an expression whose value is None is false
```
>>> true_or_false(None)
False
```

Keys Are Immutable

There is is one major constraint on dictionaries
The data type of a key entry in a dictionary must be immutable
In other words they must be things that cannot be changed
The following data types can be used as keys
- Strings
- Integers
- Floating point
- Tuples

This means we can construct the following dictionaries

>>> strings_integers = {"one":1, "two":2, "three":3}
>>> strings_integers["three"]
3
>>> integers_strings = {1:"one", 2:"two", 3:"three"}
>>> integers_strings[3]
"three"
>>> floats_strings   = {1.0:"one", 2.0:"two", 3.0:"three"}
>>> floats_strings[3.0]
"three"
>>> tuples_integers  = {(1,1):1, (2,2):2, (3,3):3}
>>> tuples_integers[(3,3)]
3
>>>

But we can't create a dictionary using lists as keys

>>> lists_integers   = {[1,1]:1, [2,2]:2, [3,3]:3}
Traceback (most recent call last):
  File <stdin>, line 1, in <module>
TypeError: unhashable type: "list"

Why this restriction?
Think back to our discussion of hash tables
The hash function is run on the key ...
to determine the slot in the table in which to store an entry
But if the key is changed ...
how are we going to find it's slot again?

Values in a Dictionary

The keys in a dictionary must be immutable

But the values in a dictionary can be anything

>>> jumble = {1:1, 2:2.0, 3:"three", 4:(4,4), 5:[5]}
>>> for key in jumble:
...    print(jumble[key])
... 
1
2.0
three
(4, 4)
[5]

In this example all the keys are integers
But all the values are of different data types
In this case all the keys were of the same data type
That is the most common situation

But it is not a requirement

>>> jumble_2 = {1:1, 2.0:2, "3":3} 
>>> for key in jumble_2:
...     print(jumble_2[key])
... 
1
2
3

Dictionary Methods

Dictionaries are objects so they have have methods

Here are some of the more useful dictionary methods

Method	Description
clear()	Clears the contents of a dictionary.
get(key)	Gets the value associated with a specified key. If the key is not found, the method does not raise an exception. Instead, it returns a default value. You can give the default value as a second argument.
pop(key)	Returns the value associated with a specified key and removes that key-value pair from the dictionary. If the key is not found, the method returns a default value. You can give the default value as a second argument.
popitem()	Returns a randomly selected key-value pair as a tuple from the dictionary and removes that key-value pair from the dictionary.
items()	Returns all the keys in a dictionary and their associated values.
keys()	Returns all the keys in a dictionary.
values()	Returns all the values in the dictionary.

The clear method

The clear method removes all entries from a directory
Whenever we call a method we must use dot notation
That means we first type the name of the variable that points to the object ...
followed by a dot, . ...
followed by the name of the method and its arguments

So if I have a dictionary

>>> quiz_scores
{"Mary Jones": [98, 95, 93, 92], "Tim Tyler": [88, 81, 79, 85], "John Smith": [100, 90, 85]}

I can remove all entries like this

>>> quiz_scores.clear()
>>> quiz_scores
{}

Now you might be thinking there is another way to get the same result
And you would be right
To clear the dictionary I could simply write
```
>>> quiz_scores = {}
```
The object pointed to by quiz_scores would not have any entries
But it would be a different object
Every time we use an assignment statement to create a dictionary ...
we create a new object
This object has a particular address in RAM ...

which we can learn using the id function

>>> quiz_scores
{"John Smith": [100, 90, 85], "Tim Tyler": [88, 81, 79], "Mary Jones": [98, 95, 93]}
>>> id(quiz_scores)
139691305684744

The picture in memory looks like this

When I use the clear method the object stays the same

But what it holds changes

>>> quiz_scores.clear()
>>> id(quiz_scores)
140083427949384
>>> quiz_scores
{}

The picture in memory now looks like this

If instead of using the clear method ...
I assigned the variable to an empty directory
```
quiz_scores = {}
```
This would create a new object as we can see by running id on quiz_scores
```
>>> id(quiz_scores)
140083428105992
```
quiz_scores now points to a different address
Because it is pointing to a new object
This means the old object is still in RAM taking up space
But we cannot use it because no variable point to it

The original object is still in RAM
This is why you should use clear

The get Method

We usually get the value for a key using the bracket operator, [ ]
```
>>> scores = quiz_scores["Mary Jones"]
>>> scores
[98, 95, 93]
```
But if we use a key not found in the dictionary ...

Python will throw an exception

>>> scores = quiz_scores["Mary Jonez"]
Traceback (most recent call last):
  File <stdin>, line 1, in <module>
KeyError: "Mary Jonez"

The get method works just like the [ ] operator ...

when we use a valid key

>>> scores = quiz_scores.get("Mary Jones")
>>> scores
[98, 95, 93]

But when we use an invalid key get does not throw an exception

Instead it return the default value None

>>> scores = quiz_scores.get("Mary Jonez")
>>> print(scores)
None

Python will treat this value as False in an if statement

If we got the key from a for loop there would be no reason to use get

>>> for key in quiz_scores:
...    print(key, ":", quiz_scores[key])
... 
Tim Tyler : [88, 81, 79]
Mary Jones : [98, 95, 93]
John Smith : [100, 90, 85]

But what if we created a function to retrieve the value for a key?

There we should use get

>>> def get_score(key):
...    score = quiz_scores.get(key)
...    if score:
...       return score
...    else:
...       print(key, "is not a valid key")
... 
>>>

If the call to get works we simply return the value
```
>>> get_score("Mary Jones")
[98, 95, 93]
```
But if we use an invalid key we can print an error message
```
>>> get_score("Mary Jonez")
Mary Jonez is not a valid key
```
If you want the default value to be something other than None...
you can supply it as a second argument.

The pop Method

The pop method takes a key as an argument
It returns the value associated with that key ..

and removes the key-value pair from the dictionary

>>> quiz_scores
{"Mary Jones": [98, 95, 93], "Tim Tyler": [88, 81, 79], "John Smith": [100, 90, 85]}
>>> quiz_scores.pop("Mary Jones")
[98, 95, 93]
>>> quiz_scores
{"Tim Tyler": [88, 81, 79], "John Smith": [100, 90, 85]}

If the key use is not in the dictionary ...
pop() will not throw an exception
Instead it will return the default value, None
To change the default value ...
supply it as a second argument to pop()

The popitem Method

The popitem method takes no argument

It returns a randomly chosen key-value pair as a tuple

>>> quiz_scores
{"Tim Tyler": [88, 81, 79], "John Smith": [100, 90, 85]}
>>> quiz_scores.popitem()
("Tim Tyler", [88, 81, 79])

It also removes that pair from the dictionary

>>> quiz_scores
{"John Smith": [100, 90, 85]}

The items Method

The items method returns all key-value pairs
In Python 2, these pairs are returned as a list of tuples

The first element of the tuple is the key and the second is the value

>>> names_usernames
{"Mary Jones": "mjones", "John Smith": "wombat", "Tim Tyler": "ttyler"}
>>> items = names_usernames.items()
>>> items
[('Mary Jones', 'mjones'), ('John Smith', 'wombat'), ('Tim Tyler', 'ttyler')]

But if we try this in Python 3 we get a slightly different result

>>> names_usernames
{"Mary Jones": "mjones", "Tim Tyler": "ttyler", "John Smith": "wombat"}
>>> items = names_usernames.items()
>>> items
dict_items([("Mary Jones", "mjones"), ("Tim Tyler", "ttyler"), ("John Smith", "wombat")])

What is is a dict_items?
dict_items are one member of a family of data types called view objects
A view object is like a window into the object on which is was created
A view object is dynamic
As the object changes, the view object will show these changes
This does not happen with the list created by the items method in Python 2

In Python 2 a call to items gives us a list

>>> items = names_usernames.items()
>>> type(items)
<type "list">

But if we change the original dictionary by adding a new key-value pair

>>> names_usernames["Fred Farnsworth"] = "fredster"
>>> names_usernames
{"Fred Farnsworth": "fredster", "Mary Jones": "mjones", "John Smith": "wombat", "Tim Tyler": "ttyler"}

The list we created by the first call to items does not change

>>> items
[("Mary Jones", "mjones"), ("John Smith", "wombat"), ("Tim Tyler", "ttyler")]

In Python 3 however the dict-items view object will show the new entry

>>> names_usernames
{"Mary Jones": "mjones", "Tim Tyler": "ttyler", "John Smith": "wombat"}
>>> names_usernames["Fred Farnsworth"] = "fredster"
>>> items
dict_items([("Mary Jones", "mjones"), ("Fred Farnsworth", "fredster"), ("Tim Tyler", "ttyler"), ("John Smith", "wombat")])

You cannot use indexing on a view object

>>> listing[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: "dict_items" object does not support indexing

But you can access the individual tuples

for pair in items:
...    print(pair)
... 
("Mary Jones", "mjones")
("John Smith", "wombat")
("Tim Tyler", "ttyler")
("Fred Farnsworth", "fredster")
>>> for pair in listing:
...    print("key:", pair[0], "value", pair[1])
... 
key: Mary Jones value mjones
key: John Smith value wombat
key: Tim Tyler value ttyler
key: Fred Farnsworth value fredster

The keys Method

The keys method returns all the keys of a dictionary

In Python 2 the method returns a list

>>> names_usernames.keys()
["Fred Farnsworth", "Mary Jones", "John Smith", "Tim Tyler"]

But in Python 3 method returns a view object called dict_keys

>>> names_usernames.keys()
dict_keys(["Mary Jones", "John Smith", "Tim Tyler", "Fred Farnsworth"])

dict_keys is also view object
It will change as the dictionary changes

The values Method

The values method returns all values as a list in Python 2

>>> names_usernames.values()
["fredster", "mjones", "wombat", "ttyler"]

But is Python 3 returns dict_values view object

>>> names_usernames.values()
dict_values(["mjones", "wombat", "ttyler", "fredster"])

A dict_values is also a view object

Real World Dictionary Example

All my Class Notes files have the same structure
There are links at the top of the page ...
that refer to different sections below
Usually when I create a new section ...
I also create a link for it at the top
But sometimes I make a mistake and the links do not work
I used to check all the links by hand
But that got to be tedious
So I developed a script to do the checking for me
This script opens the Class Notes file and goes through it line by line
As it does, it looks for two things
- Links to sections
- Section headers
It does this using regular expressions ...
a feature we will discuss in a few weeks
Two functions play an important role in this script
One uses regular expressions to return links
The other returns section headers
Actually this is a simplification
Both scripts return two values
- The href value
- The link label
I then store this data is two different dictionaries
One for the section links ...
and one for the section headers
In each of these dictionaries the key is the href value ...
and the value is the label

The first three link entries at the top of this page look like this

<li class="flat"><a href="#collecting_testing_assignments">Collecting and Testing Assignments</a></li>
<li class="flat"><a href="#notifying_missing_work">Notifying You About Missing Work</a></li>
<li class="flat"><a href="#late_work">Late Work</a>

The dictionary entries for these three lines are

"collecting_testing_assignments": "Collecting and Testing Assignments"
"notifying_missing_work": "Notifying You About Missing Work"
"late_work": "Late Work"

The first three section headers are

<h4 class="topic" id="collecting_testing_assignments">Collecting and Testing Assignments</h4>
<h4 class="topic" id="notifying_missing_work">Notifying You About Missing Work</h4>
<h4 class="topic" id="late_work">Late Work</h4>

Their dictionary entries are

"collecting_testing_assignments": "Collecting and Testing Assignments"
"notifying_missing_work": "Notifying You About Missing Work"
"late_work": "Late Work"

After the two dictionaries have been populated ...
the script checks the number of entries in each
Each dictionary should have the same length
If they don't the script prints an error message
Then the script goes through the links dictionary and finds the label for each href
It then gets the label for the same href in the sections dictionary
If it finds an entry, then the href tags in the link and section are the same
This is the most important test
But I also check that the labels are the same
This script has saved me hours of work
The script is internal_links_check.py
It is not on pe15.cs.umb.edu
But I can show it to you on my laptop

Course Requirements

Tips and Examples

Review

New Material

Questions

Announcement

Graded Quiz

Solution to Homework 2

Homework 4

Course Requirements

Collecting and Testing Assignments

Notifying You About Missing Work

Late Work

Tips and Examples

Help with Assignments

No Appointment Needed For Office Hours

Line Numbers In IDLE

Printing a Dictionary in Reverse Order

Review

Lists versus Dictionaries

The in And not in Operators

Deleting Elements from A Dictionary

Getting the Number of Elements in a Dictionary

Attendance

New Material

Values in a Boolean Context

Keys Are Immutable

Values in a Dictionary

Dictionary Methods

The clear method

The get Method

The pop Method

The popitem Method

The items Method

The keys Method

The values Method

Real World Dictionary Example

Class Exercise

Class Quiz

The `in` And `not in` Operators