IT 116: Introduction to Scripting

IT 116: Introduction to Scripting
Class 27

Today's Topics

Review

Strings Are Sequences
Accessing Characters With Indexing
Negative Numbers as Indexes
Using a for Loop with Strings
Strings Are Immutable
Garbage Collection
String Slicing

New Material

Final Exam

The final exam will be given on Thursday, December 19th from 11:30 - 2:30.

The exam will be given in this room.

It will consist of questions like those on the quizzes along with questions asking you to write short segments of Python code.

60% of the points on this exam will consist of questions from the Ungraded Class Quizzes.

Although classes do not end until Friday, December 13th, I do not have materials for a 29th class, and do not have the time to prepare them.

Instead, on day when the last class would normally be held, Thursday, December 12th I will have office hours from Noon to 6 PM.

You may come to my office at any time during this period, for help of any sort, without making an appointment.

Class 28, on Tuesday, December 10th, will be a review session.

You will only be responsible for the material in Class Notes 28 and the review for the Mid-term, which you will find here.

Although the time alloted for the exam is 3 hours, I would expect that most of you would not need that much time.

The final is a closed book exam.

To prevent cheating, certain rules will be enforced during the exam.

Course Evaluation

I will distribute the course evaluation in the last class.

The evaluations are very important.

They help me to realize what I have done right and what I have done wrong.

Please take the time to fill them out with your honest opinion of the course and my teaching.

Review

Strings Are Sequences

There are many different data types in Python
Some hold only one value
- Integer
- Float
- Boolean
Other data types hold more than one value
A sequence is a family of data types that contain many different values stored one right after the other
The last chapter in our textbook dealt with two types of sequences
- Lists
- Tuples
Strings are sequences where the elements are individual characters
Sequences are objects and have methods that operate on the data they contain
Each sequence data type has its own set of methods

Accessing Characters With Indexing

The characters in a string have a definite order, so we can use indexing to get a particular character
If we have a string
```
>>> team = "Red Sox"
```

we can use the [ ] operator to get a specific character
Inside the brackets we can use a integer literal
```
>>> team[0]
'R'
```

Negative Numbers as Indexes

You can't use any number as an index
It must be an integer

And it cannot be longer than the length of the list minus 1

>>> letters = 'abcde'
>>> len(letters)
5
>>> letters[4]
'e'
>>> letters[5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: string index out of range

You can use negative numbers as indexes
A negative index is added to the length of the string when using [ ]
In practices this means that -1 is the index of the last character
-2 is the index of the second from last

and so on

>>> letters[-1]
'e'
>>> letters[-2]
'd'
>>> letters[-3]
'c'
>>> letters[-4]
'b'
>>> letters[-5]
'a'
>>> letters[-len(letters)]
'a'

Negative index values can be used with any sequence

They can be used with lists

>>> numbers = [1, 2, 3, 4, 5]
>>> numbers[-1]
5

and with tuples

>>> values = (1, 2, 3, 4, 5)
>>> values[-1]
5

Using a `for` Loop with Strings

You can also access the individual characters in a string using a for loop
```
>>> for ch in team:
...   print(ch)
... 
R
e
d

S
o
x
```
This allows us to write some very interesting functions that work with strings

We can write a function to count the number of times a certain character occurs in a string

>>> def character_count(string, char):
...    count = 0
...    for ch in string:
...        if ch == char:
...            count += 1
...    return count
... 
>>> character_count("Mississippi", "i")
4
>>> character_count("Mississippi", "s")
4
>>> character_count("Mississippi", "p")
2

We can also use a for loop to reverse a string

>>> def string_reverse(string):
...     new_string = ""
...     for ch in string:
...         new_string =  ch + new_string
...     return new_string
... 
>>> string_reverse('Mississippi')
'ippississiM'
>>> string_reverse('radar')
'radar'

Strings Are Immutable

Strings are immutable
You cannot change a string

But you can use one string to create another

>>> string_1 = "Red"
>>> string_1 = string_1 + " Sox"
>>> string_1
'Red Sox'

In the first statement above I created the string "Red"
The string variable string_1 points to this new string
Here is how things look in memory

When I wrote
```
string_1 += " Sox"
```
two things are happening
First, I created a new string consisting of the characters in the first string and " Sox"
Second, the object variable string_1 now points to this new string
Nothing points to the old string "Red"
But that string has not gone away
Here is the new memory situation

Garbage Collection

In the example above we have two strings in memory
- "Red"
- "Red Sox"

I could created still another new string from the second string

>>> string_1
'Red Sox'
>>> string_1 = "Go " + string_1
>>> string_1
'Go Red Sox'

Now I have three string in memory

But I can only work with the newest one
If I did this in a endlessly in a script you might think it would eventually run out of memory
And crash
This is a type of bug called a "memory leak"
Of course memory cannot leak away
But if you keep creating object you will eventually run out of memory the script can use
This does not happen in Python and most scripting languages
It also does not happen in Java
In these languages the interpreter does something called garbage collection
I works like this
The interpreter keeps a table of all objects in memory
And the variables that point to them
Every so often the interpreter looks at this table and deletes objects that have no variable pointing to them
So programs written in Python will never have a memory leak

String Slicing

A string that is contained inside another string is called a substring
Many computer languages have functions that can be used to extract a substring
But Python gives us an easier way
We can use slices
Slicing works with all sequence objects including strings
So if we have the following string
```
>>> team = "Boston Red Sox"
```
we can use a slice to extract the city substring from the larger string
```
>>> team[0:6]
'Boston'
```

New Material

Testing Strings

I create a scoring sheet for each Class Exercise that looks like this

IT 116 Scores Class Exercise 24
======================================================
__  Elvis Barbosa
__  Eddie Beazer
__  George Boamah
__  Vincent Chen
...

When a student completes the exercise I replace the string "__" with "100"
Outside of class I run a Python script that finds all entries with "__" before the name
Then it runs the test script on that student's script
If the students exercise is not OK, I replace the "__" with other strings
- "EE" if the script is not executable
- "HB" if the script lacks a working hashbang line
- "SS" if the script has syntax errors
- "XX" if there is some other problem with the script
- "II" if the script is incomplete
Another script reads the Class Exercise scoring sheet and sends the student the appropriate email message
All of this involves testing strings
Python has a number of features to make this easier

Testing Strings with `in` and `not in`

Operators are special characters that perform an action using one or more values
Usually that action creates a new value
This makes them seem like functions but they operate in a totally different manner
To understand why is beyond the scope of this course
The easiest way to test for a string is to use the in operator
We first met the in operator when studying lists

The in operator returns True when a value is an element in a list or tuple

>>> numbers = [1, 2, 3, 4, 5]
>>> 2 in numbers
True
>>> "foo" in numbers
False

You can use the in operator on a string

>>> team = "Red Sox"
>>> "R" in team
True
>>> "z" in team
False

But the in operator has another trick up its sleeve when used with strings
The first operand does not have to be a single character
It can be any string

Not just a single character

>>> "Red" in team
True
>>> "Reds" in team
False

Sometimes you want to know when one string is not contained in another

In this case you can use not in

>>> "Red" not in team
False
>>> "Reds" not in team
True

The operator is two words - not in
Though not is a logical operator it is not used that way here
We can use not as a logical operator with the in operator

But if we do it must come before first operand for in

>>> not "Red" in team
False
>>> not "Reds" in team
True

not in is a single operator that returns the opposite of in

Using the `in` Operator on a File

As I said above, I have a Python script that I run on the scores list for each Class Exercise
Often I cannot find a student's script or there some problem with it
In this case the script has to send the student an email
The script needs to know what message to send to each student
The particular message is determined by characters where the grade should be
The messages need to say
- The script is missing, if "__" comes before the name
- The script is not executable, if "EE" comes before the name
- The script is has a syntax error, if "SS" comes before the name
- The script does not work properly, if "XX" comes before the name
In each line of the scores file I look for these special marks using the in operator
I create an empty list for each of these problems
Then I add the user name of any student who belongs to that list

This is done with in a for loop

for line in file:
    if "__" in line:
        add to no script list
    elif "EE" in line:
        add to not executable list
    elif "SS" in line:
        add to syntax error list
    elif "XX" in line:
        add to error list

The words in brown are not Python statements
They are descriptions of what the code block does
The script uses features of Python that are covered in IT 117
That is why I will not show it here

String Methods

Strings are objects and have methods that operate on their characters
There are three general types of string methods
- Methods that test the values of strings
- Methods that return a new string different from the original
- Methods that search and replace

String Testing Methods

The string testing methods are all boolean methods
You can think of them as answering a question about the string

All their names begin with "is"

Method	Description
isalnum()	Returns True if the string contains only alphabetic letters or digits and has a length of at least 1. Returns False otherwise.
isalpha()	Returns True if the string contains only alphabetic letters and has a length of at least 1. Returns False otherwise.
isdigit()	Returns True if the string contains only digits and has a length of at least 1. Returns False otherwise.
islower()	Returns True if all of the alphabetic letters in the string are lowercase and has a length of at least 1. Returns False otherwise.
isspace()	Returns True if the string contains only whitespace characters and has a length of at least 1. Returns False otherwise.
isupper()	Returns True if all of the alphabetic letters in the string are uppercase and has a length of at least 1. Returns False otherwise.

isalnum()

isalnum() returns True when all the characters are letters or digits

>>> first_name
'Glenn'
>>> first_name.isalnum()
True
>>> digits
'0123456789'
>>> digits.isalnum()
True
>>> user
'alpha26'
>>> user.isalnum()
True

But it will return False if the string contains a space

>>> team = 'Red Sox'
>>> team
'Red Sox'
>>> team.isalnum()
False

If it contains some other character

>>> jumble
'adbcd1234!@#$%'
>>> jumble.isalnum()
False

Or if the string is empty
```
>>> empty
''
>>> empty.isalnum()
False
```

isalpha()

isalpha() returns True when all the characters are letters

>>> first_name
'Glenn'
>>> first_name.isalpha()
True
>>> digits
'0123456789'
>>> digits.isalpha()
False

Notice that all the characters must be letters

>>> user
'alpha26'
>>> user.isalpha()
False
>>> team
'Red Sox'
>>> team.isalpha()
False

It returns False when the string is empty
```
>>> empty
''
>>> empty.isalpha()
False
```

isdigit()

isdigit() returns True when all the characters are digits

 >>> digits
'0123456789'
>>> digits.isdigit()
True
>>> user
'alpha26'
>>> user.isdigit()
False

It returns False when the string is empty
```
>>> empty
''
>>> empty.isdigit()
False
```

islower()

islower() returns True when all the alphabetic the characters are lower case

>>> noun
'watch'
>>> noun.islower()
True
>>> first_name
'Glenn'
>>>  first_name.islower()
False

Notice that all the string characters do not have to be letters

Only that all the letters must be lower case

>>> user
'alpha26'
>>> user.islower()
True
>>> jumble
'adbcd1234!@#$%'
>>> jumble.islower()
True

islower will return False if there are no letters in the string

>>> digits
'0123456789'
>>> digits.islower()
False
>>> empty
''
>>> empty.islower()
False

isupper()

isdigit() returns True when all the alphabetic the characters are upper case

>>> league
'MLB'
>>> league.isupper()
True
>>> standard
'HTML 5'
>>> standard.isupper()
True
>>> team
'Red Sox'
>>> team.isupper()
False

It also returns False when no letters are present in the string

>>> digits
'0123456789'
>>> digits.isupper()
False
>>> empty
''
>>> empty.isupper()
False

isspace()

isspace() returns True when all the characters are whitespace
The whitespace characters are
- Space - " "
- Tab - "\t"
- Newline - "\n"
```
>>> whitespace
' \t\n'
>>> whitespace.isspace()
True
```

All the characters must be whitespace

>>> team
'Red Sox'
>>> team.isspace()
False

And the string cannot be empty
```
>>> empty
''
>>> empty.isspace()
False
```

String Methods That Produce New Strings

Strings are immutable, they can never change

But string do have methods that produce a new string using an existing string

Method	Description
lower()	Returns a copy of the string with all alphabetic letters converted to lowercase. Any character that is already lowercase, or is not an alphabetic letter, is unchanged.
upper()	Returns a copy of the string with all alphabetic letters converted to uppercase. Any character that is already uppercase, or is not an alphabetic letter, is unchanged.
lstrip()	Returns a copy of the string with leading characters removed. When run without an argument, it removes leading whitespace characters. When run with an argument, removes those characters.
rstrip()	Returns a copy of the string with all trailing characters removed. When run without an argument, it removes tailing whitespace characters. When run with an argument, removes those characters.
strip()	Returns a copy of the string with leading and trailing characters removed. When run without an argument, it removes leading and tailing whitespace characters. When run with an argument, removes those characters.

lower()

lower() creates a new string with all letters converted to lower case

>>> team
'Red Sox'
>>> team.lower()
'red sox'
>>> name
'Glenn'
>>> name.lower()
'glenn'
>>>  league
'NFL'
>>> league.lower()
'nfl'

lower() does not affect any characters that are not letters

>>> jumble
'adbcd1234!@#$%'
>>> jumble.lower()
'adbcd1234!@#$%'
>>> user
'alpha26
>>> user.lower()
'alpha26'

upper()

upper() creates a new string with all letters converted to uppercase

>>> name
'Glenn'
>>> name.upper()
'GLENN'
>>> noun
'leaf'
>>> noun.upper()
'LEAF'

upper() does affect other characters

>>> jumble
'adbcd1234!@#$%'
>>> jumble.upper()
'ADBCD1234!@#$%'

lstrip()

lstrip() creates a new string with leading characters removed

If you run lstrip without an argument it removes leading whitespace

>>> str1
'   foo   '
>>> str1.lstrip()
'foo   '
>>> str2
'\t\tfoo\t\t'
>>> str2.lstrip()
'foo\t\t'
>>> str3
'\n\nfoo\n\n'
>>> str3.lstrip()
'foo\n\n'

If you give lstrip an argument, it removes those characters from the front of the string

The argument can be a single character

>>> str4
'---foo---'
>>> str4.lstrip("-")
'foo---'

Or more than one character

>>> str5 = '- - - foo- - - '
>>> str5.lstrip('- ')
'foo- - - '

rstrip()

rstrip()creates a new string with trailing characters removed

If you run lrstrip without an argument it removes trailing whitespace

>>> str1
'   foo   '
>>> str1.rstrip()
'   foo'
>>> str2
'\t\tfoo\t\t'
>>> str2.rstrip()
'\t\tfoo'
>>> str3
'\n\nfoo\n\n'
>>> str3.rstrip()
'\n\nfoo'

If you give rstrip an argument, it removes those characters from the front of the string

The argument can be a single character

>>> str4
'---foo---'
>>> str4.rstrip("-")
'---foo'

Or more than one character

>>> str5
'- - - foo- - - '
>>> str5.rstrip("- ")
'- - - foo'

strip()

strip is like combination of both rstrip and lstrip

If you run strip without an argument it removes leading and trailing whitespace

>>> str1
'   foo   '
>>> str1.strip()
'foo'
>>> str2
'\t\tfoo\t\t'
>>> str2.strip()
'foo'
>>> str3
'\n\nfoo\n\n'
>>> str3.strip()
'foo'

If you give strip an argument, it removes those characters from both the front and end of the string

The argument can be a single character

>>> str4
'---foo---'
>>> str4.strip("-")
'foo'

Or more than one character

>>> str5
'- - - foo- - - '
>>> str5.strip("- ")
'foo'

String Search and Replace Methods

Another family of string methods cans be used to search for a substring

Or to create a new string which replaces a substring with some other string

Method	Description
endswith(substring)	The method returns True if the string ends with the substring.
startswith(substring)	The method returns True if the string starts with the substring.
find(substring)	The method returns the lowest index in the string where substring is found. If string is not found, the method returns −1.
replace(old_substring, new_substring)	The method returns a copy of the string with all instances of old_ubstring replaced by new_substring.

endswith(substring)

endswith(substring) returns True the substring is at the end of the string

>>> team = 'Red Sox'
>>> team.endswith('Red')
False
>>> team.endswith('Sox')
True
>>> team.endswith(' Sox')
True

startswith(substring)

startswith(substring) returns True the substring is at the start of the string

>>> team
'Red Sox'
>>> team.startswith('Sox')
False
>>> team.startswith('Red')
True

find(substring)

find(substring) takes a string as its argument and returns an index

The index is the position of the first place where the substring appears

>>> team
'Red Sox'
>>> team.find('Red')
0
>>> team.find(' ')
3
>>> team.find('Sox')
4
>>> nonsense
'foo bar bletch foo'
gt;>> nonsense.find('foo')
0

If the string is not found in the string find returns -1
```
>>> team.find('Yankees')
-1
```

replace(old_substring, new_substring)

replace has two parameters, both strings
It creates a new string where every occurence of the first substring is replaced by the second substring
```
>>> nonsense
'foo bar bletch foo'
>>> nonsense.replace('foo', 'bam')
'bam bar bletch bam'
```
If the first substring is not found in the original string nothing is replaced
And the method returns the original string
```
>>> team.replace('Six','Socks')
'Red Sox'
```

Confidentiality

The last of the ACM general ethical principles is
```
Honor confidentiality
```
If you work in IT you often have access to private information
Things like salary, medical data and the results of disciplinary proceedings
This must not be shared with anyone outside the organization
Even within an organization only certain employees need access to this data
It is your duty to make sure that this information is only used for valid reasons
You should not discuss anything you learn with friends or family
As a teacher, I am subject to the Family Educational Rights and Privacy Act (FERPA)
This law spells out my responsibility to preserve the confidentiality of my students
I am not even permitted to reveal a student's grade to their parents without the student's permission
Unless they are under 18
I recently discovered a student bringing a hidden smart phone into a final exam
I failed the student and documented it to the appropriate authorities
I believe this student to be a serial lier who will cheat again
But I am not allow to mention the student's name to other instructors
In some hospitals there are signs in the elevators warning about discussing patients medical conditions
If you have access to privileged information, you have a duty to keep it confidential

Contract for the Web

Tim Berners-Lee is the man who created HTML
HTML stands for Hypertext Markup Language
It is the format that is used to create web pages
So you might say that Tim Berners-Lee is the man who invented the World Wide Web
He created the web to allow the free exchange of information
In the early days of the web it looked like something that would change the world for the better
Now it is debatable whether the web does more harm than good
Bigots of various sorts use the web and its platforms to spread hatred
Foreign governments spread lies to undermine our democracy
Crackpot ideas proliferate
- The earth is flat
- Vaccines cause autism
All this has been hard for Tim Berner-Lee to watch
Rescue the web he has launches an initiative called the Contract for the Web
It is the result of many months of consultation with many people in government, academia, and not-profits
Here are it's fundamental principles
- Everyone can connect to the Internet
- Keep all of the Internet available, all of the time
- Respect and protect people's fundamental online privacy and data rights
- Make the Internet affordable and accessible to everyone
- Respect and protect people's privacy and personal data to build online trust
- Develop technologies that support the best in humanity and challenge the worst
- Be creators and collaborators on the Web
- Build strong communities that civil discourse and human dignity
- Fight for the Web
Tim has asked all of us to endorse the Contract
I have done this on the Contract Web site
I would urge all of you to do so too
Rescuing the web from it's most toxic elements will take some time
But all of us must do what we can to make the web a better place
If you are not part of the solution, you are part of the problem

Today's Topics

Review

New Material

Final Exam

Course Evaluation

Review

Strings Are Sequences

Accessing Characters With Indexing

Negative Numbers as Indexes

Using a for Loop with Strings

Strings Are Immutable

Garbage Collection

String Slicing

New Material

Testing Strings

Testing Strings with in and not in

Using the in Operator on a File

String Methods

String Testing Methods

isalnum()

isalpha()

isdigit()

islower()

isupper()

isspace()

String Methods That Produce New Strings

lower()

upper()

lstrip()

rstrip()

strip()

String Search and Replace Methods

endswith(substring)

startswith(substring)

find(substring)

replace(old_substring, new_substring)

Confidentiality

Contract for the Web

Attendance

Using a `for` Loop with Strings

Testing Strings with `in` and `not in`

Using the `in` Operator on a File