Lists

• Operations
  • find
  • insert
  • remove
  • findKth
• Implementations
  • Array (vector)
  • Linked list

Lists

Stacks

• List with data handled last-in first-out
• Operations:
  • push
  • pop
  • top
• Implementations
  • Array (vector)
  • Linked list

Stacks

Queues

• First-in first-out list
• Operations:
  • enqueue
  • dequeue
• Implementations
  • Array (vector)
  • Linked lists

Queues

Trees

• Goal is Θ(log n) runtime for most operations
  • Binary search trees
  • AVL Trees
  • Red-black trees
  • Splay trees

Trees

Is There Anything Faster?

• Fastest possible search using binary comparison: Θ(log n)
  • Rephrased: binary comparison searches are Ω(log n)
• We can do better: (almost) constant (Θ(1)) is possible with hash tables
• Hash tables (lookup table)
  • Standard set of operations: find, insert, delete
  • No ordering property!
    • Thus, no findMin or findMax

Key-value pairs

• Hash tables store key-value pairs
  • Each value has a specific key associated with it
  • Keys and values need not be the same type!
• Examples
  • Definitions: "set", "v.tr. 1. To put in a specified position..."
  • Uva e-mail redirects: "aaron@", "asb2t@cms .virginia.edu"
  • Anything that can be stored in a tree
    • Userid / IDnum pairs
    • Userid / lots_of_info_about_them_in_an_object pairs

Hash Tables

Hash table fixed size array of some size, usually a prime number Should be larger than the number of elements Given a key space: hash function hash(k) →

hash table   0 1 2   ...   tablesize‑1

Hash function

• A hash function takes in a "thing"...
  • string, int, object, etc.
  • and returns an unsigned integer value
    • which is then mod'ed by the size of the hash table to yield a spot within the bounds of the hash table array
• Three required properties
  • Must be deterministic
    • Meaning it must return the same value each time for the same "thing"
  • Must be fast
  • Must be evenly distributed
  • Technically, only the first is required for correctness, but the other two are required for fast running times

Hash functions KLA

• I'm going hash all of you into 10 buckets (0-9) by your birthday
  • (you are welcome to make up another birthday, as long as you are consistent)
• The hash functions:
  • By the decade of your birth year
    • hash(birthday) = (year/10) % 10
  • By the last digit of your birth year
    • hash(birthday) = year % 10
  • By the last digit of your birth month
    • hash(birthday) = month % 10
  • By the last digit of your birth day
    • hash(birthday) = day % 10

Keys

How can we hash the keys if the keys can be anything? Best one binary comparison can do is eliminate one half of the elements Θ(log n) We want Θ(1) The keys must be bits, so we can do better!

"Hello"   ['H','i',\0]   3.14   'x'   0x42381a   01001010

Objects   Arrays   Primitive types   Addresses   bits

Lookup Table

This can work, unless the key space is sparse, or we don't know the keys ahead of time. But it's slow to look up a value in a table!

Example

Table size issues...

• Why not just have a table of size 100
  • And map them directly to the location corresponding to their key?
• We assume that the key space is too large
  • Example: mapping social security numbers for students at UVa
    • There are not 999,999,999 students at UVa, even if taken across all time
• Do you see why find max and find min are not easy?
  • We have not preserved any ordering info

Another Example

Hash Table

• Hash function: hash: key → [0, m-1]
  • Really to any unsigned integer, which is then mod'ed by m, the table size
• Here, hash(key) = firstletter(key)
   

Hash Functions

• Required properties described earlier
  • Must be deterministic
  • Must be fast
  • Must be evenly distributed
    • This implies avoiding of collisions
• A perfect hash function has:
  • No blanks (i.e., no empty cells)
  • No collisions

Sample String Hash Functions

• Key space: strings
• A string s is made up of characters s_i
• $s = s_0s_1s_2s_3\ldots s_{k-1}$

1. $hash(s) = s_0 \mod table\_size$
    
2. $hash(s) = \left( \sum_{i=0}^{k-1}s_i \right) \mod table\_size$
    
3. $hash(s) = \left( \sum_{i=0}^{k-1}s_i*37^i \right) \mod table\_size$
    

Hash function notes

• They should always return an unsigned int
  • Otherwise your program will be trying to find a negative array index
• Integer overflow is fine, as long as it overflows deterministically
  • Meaning the same way each time
  • This will especially be true with the last of the string hash functions presented on the previous slide

Collision Resolution

• Collision: when two keys map to the same location in the hash table
• Two primary ways to resolve collisions:
  1. Separate Chaining (make each spot in the table a 'bucket' or a collection)
  2. Open Addressing, of which there are 3 types:
     • Linear probing
     • Quadratic probing
     • Double hashing

Separate Chaining

• An animation of this can be found here
  • Although it is really called "separate chaining", that site calls it "open hashing"

Separate Chaining

Analysis of find

• Definition: The load factor, λ, of a hash table is the ratio of the number of elements divided by the table size
• For separate chaining, λ is the average number of elements in a bucket
  • Average time on unsuccessful find: λ
    • Average length of a list at hash(k)
  • Average time on successful find: 1 + (λ/2)
    • One node, plus half the average length of a list (not including the item)

Load factor

• How big should we make the hash table?
  • Well, we want "constant" time for find and insert...
• Possible sizes for hash table with separate chaining
  • λ = 1
    • Make hash table be the number of elements expected; average bucket size is 1
    • Also make it a prime number
  • λ = 0.75
    • Java's Hashtable but can be set to another value
    • Table will always be bigger than number of elements
      • This reduces the chance of a collision!
    • Good trade-off between memory use and time
  • λ = 0.5
    • Uses more memory, but fewer collisions

Separate Chaining: find()

• Note that we now have to keep each key in the chain, as well as the value!
• What is the worst case?
  • Hint: Wikipedia is wrong on this one...
  • In the worst case, every key could hash to the same spot!
    • As nobody uses anything other than a linked list as the secondary data structure, this means it will be a Θ(n) algorithm to perform a find!
• What is the "hopeful" case?

What data structure to use for the buckets?

• AVL & red-black trees will give the best running time
  • But that's a lot of overhead!
• Vectors are easier and simpler, but take up a lot of space
  • All those extra, unused, cells
  • Don't ever use vectors for this!
• Linked lists are quick and easy, and take up very little extra space
  • That's Θ(n)!
  • Still faster in practice than trees due to having a very small number of items in the bucket

Requirements for "Hopeful" Case

• Our ideal hash function and hash table:
  • Function hash(k) is well distributed for key space
    • For a randomly selected k ∈ K,
    • probability(hash(k) = i) = 1/table_size
  • Size of table scales linearly with number of elements
    • Expected bucket size is Θ(num_elements / table_size)
• Finding a good hash function can be tough

Separate chaining insert is Θ(1)

• In an unsorted linked list, you can just put it on the front
• So all inserts into a separate chained hash table, that uses linked lists, are actually in constant time
  • If you were to sort the linked list, that would be linear time
  • And finds (and thus deletes) are still linear time

Open addressing

• An animation of all three open addressing strategies can be found here
  • Although it is really called "open addressing", that site calls it "closed hashing"

Saving Memory

Three Types of Probing Strategies

• The three types:
  • Linear
  • Quadratic
  • Double hashing
• The general idea with all of them is that, if a spot is occupied, to 'probe', or try, other spots in the table to use
  • How we determine where else to probe depends on which strategy we are using

Linear Probing

Linear Probing

• With all open addressing schemes, we examine ('probe') the cells in the order:
  • p₀(k), p₁(k), p₂(k), ...
  • where: p_i(k) = (hash(k) + f(i)) mod table_size
• With linear probing, f(i) = i
  • After searching spot hash(k) in the array, look in:
    • hash(k) + 1
    • hash(k) + 2
    • hash(k) + 3
    • etc.

Problems with Linear Probing

• Primary clustering
  • Large blocks of occupied cells
  • As table fills, increased number of attempts required to solve collision
    • And thus slower lookup times
  • "Holes" when an element is removed
    • We'll see how to solve this later
  • When to stop looking?

Quadratic Probing

• With all open addressing schemes, we examine ('probe') the cells in the order:
  • p₀(k), p₁(k), p₂(k), ...
  • where: p_i(k) = (hash(k) + f(i)) mod table_size
• With quadratic probing, f(i) = i²
  • After searching spot hash(k) in the array, look in:
    • hash(k) + 1
    • hash(k) + 4
    • hash(k) + 9
    • etc.

Quadratic Probing

Double Hashing

• With all open addressing schemes, we examine ('probe') the cells in the order:
  • p₀(k), p₁(k), p₂(k), ...
  • where: p_i(k) = (hash(k) + f(i)) mod table_size
• With double hashing, f(i) = i * hash₂(k)
  • Which means we have to define a secondary hash function!
  • After searching spot hash(k) in the array, look in:
    • hash(k) + 1 * hash₂(k)
    • hash(k) + 2 * hash₂(k)
    • hash(k) + 3 * hash₂(k)
    • etc.

Double Hashing

Double Hashing Thrashing

Table size must be prime!

• The table size must always be a prime number
  • It will prevent the thrashing from the previous slide
    • Thrashing will only occur when the double hash value is a factor of the table size
    • The only factors of a prime number p are 1 and p
      • 1 is effectively linear probing, which is fine
      • p will mod to zero, which is an invalid return value for a secondary hash function
  • It will provide better distribution of the hash keys into the table
    • Less clustering, etc.
• A prime number table size does not remove the need for a good hash function!

Rehashing

• Problem: when the table gets too full, running time for operations increases
• Solution: create a bigger table and hash all the items from the original table into the new table
  • The position in a table is dependent on the table size, which means we have to rehash each value
  • This means we have to re-compute the hash value for each element, and insert it into the new table!

Rehashing

• When to rehash?
  • When half full (λ = 0.5)
  • When mostly full (λ = 0.75)
    • Java's hashtable does this by default
  • When an insertion fails
  • Some other threshold
• Cost of rehashing
  • Let's assume that the hash function computation is constant
  • We have to do n inserts, and if each key hashes to the same spot, then it will be a Θ(n²) operation!
    • Although it is not likely to ever run that slow

Removing an element

• How to handle this?
• You could:
  • Rehash upon each delete, which is very expensive
  • Put in a 'placeholder' or 'sentinel' value
    • But the table gets filled with these rather fast
    • Perhaps rehashing after a certain number of deletes
  • Disallow deletes entirely; you can do this for lab 6
• Hash tables are not an ideal data structure if you need to perform a lot of deletions

Hashing: MD5

• MD5: Message Digest 5
• Given a string (or file contents, etc.) generate a 128-bit hash
  • 2¹²⁸ = 3.4*10³⁸ (coincidentally, this is is also the maximum finite value of a float)
  • Typically an MD5 is always written in hex: 16e28b7986fd74f65b061de89dc8b78e
• This could then be used as the key
  • Obviously having to mod it by the table size
• (Was) good for checking if a download completed successfully

Can you reverse an MD5 hash?

• Technically, no
  • A 129-bit file has 2¹²⁹ possibilities, and if you were to hash each one, it would go into 2¹²⁸ buckets
  • By the pigeonhole principle, there would be at least one hash value (pigeonhole) with multiple keys (pigeons), and you don't know which one
    • In reality, many (and eventually all) would have multiple keys
• But if a password is stored by it's MD5 hash...
  • ... then there are enough online hash libraries that you can find at least one password that hashes to that value
  • Try Googling for 3858f62230ac3c915f300c664312c63f
• Plus there are lots of weaknesses in MD5...

More hashing: SHA

• MD5 has been "broken"
  • One can generate two files that have the same hash; this is called a collision attack
    • In fact, I have the students do something similar when I teach Defense Against the Dark Arts (albeit with a weaker hashing algorithm)
  • So it is useless for any security-related purposes
• SHA (Secure Hash Algorithm) is a family of algorithms that (the more recent ones) are much more secure
  • Same overall idea: it generates a hash value up to 512 bits
  • SHA-1 has been broken also, but more recent SHAs are secure