Graph examples

• Google Maps, of course
  • Which actually started off using MapQuest data
  • But we'll call it Google Maps data
• Others?

Airline Routes

Flowcharts

Pre-requisite Diagrams

Graphs

• G = (V, E)
• V are the vertices; E are the edges
• Edges are of the form (v, w), where v, w ∈ V
  • ordered pair: directed graph or digraph
  • unordered pair: undirected graph

How big are these graphs?

• Airline routes: Openflights, as of January 2012, has 59,036 routes between 3,209 airports (source)
• Google maps: information is hard to find
  • So this is a not-so-educated guess
  • There are probably 50 million intersections (vertices) in the US
  • Assume each one connects to four others
  • That's 4 * 50 million = 200 million edges

Terminology

• A weight or cost can be associated with each edge
• w is adjacent to v iff (v, w) ∈ E
• path: sequence of vertices w₁, w₂, w₃, ..., w_n such that (w_i, w_i+1) ∈ E for 1 ≤ i < n
• length of a path: number of edges in the path
• simple path: all vertices are distinct

How to weight a graph?

• For Google maps?
• For airline routes?

More terminology

• cycle:
  • directed graph: path of length ≥ 1 such that w₁ = w_n
  • undirected graph: same, except all edges are distinct
• connected: there is a path from every vertex to every other vertex
• loop: (v, v) ∈ E
• complete graph: there is an edge between every pair of vertices

Digraph terminology

• directed acyclic graph: no cycles; often called a "DAG"
• strongly connected: there is a path from every vertex to every other vertex
• weakly connected: the underlying undirected graph is connected

Representation: Adjacency Matrix

$A[u][v] = \left\{ \begin{array}{l l} weight & \quad \text{if ($u$,$v$) $\in$ $E$}\\ 0 & \quad \text{if ($u$,$v$) $\notin$ $E$}\\ \end{array} \right.$

1 2 3 4 1 2 3 4

Representation: Adjacency List

Representation in the real world

• Two types of representation
  • Adjacency matrix
  • Adjacency list
• How does Google maps probably store it?
• How do airline routes probably store it?

Topological Sort

• Given a directed acyclic graph, construct an ordering of the vertices such that if there is a path from v_i to v_j, then v_j appears after v_i in the ordering
  • The result is a linear list of vertices
• indegree of v: number of edges (u, v) -- meaning the number of incoming edges

Topological Sort

One valid topological sort is: v1, v6, v8, v3, v2, v7, v4, v5

What is the topological sort?

What is the topological sort?

This is already topologically sorted!

Topological sort

void Graph::topsort() {
  Vertex v, w;
  for (int counter=0; counter < NUM_VERTICES; 
                                      counter++) {
    v = findNewVertexOfInDegreeZero();
    if (v == NOT_A_VERTEX)
      throw CycleFound();
    v.topologicalNum = counter;
    for each w adjacent to v
      w.indegree--;
  }
}

• What's the big-Theta running time?
• Observation: The only new (eligible) vertices with indegree 0 are the ones adjacent to the vertex just processed

Topological sort

void Graph::topsort() {
  Queue q(NUM_VERTICES);
  int counter = 0;
  Vertex v, w;

  q.makeEmpty(); // initialize the queue
  for each vertex v
    if (v.indegree == 0)
      q.enqueue(v);
  while (!q.isEmpty()) {
    v = q.dequeue(); // get vertex of indegree 0
    v.topologicalNum = ++counter;
    for each w adjacent to v: // insert eligible verts
      if (--w.indegree == 0)
        q.enqueue(w);
  }
  if (counter != NUM_VERTICES)
    throw CycleFound();
}

Another Topological Sort Example

Why do we care about shortest paths?

• The obvious answers:
  • Map routing (car navigation systems, Google Maps, flights)
  • 6 degrees of separation
• But what else?
  • Internet routing
  • Puzzle answers (Rubik's cube)

Three types of algorithms

• Single pair
• Single source
• All pairs
  • We won't see this in this course
• And a variant that we'll see later:
  • Travelling salesperson

Shortest Path Algorithms

• This version is called the "single-source" shortest path
• Given a graph $G = (V, E)$ and a single distinguished vertex s, find the shortest weighted path from s to every other vertex in G

The weighted path length of $v_1, v_2, \ldots , v_n$:

$\sum_{i=1}^{n-1}c_{i,i+1}$ where $c_{i,i+1}$ is the cost of edge $(v_i,v_{i+1})$

Unweighted Shortest Path

• Special case of the weighted problem: all weights are 1
• Solution: breadth-first search; similar to level-order traversal for trees

Unweighted Shortest Path

void Graph::unweighted (Vertex s) {
  Queue q(NUM_VERTICES);
  Vertex v, w;
  q.enqueue(s);
  s.dist = 0;

  while (!q.isEmpty()) {
    v = q.dequeue();
    for each w adjacent to v
      // each edge examined at most once,
      // if adjacency lists are used
      if (w.dist == INFINITY) {
        w.dist = v.dist + 1;
        w.path = v;
        q.enqueue(w); // each vertex added at most once
      }
    }
  }

What is the big-Theta running time?

Weighted Shortest Path

• We assume no negative weight edges
• Dijkstra's algorithm: uses similar ideas as the unweighted case
• Greedy algorithms: do what seems to be best at every decision point

Dijkstra's algorithm

• Initialize each vertex's distance as infinity
• Start at a given vertex s
  • Update s's distance to be 0
• Repeat
  • Pick the next unknown vertex with the shortest distance to be the next v
    • If no more vertices are unknown, terminate loop
  • Mark v as known
  • For each edge from v to adjacent unknown vertices w
    • If the total distance to w is less than the current distance to w
      • Update w's distance and the path to w

Dijkstra's Algorithm

V Known? Dist Path v0 v1 v2 v3 v4 v5 v6

Dijkstra's algorithm

void Graph::dijkstra(Vertex s) {
  Vertex v,w;
  s.dist = 0;

  while (there exist unknown vertices, find the 
        unknown v with the smallest distance) {
    v.known = true;

    for each w adjacent to v
      if (!w.known)
        if (v.dist + Cost_VW < w.dist) {
          w.dist = v.dist + Cost_VW;
          w.path = v;
        }
  }
}

Another Dijkstra's Algorithm Example

V Known? Dist Path v1 v2 v3 v4 v5 v6

This is the same graph as in the Wikipedia article on Dijkstra's algorithm

Analysis

• How long does it take to find the smallest unknown distance?
  • Simple scan using an array: Θ(v)
• Total running time:
  • Using a simple scan: Θ(v²+e) = Θ(v²)
• Optimizations?
  • Use adjacency lists and heaps
  • Assuming that the graph is connected (i.e. e > v-1), then the running time decreases to Θ(e log v)
    • Although we won't see how to do that in this course

Negative Cost Edges?

• Perhaps the graph weights are the amount of fuel expended
  • Positive means fuel was used
  • And passing by a fuel station is a refueling, which is a negative cost edge
• Dijkstra's algorithm does not work for negative cost edges
  • Others do (such as the Bellman-Ford algorithm), but are much less efficient
• What about negative cost cycles?

Shortest Path Example Problem

From the ICPC Mid-Atlantic Regionals, 2009

More on shortest path

• We studied finding the shortest path from a single vertex to every vertex
• But what about just 1 destination?
• Solution: do the same algorithm, but stop when the destination enters the set S
• Thus, the running time is the same!

How would you drive to Seattle?

And what constitutes a "highway"?

The Eisenhower Interstate System

A Google Maps Screenshot

GPS shortest path algorithm

(Google Maps likely uses something much more complicated)

• Assume you are starting on a "side road"
• Transition to a "main road"
• Transition to a "highway"
• Get as close as you can to your destination via the highway system
• Transition to a "main road", and get as close as you can to your destination
• Transition to a "side road", and go to your destination

Travelling Salesperson Problem (TSP)

• Given a number of cities and the costs of traveling from any city to any other city, what is the least-cost round-trip route that visits each city exactly once and then returns to the starting city?
• Really important problem for:
  • UPS, Federal Express, USPS
  • Any transport delivery system
  • Cost = distance because more fuel is used
• We'll assume the graph is fully connected

Easy

Hard

Really Hard

Analysis

• Hamiltonian path: a path in a connected graph that visits each vertex exactly once
  • Hamiltonian cycle: a Hamiltonian path that ends where it started
• The traveling salesperson problem is thus to find the least weight Hamiltonian path (cycle) in a connected, weighted graph
• The size of the solution space is 1/2 (n-1)!
  • Which means it's a Θ(n!) algorithm
  • That's exponential (specifically, 2ⁿ < *n*! < *n*ⁿ for large n)
  • For 10 cities: 181,440 possible cycles
  • For 20 cities: 6 * 10¹⁶ possible cycles
  • For 21 cities, there are more possibilities than can be stored in a 64-bit unsigned long

More Analysis

• This problem is NP-complete
  • Meaning there is no known efficient solution
  • Just to try every possible path
• But there are ways to get a somewhat efficient solution (a heuristic)
  • It just might not be the most efficient path
• What's the (usually) least expensive way to get between two US cities?
  • And is that significantly slower than the "best" algorithm?

The Record

• In April 2006, a computer cluster computed a path of 85,900 cities visited in 136 CPU years (source)
  • About 3-6 months of "wall time"
• 85,900! = 9.61 * 10^386,526
• Assume you can compute 1 million paths each second
  • That would take only 3.04 * 10^386,516 years!
    • (the exponent lowered by 10)
  • They used acceleration techniques, obviously...

Lab 11

• The parts of lab 11:
  • Pre-lab: implement a topological sort
  • In-lab: implement a brute-force traveling salesperson problem
    • Using locations in Tolkien's Middle Earth
  • Post-lab: shortest path solution to a puzzle problem

Spanning Tree

• Suppose you are going to build a transport system:
  • Set of cities
  • Roads, rail lines, air corridors connecting cities
• Trains, buses or aircraft between cities
• Which links do you actually use?
  • Cannot use a complete graph
  • Passengers can change at connection points
• Want to minimize number of links used
• Any solution is a tree

Spanning Tree (almost...)

Spanning Tree

• A spanning tree of a graph G is a subgraph of G that contains every vertex of G and is a tree
• Any connected graph has a spanning tree
• Any two spanning trees of a graph have the same number of nodes
• Construct a spanning tree:
  • Start with the graph
  • Remove an edge from each cycle
  • What remains has the same set of vertices but is a tree

Spanning Trees

Original graph:

Possible spanning trees:

Minimal Spanning Tree

• Spanning trees are simple
• But suppose edges have weights!
  • "Cost" associated with the edge
  • Miles for a transport link, for example
• Spanning trees each have a different total weight
• Minimal-weight spanning tree: spanning tree with the minimal total weight

Minimum Spanning Trees

• Given a connected and undirected graph G = (V,E), find a graph G' = (V,E') such that:
  • E' is a subset of E
  • |E'| = |V| - 1
  • G' is connected
  • $\sum_{(u,v) \in E'} c_{uv}$ is minimal
• G' is then a minimal spanning tree
• Applications: wiring a house, cable TV lines, power grids, Internet connections

Generic Minimum Spanning Tree Algorithm

• KnownVertices <- {}
• while KnownVertices does not form a spanning tree, loop:
  • find edge (u,v) that is "safe" for KnownVertices
  • KnownVertices <- KnownVertices U {(u,v)}
• end loop

 
But how to find a "safe" edge?

Prim's algorithm

Idea: Grow a tree by adding an edge to the "known" vertices from the "unknown" vertices. Pick the edge with the smallest weight.

Prim's Algorithm for MST

• Pick one node as the root,
• Incrementally add edges that connect a "new" vertex to the tree.
• Pick the edge (u,v) where:
  • u is in the tree, v is not, AND
  • where the edge weight is the smallest of all edges (where u is in the tree and v is not)

Prim's MST Algorithm

Edges: (v1,v2), (v1,v4), (v3,v4), (v4,v7), (v5,v7), (v6,v7)

Analysis

• Running time: Same as Dijkstra's: Θ(e log v)
• Correctness:
  • Suppose we have a partially built tree that we know is contained in some minimum spanning tree T
  • Let (u,v) ∈ E, where u is "known" and v is "unknown" and has minimal cost
  • Then there is a MST T' that contains the partially built tree and (u,v) that has as low a cost as T

Kruskal's MST Algorithm

Idea: Grow a forest out of edges that do not create a cycle. Pick an edge with the smallest weight.

Kruskal's MST Algorithm

Edges: (v1,v4), (v6,v7), (v1,v2), (v3,v4), (v4,v7), (v5,v7)

Kruskal code

void Graph::kruskal() {
  int edgesAccepted = 0;
  DisjSet s(NUM_VERTICES);

  while (edgesAccepted < NUM_VERTICES - 1) {
    // The next line has |E| heap ops
    e = smallest weight edge not deleted yet;
    // edge e = (u, v)
    uset = s.find(u);    // |E| finds
    vset = s.find(v);    // |E| finds
    if (uset != vset) {
      edgesAccepted++;
      s.unionSets(uset, vset);    // |V| unions
    }
  }
}

When optimized, it has the same running time as Prim's and Dijkstra's: Θ(e log v)