Algorithms (F2008)

If we have an NP-Complete problem X, then we can show that a new problem Y is NP-Complete by

• Showing that Y is in NP
• Showing that X reduces to Y.

But where did this first NP-Complete problem come from, if there aren’t turtles all the way down…

The answer is the Cook-Levin Theorem, one of the most profound results in complexity theory:

CIRCUIT-SAT is NP-Complete

We’ll handwave a proof of this, and look at some examples to understand how the proof works.

Material for this lecture is drawn from Chapter 8 of the textbook, and archives of Ronald Reagan’s speeches on the Cold War.

Thus far we’ve been looking at techniques for designing efficient algorithms for problems. We’ll now stare failure in the face, and try to understand when we might fail at doing this, and why. In order to do so, we’ll reason about entire classes of problems at a time, rather than about a single problem. The central question of study is the (in)famous question: Is P = NP ?

We’ll consider two applications of max flows: image segmentation (Chapter 7.10), and analyzing playoff scenarios in league sports (NBA, NFL, MLB, etc) (Chapter 7.12)

We looked at the problem of computing a maximum matching in a bipartite graph, and showed that a simple max flow formulation yields the optimal matching. As an aside, I asked you to consider the following strategy:

Take any edge that doesn’t intersect the set of edges chosen thus far.

How badly might this do ?

We also saw that Hall’s theorem, a structural way of characterizing the existence of a perfect matching in a bipartite graph, can be proved using flow principles.

Finally, we examined some variants of the max flow problem that can all be reduced to the standard formulation. These were:

• circulations: having multiple sources and sinks, each with their own specified demands
• Lower bounds on edge capacities (like lower limits on speeds on a highway)
• Vertex capacities: limiting the amount of flow through a vertex (for example, this can arise when you have a warehouse that’s storing shipments, but can only store a limited amount of material)

The first two sections are taken from Section 7.5 of the text, and the last part is taken from Section 7.7.

The material for this lecture is drawn partially from the textbook: however, I’ll be following Jeff Erickson’s notes more closely.

We introduced the idea of a flow network, and the concept of a flow from a source to a sink. We then asked the question: “How can we maximize the flow from source to sink”, and developed the Ford-Fulkerson augmenting-path-based algorithm for this problem. A deep relationship to cuts in a graph proved the optimality of this method.

The material is drawn from Sections 7.1 and 7.2 of the textbook.

All you needed to know, right here.

In this lecture, we will cover two dynamic programming-based methods for computing shortest paths in graphs. The first one is called the Bellman-Ford algorithm and is used to find all shortest paths from a single source. The second one is the Floyd-Warshall algorithm for finding all shortest paths between all pairs of vertices in a graph.

In both cases, the graphs under consideration have edge weights that could be negative, which complicates matters. However, as long as there are no cycles of negative weight, we can return meaningful answers.

The Bellman-Ford algorithm is in Chapter 6 of the textbook. The Floyd-Warshall algorithm is NOT in the textbook. It can be found in ‘Introduction to algorithms’ by Cormen, Leiserson, Rivest and Stein (Chapter 25). A brief but complete description of the algorithm is presented in Wikipedia.

Although the material in this lecture comes from the book, I found that it was more effective to write my own notes. They are linked here: please skim them before class.

I particularly want to make sure that you at least read

• The section on complex numbers
• The section describing the FFT

This is tough material (in my experience) and will take some effort to get through.

In today’s lecture we’ll cover the basics of divide and conquer, looking at

• Mergesort
• Integer multiplication
• Closest pair (among points in the plane)

All of this can be found in the textbook, in chapter 5.

Divide and conquer is intimately connected with recursion. Paradoxically though, the ability to implement a divide and conquer algorithm has more to do with the ability to combine than the ability to recurse. In the three examples, we’ll see that the merge step is the start of the process by which we design a D&C algorithm, and the merge step gets more and more complicated as we go along.

After notes:

• One point I omitted to make: once we construct the grid of side length D/2 in the merge step of the closest pair algorithm, we can make the strong claim that AT MOST one point lies in each of the grid cells. this is because any cell lies either wholly on the left or wholly on the right. If two points occupied a single grid cell, then they’d be closer than D apart (check this!) and that would contradict the claim that D was the closest pair distance on that side. This is important, because if a cell could contain more than one point, then we’d need to recurse once again inside this cell, messing up the analysis.
• Some people raised the question of whether this algorithm works in three dimensions (i.e find a splitting plane, recurse, and then do a merge as before, with cubes instead of squares). In fact, this can be done: the trick is that the running time involves numbers that are exponential in the dimension (4*4*4 = 64 in 3D, and so on), but the n log n part stays the same.